Disentangled Representations

Definition of disentangled representation with group theory

Disentangled representations can be defined with the help of Group Theory as shown by I Higgins et al. 2019. Representation Z is disentangled with respect to the decomposition $G = G_1 \times G_2 ... G_n$ if:

1. There is a group action $h: G \times Z → Z$
2. The map f:W→Z is equivariant between the actions on W and Z
3. There is decomposition of $Z= Z_1 \times Z_2 ... Z_N$ such that $Z_i$ is affected only by $G_i$.

Challenging common assumptions of unsupervised learning

Some important results from F. Locatello et al. (2019)
1. Theoretically proved that unsupervised learning of disentangled representation is impossible without inductive biases in both models and data.
2. Cannot validate the assumption that disentanglement is useful for downstream tasks, but useful for fairness and interpretability.

Disentanglement with Biological Constraints

References

1. Towards a Definition of Disentangled Representations
2. Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations