Arxiv_mode
Variational inference (VI) is the problem of approximating a complicated distribution with a distribution which is simpler to sample. A major hinder for VI is mode collapse: the phenomenon where the model concentrates on a few modes of the target distribution during training, despite being expressive enough to cover all modes. This dynamical phenomenon is poorly understood theoretically. In our recent pre-print we identify different mechanisms driving mode collapse in a Gaussian mixture model setting, showing that they share some similarities with what is found in normalising flows.
Enjoy Reading This Article?
Here are some more articles you might like to read next: