Diederik P. Kingma
Person on 3 sites
Diederik "Durk" Kingma is a Dutch computer scientist working on machine learning and generative models.
-
6 facts on wiqqi (opens wiqqi.app)
- Work: Research on large-scale machine learning at Anthropic (from 2024) (opens wiqqi.app)
- Earlier: OpenAI founding team, led the Algorithms team (2015 to 2018); Google Brain / DeepMind (2018 to 2024) (opens wiqqi.app)
- Created: Variational Autoencoder (VAE), Adam optimizer, Glow, Variational Diffusion Models (opens wiqqi.app)
-
2 channels on ppll (opens ppll.app)
-
x.com: @dpkingma (opens x.com)
“Obviously I strongly support Anthropic's position here.”
-
Mastodon: @dpkingma@sigmoid.social (opens sigmoid.social)
““Adam can converge without any modification on update rules” https://arxiv.org/abs/2208.09632 Proves that (vanilla) Adam is theoretically justified without any modification. Presented at NeurIPS'22. By Yushun Zhang,…”
-
x.com: @dpkingma (opens x.com)
-
4 beliefs on korrents (opens korrents.com)
-
Stochastic optimization can be done well using only adaptive estimates of the gradient's lower-order moments. (opens korrents.com)
“We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments.”
-
An optimizer that is invariant to rescaling the gradients and cheap in memory is what makes very large models practical to train. (opens korrents.com)
“The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of…”
-
The same optimizer should handle objectives that move under it and gradients that are noisy or sparse. (opens korrents.com)
“The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients.”
-
Stochastic optimization can be done well using only adaptive estimates of the gradient's lower-order moments. (opens korrents.com)