Came across this piece talking about the current #machinelearning status by Mark Saroufim. Some views are a bit controversial but good points nonetheless.

Machine Learning: The Great Stagnation

I’ll summarize the key points below: 🧵👇

1. ML Researchers — Supposed to be risk-taking and less commercial oriented so ground-breaking progress can be made. Rather, ML Researchers found ways to not taking any risk but getting good pay through FANNG, media, YouTube, etc., and SOTA chasing.
2. Math is overrated in Deep Learning. Matrix multiplication is mostly what you need and auto grad removes the real needs for manual gradient calculation. Be real now.
3. The empiricism tendency of DL formed a feedback loop, where people/company has more computing power to do many experiments in parallel wins, thus gaining more resources to do even more experiments. Google Brain, OpenAI, DeepMind.
4. Transformers are everywhere and so popular
5. Graduate Student Descent. Again, more people to do experiments leads to faster breakthroughs. Also, cargo-culting configs, loss functions, frameworks make it possible but less well-thought-out.