Trends in ML @ICML 2022

ICML took place in mid-July in hybrid mode, two months after ICLR 2022. In our analysis of ICLR, we pointed out the spread of Language Models and self-supervision.
** Watch Trends in ML @ ICLR 2022 Part 1 here - • Trends in Machine Lear...
** Watch Trends in ML @ ICLR 2022 Part 2 here - • Trends in Machine Lear...
Things are not that different at ICML. Knowledge from papers quickly passes to tutorials and workshops that make it more accessible to practitioners. It is worth mentioning that Language Models (LMs) are invading Reinforcement Learning, acting as keepers of knowledge and very effective planners.
We observed that the Research goal in LM can be consolidated into cost reduction, which is equivalent to scale and low carbon emissions. There are multiple angles that researchers attack this problem, sparsity, hardware acceleration, vector databases, 1-bit representations, etc. We are many decades away from having a GPT-3 on a chip unless we see a new breakthrough.
The idea of ML as a service (MLaS has been around for many years, but it became popular and promising with the rise of LMs. We noted a couple of papers discussing how to use MLaS cost-effectively.
The trend of Biomedical AI continues to grow (more papers than ICLR and 2 workshops) fueled by Covid-19 and the breakthroughs of Alphafold.
At last we can not leave unnoticed the return of Vowpal Wabbit, probably the oldest open source ML project, long before the deep learning era, which is still relevant and very useful!
We hope you enjoy this summary. You can find the link to our slide presentation here - bit.ly/3dnBMPY

[@cristovaoiglesias523]

Nice overview.