Parts of the machine learning stack of the 2010s are quickly drifting into irrelevancy.

What stayed the same? The developer tools and libraries — TensorFlow, PyTorch. Although even these have started to be consolidated into bigger building blocks (e.g., HuggingFace’s transformers library). Generally, these libraries have absorbed new developments fairly easily, and the primitives that they provide (abstracting the underlying hardware, matrix multiplication, etc.) are generic enough.

Notebooks are still the place for data exploration (and even exploratory training). However, now they are much more likely to be remote (Google Colab or hosted on a cluster). Models continued to grow larger.

Of course, it’s not just the ML stack. Software half-life seems to be decreasing rapidly across the board — even for infrastructure.