I worked on machine learning infrastructure in the last AI cycle, which was marked by stepwise advancements in deep learning. Some lessons learned and other observations: