Profiling CPython at Instagram

Profiling CPython at Instagram

6 years ago
Anonymous $oIHRkISgaL

https://instagram-engineering.com/profiling-cpython-at-instagram-89d4cbeeb898

Instagram employs Python in one of the world’s largest settings, using it to implement the “business logic” needed to serve 800 million monthly active users. We use the reference implementation of Python, known as CPython, as the runtime used to execute our code. As we’ve grown, the number of machines required toserve our users has become a significant contributor to our growing infrastructure needs. These machines are CPU bound, and as a result, we keep a close eye on the efficiency of the code we write and deploy, and have focused on building tools to detect and diagnose performance regressions. This continues to serve us well, but the projected growth of our web tier led us to investigate sources of inefficiency in the runtime itself.

To determine what we needed to optimize and which techniques we should use, we had to figure out: