Inside the Faster CPython Team at Microsoft

The last few years have seen Python rise to prominence. TIOBE and IEEE both put the language in the top spot. The versatility of the language often comes with critiques around Python’s performance. Performance-sensitive packages are often written in other languages, most notably C. In recent years, there has been a concerted effort to increase Python’s CPU performance.

In 2020, Core Developer Mark Shannon drafted an Implementation plan for speeding up CPython (the most common implementation) by five times. This plan proposed a 4-stage process that, as Python’s creator Guido van Rossum says, “was an effort that was too much for one volunteer to accomplish”.

“Right from the start, my thought was well, we should try to see if Microsoft can hire Mark and a small team of people to support him.” In the previous year Van Rossum came out of retirement and joined Microsoft as a Distinguished Engineer. “It was an important effort and it was too much for one person.” Microsoft was open to the idea and a team of 6 engineers, including Van Rossum were established. That team has assisted other core developers in acting on this plan. Python 3.11 has already seen speedups of 10-60% in some areas of the language.

Specialized Knowledge and Collaboration

“If we have people with different backgrounds and different skill sets, as a team we’ll make better decisions and we’ll be more open to unexpected opportunities.” This was what Van Rossum told me they were looking for when building this team. CPython uses C code to interpret the Python Language. While Python’s success comes in its simplicity for the user, the underlying technology requires more care. L Pereira, one of the team members, gained many of these skills working on classic computers from the 1970s and ’80s as a hobby. This hobby gave her insight in working on .NET. Now she sets her focus on Python. “To do even the most basic things [on older systems], you have to optimize your programs. [Developers] have to really think about how memory is going to be laid out, how you’re going to do this and that. This kind of thing is very useful for Python or any other language interpreter that people are going to use for real world use cases.”

The team consists of folks with some very specialized knowledge. Eric Snow has been working on sub-interpreters for the last several years. Irit Katriel has been working with exceptions, she implemented the new Exception groups and except* features in 3.11. Brandt Bucher helped create structural pattern matching and other features debuted in Python 3.10.

There’s also the people side of it. Michael Droettboom not only brings to the team knowledge of the scientific Python space, but also management principles. He has to manage the expectations of the team as well as the Python community. “How do we make sure we’re showing up to the Python community as helpful, not driving an agenda or any of those kinds of things…Because what we are doing is kind of risky. There has probably been a higher frequency of unintended consequences and bugs that have come about because we are changing how the core works. It’s really important that the team supports the community to keep quality up.”

Handling bug triaging is something that Bucher knows about. He was a member of the CPython triage team just 2 years ago. The team coordinates with many of the language’s core devs. Droettboom says he hopes they don’t feel “overloaded” and works to take off “some of that burden”. “We’ve kind of created a little more burden on those people. So it’s only fair that we also ask what we can do to help.”

The Consistency of Team-Based Contribution

Perhaps the biggest benefits of a team dedicated to working on CPython are consistent, collective effort and constant communication.

Most of the team’s members had been volunteers. Occasionally, employers granted members some time to focus on CPython. Every contributor that made the switch from part-time to full-time contribution mentioned being able to get deeper into their work on the language.

For example, Snow talked about how hard context switching can be. He was originally working with Python at Microsoft as an Engineer and had 20% work time to focus on CPython. “Working on CPython one day a week, there’s a lot of thrashing involved… as time goes by…life got busier so it was harder and harder to find my own time to spend on this.”

“In my previous job they gave me a lot of time to contribute to Python… but I had other responsibilities as well,” says Katriel. “It was a great opportunity to join a team like this and be able to focus full time on Python.”

Katriel has been working on exception groups, one of Python 3.11’s new features, with Van Rossum and others over the last two years. “Joining the Faster CPython team did change my focus to towards performance. In that sense my focus did change, but there was continuity in that I did some work on performance of exceptions. By simplifying the interpreter’s internal representation of raised exceptions, I reduced the time it takes to raise and catch an exception by about 10%. In another project I made the compiler emit the code of except blocks in a separate area, so that they don’t need to be jumped over in the case that no exception is raised.”

The team meets regularly to discuss these things. “All six of us meet every Monday,” says Van Rossum. “There’s always more than enough to talk about. That is very different than as a core dev community getting together for a Sprint twice a year, like one day after the conference. That is a very special event, of course, but it doesn’t feed me throughout the year.” Van Rossum believes that knowledge of one another and their collaborative work gave the team a “leg up” because everyone “knows what communication styles people have and what everybody’s weaknesses and strengths are.”

“As a core (Python dev) team across the whole community, we collaborate pretty well,” says Snow. He’s been a core developer for over a decade. “having a number of core devs working on the same stuff in CPython that are within the company, there’s a little better accessibility…the degree of collaboration is a bit more solid”.

“It’s kind of nice to have someone working while you’re sleeping. Like half the team working…” Bucher mentions. This is his first time working on a remote-first, internationally distributed team. “There are lots of times when, at the end of the day, I’ll push my work… and the next morning it’s reviewed.” Bucher, in California, has been doing a lot of work with Shannon, in the UK, on a Specialized Adaptive Interpreter, as well as tools, like Specialist, to help users moving to Python 3.11.

Working towards 3.12 and later

When a team has a shared vision, it’s easier to think and plan further down the road. Much of 3.11’s features were proposed before the team was assembled. The team worked to bring many of the performance features to the release, but also have already begun working on features in future versions.

Pereira is working on a change to how integers are represented inside Python. She intends to change smaller integers to use native computation instead of the slower algorithms for arbitrarily large numbers.

Shannon’s original 4 stage plan has continued to evolve to have continuous optimizations for the next several years.

“To make that as smooth as possible, you have to think in terms of smaller steps, right?” says Droettboom. Droettboom has worked on long-term projects in the scientific community including the Hubble Space Telescope and more recently the James Web Space Telescope. “Mark has a very long term goal. What we need to do is break that goal up into smaller chunks to find the things we can do now that clearly improve things at each step of the way.”

Working Beyond Microsoft

“We hope we can bring some knowledge from really large proprietary systems into what we develop for the Community.” says Droettboom. “I think that’s really valuable because then you’re not just doing it in the abstract. Not just imagining what’s going to make Python faster for real use cases, but actually measuring it.” Bucher adds in that developers working with these teams can test the impact of changes, “getting useful insights and contributions from people who maintain large, diverse codebases.”

The team has done a lot to make sure there is transparency in their work. The Faster CPython org on GitHub is public and they share their ideas and tools via public repos. Many of the team’s meetings feature core developers from other teams and companies. It is an opportunity to learn from others. Pereira mentions that discussions are “very productive”. “People know details I did not even know were a thing about the language. I end up learning quite a bit of Python, even though I’ve been using it for almost 20 years now.”

“It’s also really helping me with empathy,” says Droettboom. “I’m experiencing what it is like to be a new contributor to Python and some of the glitches that people run into, like places where the developer docs are not up to date. It’s given me a chance to be a new developer on a project, which is something I haven’t been in a while.”

Microsoft’s Commitment to Python

The development of this team follows a long history of Microsoft’s contribution to the Python community. Microsoft increased their long-time sponsorship to the Visionary tier. They regularly sponsor PyCon US and other regional conferences. They also sponsor other Python projects like NumFOCUS and provide Azure and Visual Studio resources for all core devs.

The team’s goal is to “add value” to the Python Community, according to Snow. Ultimately, “that’s our primary purpose as a team and I think that’s a pretty significant point. What I really appreciate about our team is that everybody gets it… we’re really more focused on the actual work to make Python better and to help the Python community outside of the technical area.” Droettboom believes that by focusing on the needs of the Python Community, “everyone benefits”.

Python 3.11 is now available and faster than ever! You can download it at Python.org. Check out the release notes to learn about all the features and optimizations that many developers including those on the Faster CPython Team have been working on for the past few years. You can also find out more about what the Faster CPython Team has in mind for 3.12 in their ideas repo on Github.