Package Management in the Python extension for Visual Studio Code

Courtney Webster

Python, with its simple syntax and powerful libraries, has become one of the most popular programming languages and a great language for those new to programming. However, as your projects grow in complexity and size, so does the complexity of managing dependencies. As new users take on more mature projects, they learn about the importance of virtual environments and managing dependencies. While the benefits of virtual environments and package management in Python are undeniable, it’s essential to acknowledge the potential challenges they pose, particularly for beginners. For those new to Python development, the terminology, concepts, and the variety of tools available can be overwhelming.

The Python extension for Visual Studio Code set to demystify virtual environments and streamline environment creation with the Python: Create Environment command. This command helps developers of any skill level create isolated environments, each with its own set of installed packages and dependencies. However, we believe, there is still progress to be made in our support for package management tools in the extension. At a high level, package management tools simplify the process of installing packages, resolving dependencies, and handling version constraints. While there are a number of package management tools available, we wanted to see how far we can get with support from pip and venv, which are included in a standard Python install. Part of this exercise included researching common CLI commands offered by popular package management tools to discover the intersection of functionality and identify where pip and venv don’t offer direct support. Our hope is to provide an opinionated workflow that integrates with the Python: Create Environment command to streamline package management.

Goals

Propose and implement an opinionated workflow as part of the Python extension for VS Code to:

• Support beginners who are working with Python packages and virtual environments.
• Streamline the package management story with the environment creation story.
• Allow third-party package management extensions to integrate into the workflow to provide their own support.

Current Proposal

The Python extension in VS Code would like to address a common hurdle for beginners in package management by providing an opinionated workflow, similar to that in the Python: Create Environment command. However, we acknowledge no single approach can cater to every user scenario and preference. Initially, these features primarily focus on assisting beginners who work on scripts, applications, and projects that don’t require building a wheel file. Beginners are primarily concerned with achieving successful execution and ensuring reproducibility. As the Python extension for VS Code considers how we can approach this problem, we want to work closely aligned with standards outlined by the community but are choosing to be opinionated in some areas. We are thinking about this proposal in two parts: command coverage and documenting dependencies for the user. As always, we are open to feedback on how this can be improved and implemented (vscode-python#21627)!

There are several popular dependency and package management tools available. For this research, I compared Pip, Poetry, Hatch, conda, PDM, and Pipenv CLI commands to find the intersection of functionality they provide for package management and where pip and venv do not offer direct support that these other tools have.

CLI command coverage that we will want to match in our opinionated workflow:

• Create/update a dependency file
• Read, resolve, and install dependencies from a file
• Update dependencies
• Remove a package
• Add a package (and update the dependency file)
• Create environment ✅
• Run code ✅

Documenting Dependencies

While we have an outline of the functionality we hope to support, the question of how to write down dependencies still remains. Although there are no firm standards behind the “where” and “how” to write down dependencies, we are keeping the main motivators for beginners in mind: successful execution and reproducibility.

Initially, we believe the default should be to write to a requirements.txt file, including the package version that was installed. Opting to write the version, by default, supports the users’ need for reproducibility and ease of collaborating on work. Given the nature of the work done by the target audience and our target usage, a pyproject.toml is out of scope as the target user typically does not build their project nor is there a direct way to only install dependencies listed in a pyproject.toml file.

While this workflow is initially focused on helping beginners who may be uncomfortable or unfamiliar with these concepts, we want this to be a tool for more advanced developers as well. Our hope is to allow your preferred package managers to integrate into this workflow via extensions to support more advanced workflows.

Conclusion

The Python extension in VS Code is continuously evaluating ways in which we can improve users’ experience in the extension, whether they are new or not. Package management is a core stepping stone when learning and using Python, and we want to reduce the hurdles necessary to be successful. As always, we would love to hear your thoughts and feedback regarding our features. Specific feedback regarding this proposal can be given at vscode-python#21627.

CLI Command Coverage

As part of this investigation, I populated a table to track CLI commands across package management tools. This table does not account for all the commands supported but, rather those specifically related to environment creation and package management. Additionally, many of the commands listed below support options to extend the command’s functionality that are not documented here for the sake of brevity.

Courtney Webster Program Manager, Python Extension in Visual Studio Code

Follow