Python Development Environments

How to install multiple versions of Python for a professional development environment.

What This Post Covers

• How to setup a development environment that supports multiple separate versions of Python
• Why it is important to avoid using the system version(s) of Python
• Other helpful tools for working with Python environments

Installing First Version of Python

It is recommended to use an all-in-one tool to manage multiple installed versions of Python with a single interface. This makes updating, installing new, and managing existing versions as simple as possible.

One example of such a tool is pyenv.

From the documentation… [1]

What pyenv does…

• Lets you change the global Python version on a per-user basis.
• Provides support for per-project Python versions.
• Allows you to override the Python version with an environment variable.

Usage is as simple as:

1
2
3
4
5
6
7

pyenv install 3.11  # Install specific version

pyenv global 3.11   # Set global (default) version to use instead of
                    #   falling back to "system"

pyenv version       # e.g. `3.11.7 (set by ...)`
python --version    # e.g. `Python 3.11.7`

The documentation for pyenv covers installation and usage of all available sub-commands. I will draw special attention to the “Usage” section, as this is the most efficient description of how to get started using the tool.

The Version Might Matter

Usually, the first version of Python installed with this approach should not matter. However, there are certain cases where the choice might affect how other versions behave with the tooling chosen.

For example, when setting up a default Python to be able to use for system-wide tooling, the choice of version will depend on the support for that tooling in each version of Python. If using a pip-installed package that drops support for Python 3.6 and prior, installing version 3.7 or later is required [3].

Leave System Python Alone

I learned most of the details of this approach from a talk given at PyCon 2022 by Calvin Hendryx-Parker. In my opinion the most important takeaway from that talk was to completely leave system-installed versions of Python alone: opting to avoid versions installable via the system package-manager entirely (as those are often what your system will use).

The main reason for avoiding system-installed Python is because that is likely what your system is using to do many system-management tasks: from installing packages to letting installed packages take advantage of the Python ecosystem as well. The installation, management, and update of the system version(s) of Python should be left up to the system itself. Think of this like the OS was the project you were developing: you should have control over the version you are using and when it needs to be changed or updated in order to keep everything about the project (the OS) working.

If the system decides it needs to replace, say, Python 2 with Python 3 in order to meet its needs, then you’ve just lost your Python 2 development environment’s runtime. A more explicit –and potentially less obvious– problem that can occur is when you rely on a specific minor version of Python, but your system needs a later one. By upgrading, your system continues to function but your projects may break.

Another reason it is a good practice to leave system Python alone is because you can avoid headaches related to installation issues. Say you mess with your Python config and need to re-install. If using system Python, you are putting your system’s capability to function properly at risk. All the more reason to keep your development runtime as isolated as possible.

Other Tools

Here are some other tools that can help take your environment management to the next level.

virtualenvwrapper

virtualenvwrapper [4] is a convenience tool for creating and managing Python virtual environments. With a single command, mkvirtualenv, a new Python environment can be created that automatically pip installs from a requirements file and places the shell into a desired project directory.

There even exists a pyenv-virtualenvwrapper plugin for pyenv that integrates the two tools so that there is no hassle when dealing with virtual environments across all managed versions of Python. Again, installation and usage is best learned from the docs for the project.

virtualenvwrapper seamlessly integrates into my tmux setup: after setting up the shell environment, a simple workon <etc> initiates the Python virtual environment with the proper version and any other wrapper hooks or environment variables. Multiple layers of environment configuration like this can be slightly confusing, however, it makes a big difference as projects become more numerous and complex.

pipx

Also highlighted in the PyCon talk mentioned prior, pipx is a tool for installing Python programs without cluttering local Python environments (as is typically the case when pip-installing programs to be run with a specific “global” Python install).

I don’t personally have a lot of experience with this tool, however, it seems very well suited for developers who just “want to get stuff done” without worrying about messing up their Python environments with random installed packages. It also provides a very convenient way to run programs from a “global” shell context (avoiding the need to activate a given Python environment before the command can be found on PATH).

Wrapping Up

Managing many installed versions of Python and the various project-specific configs that come with 3rd-party package installs can be quite tricky. However, using certain tools like pyenv to your advantage can save a lot of time and trouble. I highly recommend the above described approach and even suggest putting some shell snippets in each project to allow contributors the ease of being able to reproduce a consistent development environment:

1
2
3
4
5
6
7
8

pyenv install 3.11.7
mkvirtualenv \
    --python $(pyenv prefix 3.11) \
    -a /home/tim/example-project \
    -r /home/tim/example-project/requirements.txt \
    example

workon example

For context, here is an abridged snippet of commands I ran to setup my machine with pyenv and virtualenvwrapper (on Linux Mint 21.2):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

# Update package refs and install pre-dependencies
sudo apt update
sudo apt install \
    git wget curl \
    build-essential gcc \
    libssl-dev zlib1g-dev libbz2-dev libreadline-dev \
    libsqlite3-dev curl libncursesw5-dev xz-utils \
    tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev
# Install homebrew
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
# Initialize brew
(echo; echo 'eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"') >> /home/tim/.bashrc
eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
# Install pyenv and virtualenvwrapper
brew install pyenv pyenv-virtualenv pyenv-virtualenvwrapper
# Init pyenv
eval "$(pyenv init -)"
# Install python
brew unlink pkg-config  # This fixes an issue installing Python on the next step
pyenv install 3.12
# Init virtualenvwrapper
pyenv virtualenvwrapper

References

[1] https://github.com/pyenv/pyenv#what-pyenv-does

[2] https://brew.sh/

[3] This can be remedied by simply updating the default version, however, the example is intentionally simple –for ease of understanding– and is not the exact issue seen by the author. The real issue encountered was related to how virtualenvwrapper behaves when creating virtual environments for versions of Python other than the version for which the virtualenvwrapper package, itself, was installed.

[4] https://virtualenvwrapper.readthedocs.io/en/latest/