Every language and its community seems to have its own story (and at times drama) on how to handle dependencies. We even have a conference just to talk about it. In Python, isolating dependencies and interpreter versions (together known as a "virtual environment") is notorious for tripping up beginners just getting started. The Python community understands this but is still deciding on a solution. Since Python is not my main daily language, I tend to forget about virtual environments for months at a time. This post is mostly for me to remember how I do it each time, but it's simple enough and uses familiar tooling that I hope it helps someone else.
In Python 3.3, the
venv module got added natively to the default Python interpreter.
This is the main workhorse between virtual environments and by learning a little about how it works, we can have a simple workflow to automatically create virtual environments when we cd into a project directory.
The three main tools involved are asdf, venv (the module), and direnv.
Overview on each tool
asdf is a generic version manager for almost all of the popular language runtimes.
Alongside python, I use it to manage my golang, nodejs and perl installations.
One package manager to rule them all.
direnv allows you to change your shell's environment based on which current directory your shell session lives in.
There are some neat features tucked into this tool so I suggest you dive into their documentation.
As you soon will see, you use the
venv module to actually create the virtual environment.
Once that happens, you'll have access to executables that can be used to
activate these virtual environments.
.venv/bin/activate does is change the shell's environment variables, we skip right over using it and change the environment the same way with
My simple workflow
Change into (or create) your project directory:
$ cd /the/path/to/your/project
Start by creating the virtual environment:
$ python -m venv .venv # Directory name can be anything
Then create an
direnv looks for) to set relevant
PATH vars for the right executables to be picked up:
$ echo "export VIRTUAL_ENV=$PWD/.venv\nexport PATH=$PWD/.venv/bin:\$PATH" > .envrc
I'm pretty sure
VIRTUAL_ENV isn't necessary to be set but since explicit activation does it, there's no harm.
Direnv should then warn you that these new environment variables are going to be loaded. To continue,
$ direnv allow
That should be it! To confirm that
pip are pointing to the right places:
$ which python3 /the/path/to/your/project/.venv/bin/python3 $ which pip /the/path/to/your/project/.venv/bin/pip
You should have now a virtual environment activated each time to change into this directory from your shell!
$ cd /the/path/to/your/project direnv: loading ~/path/to/your/project/.envrc direnv: export +VIRTUAL_ENV ~PATH $ pip install -r requirements.txt # Uses virtual environment