Workaround for bad interaction between pipenv and pyenv

I use Arch Linux these days, and it is a ‘bleeding-edge’ distro, which means that, typically, only the latest stable versions of upstream packages are available.

Recently, python 3.8 entered the arch repos, replacing python 3.7 altogether. Note that python 2.7 is still available, although I expect that it will be dropped on January.

So, if you are working on a project that depends on python 3.7, you can leverage pyenv, which is readily available from the Community repo.

pyenv requires a modicum of setup, which is described here.

pipenv supports working together with pyenv, and will detect if the required python version in your Pipfile is not installed in your system, and prompt you to install it on your behalf, via pyenv:

$ tail -2 Pipfile
[requires]
python_version = "3.7"
$ pipenv sync
Warning: Python 3.7 was not found on your system…
Would you like us to install CPython 3.7.5 with pyenv? [Y/n]:
Installing CPython 3.7.5 with pyenv (this may take a few minutes)…
✔ Success!

Warning: The Python you just installed is not available on your PATH, apparently.

Oops! Something is not working here, even though the PATH is setup OK.

It is a known issue with pipenv that is possibly fixed in master, but definitely not in the latest release, which, as of today, is almost a year old. Be patient, a lot has been happening, but work is underway.

We can work around this issue by ensuring that the required python version is installed, selecting it for the current project, and forcing pipenv to use it.

$ pyenv install 3.7.5
Downloading Python-3.7.5.tar.xz...
-> https://www.python.org/ftp/python/3.7.5/Python-3.7.5.tar.xz
Installing Python-3.7.5...
Installed Python-3.7.5 to /home/pancho/.pyenv/versions/3.7.5
$ cd someproject
# This creates a =.python-version= file containing the selected version
$ pyenv local 3.7.5
$ pipenv --python "$(pyenv root)/shims/python" sync
... Works OK now ...

This can be inconvenient if pipenv is being run from a Makefile that we don’t want to edit. Fortunately, there is a workaround for that, too.

Instead of using the --python option to pipenv, we can just define the env var PIPENV_PYTHON, which controls the same behaviour (seen here).

To sum up:

$ export PIPENV_PYTHON="$(pyenv root)/shims/python"
$ pyenv install 3.7.5
... yadda yadda ...
$ pyenv local 3.7.5
$ pipenv sync

As a final note, we can add the definition of PIPENV_PYTHON to your shell init script, so we don’t have to bother to set it up everytime this comes across.

pancho horrillo
I do things with computers.