Python dependency resolution
pipenv install broke on the project I was working on. It worked yesterday and the day before that, but today it failed.
At first, I thought my Python or pipenv installation was broken. The project’s integration tests ran in a Docker container, so I
docker run -it <image> bash'd into a container and tried
pipenv install. The result was the same—“dependencies could not be resolved”. This was a surprise…how could a Dockerized application suddenly stop working? The image worked yesterday and had not changed. I downloaded a fresh copy of the repo and tried again, but the same error occurred.
If nothing changed locally, the source of the malfunction must lie with how pipenv install operates. Looking at the Pipfile, I noticed that few dependencies were pinned to a specific version. So the version that ultimately gets installed must be decided by pipenv each time
pipenv install is run. If pipenv prefers to pick newer versions, then it’s possible one of my project’s dependencies updated and somehow confused pipenv to the point of being unable to resolve dependencies it had succeeded with only 24 hours ago.
Further investigation supported this idea. The dependency conflict was over a package called PyYAML; one package required PyYAML <= 3.13 while another package required PyYAML >= 4.1. Four days ago the second package had updated it’s dependency on PyYAML from >= 3.13 to >= 4.1 (presumably) causing
pipenv install to break. Pipenv was not able to figure out that the previous version of the second package only required PyYAML >= 3.13.
The resolution was to pin the package that had recently changed to its previous version. Forcing pipenv to use the old version allowed it to resolve the dependencies and generate a new lockfile. This was not a clean fix as it required adding a package that is not a direct dependency to our project’s Pipfile.
Figuring this out was not a great experience.
pipenv install --verbose shows how it decides what versions to install, but reading through the rounds is tedious.
pipenv graph and
pipenv install stopped working.