Requirements for Python’s pip
Posted on Sun 21 February 2016 in Code • Tagged with Python, pip, packages, dependencies • Leave a comment
In this post I’ll describe all (hopefully all!) the various ways you can specify a single dependency for a Python package.
This assumes pip is used for installation. The list of dependencies then goes either in install_requires=
parameter of the setup
function within setup.py, or as a separate requirements.txt file. Commonly, it will actually
go in both places, with the latter being the canonical source of truth:
from setuptools import setup
with open('requirements.txt') as rf:
setup(
# ...
install_requires=rf.readlines(),
)
More details about this approach can be found in one of my previous posts.
Here, I will concentrate on the format of a single line in requirements.txt that defines a dependency. There are numerous variants that pip supports, and they are all described in excruciating detail in PEP 440. This post shall serve as a short reference on the most useful ones.
Package name (and version)
The simplest and most common option is to identify a dependency by its package name:
SQLAlchemy
This will locate it in a global index of packages, which is sometimes called a “cheese shop”. Currently, by far the most popular package registry for Python is PyPI, and pip uses it by default1.
Without any further modifiers, pip will download and install the “current” version of the package — either the newest, or the one designated explicitly by a maintainer. This obviously makes the dependency somewhat unpredictable, for it can mean unintended upgrades that introduce breaking changes to your code.
To prevent this, you’d normally pin the dependency to an exact version2:
SQLAlchemy==0.9.10
Other comparison operators are also available:
SQLAlchemy>=0.9.10
SQLAlchemy<1.0.0
and can even be combined:
SQLAlchemy>=0.9,<1.0.0
Specs like that will make pip find the newest version that’s within given range. Assuming your dependency follows the semantic versioning scheme, this will allow you to stay on top of any minor bugfixes and improvements to an older release (0.9.x here), without the risk of accidentally upgrading to a new one (1.x) that your code is not compatible with yet.
Repository URL
Sometimes you want to live on the bleeding edge, though, and depend not just on the latest release, but the head commit to the package’s repository. This makes sense especially in large systems that are distributed among multiple repos, and where development happens in lockstep.
For those occasions, and a few others, pip can recognize direct repository URLs. They are in the format:
$VCS+$PROTOCOL://$URL@$LABEL#egg=$PACKAGE
where the $PROTOCOL
part can be optional if the version control system has a default there. That’s for example
the case for git
, which is of course the most important VCS you’d be interested in3:
git://git.example.com/somepackage#egg=somepackage
Note that the #egg=$PACKAGE
part is not a part of the $URL
, and it’s only there to give a local name for the package
distribution. This is what makes it possible to refer to it later via pip, if only to remove it with
pip uninstall $PACKAGE
.
Of course, the sanest practice is to use the PyPI moniker if possible.
When no $LABEL
is given, pip will use the HEAD, trunk, tip, or the equivalent default/current revision from the repo.
Often though (at least in case of Git), you would also pick a branch, tag, or even a particular commit hash:
git+https://github.com/Xion/unmatcher.git@0.1.3.1#egg=unmatcher
git+ssh://github.com/You/yourpackage.git@master#egg=yourpackage
git+https://github.com/mitsuhiko/jinja2.git@5b498453b5898257b2287f14ef6c363799f1405a#egg=Jinja2
The last two options could be a good choice even with third party packages, when you don’t want to wait for a new PyPI release to get a necessary feature or an urgent bug fix.
Local filesystem
Lastly, you can ask pip to install a package from a local directory or archive. The former option is often used
with the -e
(--editable
) flag for pip install
. This installs the package in the so-called development mode,
allowing you to edit its source code in-place:
$ pip install -e /home/me/Code/myotherpackage
You almost certainly don’t want to put this line in requirements.txt: you should still be pulling the other package from PyPI. But if it’s your own one — maybe a self-contained utility library used by your main program — this setup will be very helpful for making changes to it, informed by your own usage of the package.
-
This can be changed with
--index_url
flag topip install
. Running local indexes is a good practice for Python shops, especially those that rely onpip install
as part of their deployment process. ↩ -
If the package uses semantic versioning, a possible alternative to
==
is~=
, which means “compatible” version. The precise meaning of this is somewhat complicated, but it roughly means that upgrades are permitted as long as nothing in the public interface changes. ↩ -
Other options include
hg
(Mercurial),svn
, andbzr
(Bazaar). ↩