How to speed up an already cached pip install?
- by Maxime R.
I frequently have to re-create virtual environments from a requirements.txt and I am already using $PIP_DOWNLOAD_CACHE. It still takes a lot of time and I noticed the following:
Pip spends a lot of time between the following two lines:
Downloading/unpacking SomePackage==1.4 (from -r requirements.txt (line 2))
Using download cache from $HOME/.pip_download_cache/cached_package.tar.gz
Like ~20 seconds on average to decide it's going to use the cached package, then the install is fast. This is a lot of time when you have to install dozens of packages (actually enough to write this question).
What is going on in the background?
Are they some sort of integrity checks against the online package?
Is there a way to speed this up?
edit: Looking at:
time pip install -v Django==1.4
I get:
real 1m16.120s
user 0m4.312s
sys 0m1.280s
The full output is here http://pastebin.com/e4Q2B5BA. Looks like pip is spending his time looking for a valid download link while it already has a valid cache of http://pypi.python.org/packages/source/D/Django/Django-1.4.tar.gz.
Is there a way to look for the cache first and stop there if versions match?
