In the previous part of this series we were able to connect a Docker network to a virtual interface on our host, neither of which have access to the internet. That means we are ready to host content for the container locally. And we will start out with creating a local Python repository.
Packaging the packages
I'll be so bold as to assume that you are using pip to manage your packages. It gives you not only the option to install packages, but merely download them to storage aswell. So let's do that and try to serve the packages.
$ pip download faker [...] Successfully downloaded faker python-dateutil six text-unidecode $ ls Faker-8.1.0-py3-none-any.whl python_dateutil-2.8.1-py2.py3-none-any.whl six-1.15.0-py2.py3-none-any.whl text_unidecode-1.3-py2.py3-none-any.whl $ python -m RangeHTTPServer
$ python -m venv .venv $ . .venv/bin/activate (.venv) $ pip install --index http://localhost:8000 faker Looking in indexes: http://localhost:8000 ERROR: Could not find a version that satisfies the requirement faker (from versions: none) ERROR: No matching distribution found for faker
Dag nabbit. Apparently not that simple. And indeed, if we read up on the spec for PEP 503 spec for Simple Repository API, we learn that we are going to stick those package files under directories named after the packages.
Well, nothing that a bit of bash scripting can't sort out:
#!/usr/bin/env s=$1 d=$2 if [ -z $d ]; then >&2 echo "usage: pep503.sh <source_dir> <dest_dir>" exit 1 fi for df in `find $s -name "*.whl" -type f`; do f=`basename $df` pd=`echo $f | sed -e "s/^\(.*\)-[[:digit:]]*\.[[:digit:]].*$/\1/g" | tr "[:upper:]" "[:lower:]" | tr "_" "-"` mkdir -v $d/$pd cp -v $df $d/$pd/ done for df in `find $s -name "*.tar.gz" -type f`; do f=`basename $df` pd=`echo $f | sed -e "s/^\(.*\)-[[:digit:]]*\.[[:digit:]].*$/\1/g" | tr "[:upper:]" "[:lower:]" | tr "_" "-"` mkdir -v $d/$pd cp -v $df $d/$pd/ done for df in `find $s -name "*.zip" -type f`; do f=`basename $df` pd=`echo $f | sed -e "s/^\(.*\)-[[:digit:]]*\.[[:digit:]].*$/\1/g" | tr "[:upper:]" "[:lower:]" | tr "_" "-"` mkdir -v $d/$pd cp -v $df $d/$pd/ done
Armed with this, let's try again:
$ sh /home/lash/bin/shell/pep503.sh . packages $ mkdir packages $ bash /home/lash/bin/shell/pep503.sh . packages mkdir: created directory 'packages/text-unidecode' './text_unidecode-1.3-py2.py3-none-any.whl' -> 'packages/text-unidecode/text_unidecode-1.3-py2.py3-none-any.whl' mkdir: created directory 'packages/six' './six-1.15.0-py2.py3-none-any.whl' -> 'packages/six/six-1.15.0-py2.py3-none-any.whl' mkdir: created directory 'packages/python-dateutil' './python_dateutil-2.8.1-py2.py3-none-any.whl' -> 'packages/python-dateutil/python_dateutil-2.8.1-py2.py3-none-any.whl' mkdir: created directory 'packages/faker' './Faker-8.1.0-py3-none-any.whl' -> 'packages/faker/Faker-8.1.0-py3-none-any.whl' $ find packages/ -type f packages/faker/Faker-8.1.0-py3-none-any.whl packages/python-dateutil/python_dateutil-2.8.1-py2.py3-none-any.whl packages/six/six-1.15.0-py2.py3-none-any.whl packages/text-unidecode/text_unidecode-1.3-py2.py3-none-any.whl k $ python -m RangeHTTPServer
$ pip install --index http://localhost:8000/packages faker Looking in indexes: http://localhost:8000/packages Collecting faker [...] Successfully installed faker-8.1.0 python-dateutil-2.8.1 six-1.15.0 text-unidecode-1.3
There are some basic packages you will most always need, and which pip often will expect to find in at least one of its available repositories, even regardless of whether its installed or not. If you don't have this on your local offline repository when the internet goes poof, then that will block any build you are trying to do.
Let's make sure we have the packages around all the time:
$ pip download pip setuptools setuptools-markdown wheel [...] $ bash /home/lash/bin/shell/pep503.sh . packages [...]
Choosing a server
As I tend to favor the classics, I still use Apache Web Server to host stuff to my local environment. One practical (if not all too safe) thing about it is that it will automatically bind to all interfaces. So to make the repository available, we simply link or add the packages directory to the virtual root, and restart the server e.g. with
systemctl restart httpd
Of course you can use any HTTP server you like, as long as you know how to bind it to the virtual interface.
To verify that the service is providing what's needed, simply point your web browser to the location, e.g.
Serving the packages
Now that we are all prepped, the next step is to do install packages from within a Docker container.
Let's start with an Archlinux base image with basic python provisions.
FROM archlinux:latest RUN pacman -Sy && \ pacman -S --noconfirm python python-pip
Build and tag it with pythonbase, and then let's add the Dockerfile to test the repository:
[...] RUN pip install --index http://10.1.2.1/packages --trusted-host 10.1.2.1 faker
Now, turn internet off (and lights, too, if you'd like some extra suspense), and build the second file.
$ docker build . Sending build context to Docker daemon 3.072kB [...] Successfully installed faker-8.1.0 python-dateutil-2.8.1 six-1.15.0 text-unidecode-1.3 Removing intermediate container 2c10ffcdf3ad ---> 1ba83bb8e111 Successfully built 1ba83bb8e111