[PATCH v2 5/6] Allow use of Docker for development
Andrew Donnellan
andrew.donnellan at au1.ibm.com
Thu Aug 4 16:36:38 AEST 2016
On 04/08/16 16:10, Daniel Axtens wrote:
> This makes it possible to use Docker and docker-compose for development
> as an alternative to Vagrant.
>
> I quite liked vagrant a couple of years ago, but currently:
>
> * Trying to install VirtualBox on Ubuntu wants me to disable
> Secure Boot, and I don't want to do that.
>
> * Trying to use the libvirt plugin for vagrant requires I pick
> from a very small set of possible images, and requires that I
> install the upstream vagrant rather than the vagrant shipped
> with Ubuntu 16.04
>
> * I find docker containers faster to work with and more transparent.
>
> So I've done the work to make docker work for Patchwork development.
> This doesn't break or in any way interfere with using Vagrant, it just
> provides an alternative.
>
> It includes support for headless selenium tests using Chromium.
>
> Signed-off-by: Daniel Axtens <dja at axtens.net>
Excellent! I don't use either Docker or Vagrant very much these days,
but definitely Docker more than Vagrant...
Haven't tested this yet and my Docker-fu is pretty rusty, so I'll hold
off on my Reviewed-by until I've given it a go, but I didn't see any
major issues.
Minor comments below.
> +# Development Installation using Docker
> +
> +1. Install Docker and docker-compose.
> +2. Clone this repo, as with vagrant.
> +3. Build the images. This will download ~200MB from the internet:
> +
> + $ docker-compose build
> +
> +3. Run as follows:
Should that be 4?
> +
> + * Regular server:
> +
> + $ docker-compose up
> +
> + This will be visible on http://localhost:8000/.
> +
> + * Shell:
> +
> + $ docker-compose run --rm web --shell
> +
> + * Quick test (not including selenium UI interaction tests):
> +
> + $ docker-compose run --rm web --quick-test
> +
> + * Full tests, including selenium, run headlessly:
> +
> + $ docker-compose run --rm web --test
> +
> + * To reset the database before beginning, add `--reset` to the command line after `web` and before any other arguments.
> +
> + * If you want to run non-headless tests, you'll need something like this ugly hack:
> +
> + $ docker run -it --rm -v (pwd):/home/patchwork/patchwork/ --link patchwork_db_1:db -p 8000:8000 -v /tmp/.X11-unix:/tmp/.X11-unix -e PW_TEST_DB_HOST=db -e DISPLAY patchwork_web bash
> +
> +With both vagrant and docker, any edits to the project files made locally are immediately visible to the VM/container, and so should be picked up by the Django auto-reloader.
> +
> # Talks and Presentations
>
> * [**A New Patchwork**][pdf-fosdem] - FOSDEM 2016
> diff --git a/docker-compose.yml b/docker-compose.yml
> new file mode 100644
> index 000000000000..80d2571a2dbc
> --- /dev/null
> +++ b/docker-compose.yml
> @@ -0,0 +1,23 @@
> +# the version of docker-compose shipped in ubuntu 16.04 is
> +# 1.5.2, which doesn't support version 2 syntax. Yay!
> +# also, v1 doesn't support explicit build args, so if you're not
> +# uid 1000, you will either need to manually hack the Dockerfile
> +# or upgrade to v2 and use the build-arg to override it.
Gross :(
> +
> +db:
> + build: docker/db
> + volumes:
> + - ./docker/db/data:/var/lib/mysql
> +web:
> + build: .
> + dockerfile: ./docker/Dockerfile
> + command: python3 manage.py runserver 0.0.0.0:8000
> + volumes:
> + - .:/home/patchwork/patchwork/
> + ports:
> + - "8000:8000"
> + links:
> + - db
> + environment:
> + - PW_TEST_DB_HOST=db
> + - PW_TEST_DB_PORT=3306
> \ No newline at end of file
> diff --git a/docker/Dockerfile b/docker/Dockerfile
> new file mode 100644
> index 000000000000..b5262cc516a3
> --- /dev/null
> +++ b/docker/Dockerfile
> @@ -0,0 +1,44 @@
> +FROM ubuntu
> +
> +ARG UID=1000
> +
> +ENV PROJECT_HOME /home/patchwork/patchwork
> +
> +ENV db_user root
> +ENV db_pass password
> +
> +ENV DJANGO_SETTINGS_MODULE patchwork.settings.dev
> +ENV DEBIAN_FRONTEND noninteractive
> +ENV PYTHONUNBUFFERED 1
> +
> +# System
> +RUN apt-get update -qq && \
> + apt-get install -y --no-install-recommends \
> + python-dev python-pip python-setuptools python-wheel \
> + python3-dev python3-pip python3-setuptools python3-wheel \
> + libmysqlclient-dev mysql-client curl unzip xvfb chromium-chromedriver \
> + chromium-browser build-essential && \
> + ln -s /usr/lib/chromium-browser/chromedriver /usr/bin/
> +
> +# User
> +RUN useradd --uid=$UID --create-home patchwork
> +
> +# Python requirements.
> +# If you update requrirements, you should rebuild the container.
requirements
> +COPY requirements-*.txt /tmp/
> +RUN pip3 install virtualenv tox && \
> + pip3 install -r /tmp/requirements-dev.txt
> +# we deliberately leave the requirements files in tmp so we can
> +# ping the user if they change it!
> +
> +COPY docker/bashrc /tmp/bashrc
> +
> +# we put the code in ~/patchwork rather than ~ so that we
> +# can put in these bashrc snippets
> +RUN cat /tmp/bashrc >> /home/patchwork/.bashrc
> +
> +COPY docker/entrypoint.sh /usr/local/bin/entrypoint.sh
> +
> +ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
> +USER patchwork
> +WORKDIR /home/patchwork/patchwork
> diff --git a/docker/bashrc b/docker/bashrc
> new file mode 100644
> index 000000000000..eb2ed7dddb85
> --- /dev/null
> +++ b/docker/bashrc
> @@ -0,0 +1,5 @@
> +# This snippet is appended to ~/.bashrc when the container is created
> +
> +alias runserver='python3 $PROJECT_HOME/manage.py runserver 0.0.0.0:8000'
> +alias createsu='python3 $PROJECT_HOME/manage.py createsuperuser'
> +
> diff --git a/docker/db/.dockerignore b/docker/db/.dockerignore
> new file mode 100644
> index 000000000000..1269488f7fb1
> --- /dev/null
> +++ b/docker/db/.dockerignore
> @@ -0,0 +1 @@
> +data
> diff --git a/docker/db/.gitignore b/docker/db/.gitignore
> new file mode 100644
> index 000000000000..60baa9cb833f
> --- /dev/null
> +++ b/docker/db/.gitignore
> @@ -0,0 +1 @@
> +data/*
> diff --git a/docker/db/Dockerfile b/docker/db/Dockerfile
> new file mode 100644
> index 000000000000..5df9b5acb486
> --- /dev/null
> +++ b/docker/db/Dockerfile
> @@ -0,0 +1,10 @@
> +FROM mysql:5.7
> +
> +ENV MYSQL_ROOT_PASSWORD password
> +ENV MYSQL_USER patchwork
> +ENV MYSQL_PASSWORD password
> +
> +# We don't want to use the MYSQL_DATABASE env here because
> +# we want to be able to create the database with UTF-8 explictly.
> +# We also can't load in the data because it's in XML, yay.
> +
> diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh
> new file mode 100755
> index 000000000000..c6cf18713fd8
> --- /dev/null
> +++ b/docker/entrypoint.sh
> @@ -0,0 +1,80 @@
> +#!/bin/bash
> +set -euo pipefail
> +
> +# functions
> +test_db_connection() {
> + mysqladmin -h $PW_TEST_DB_HOST -u patchwork --password=password ping > /dev/null 2> /dev/null
> +}
> +
> +reset_data() {
> + mysql -u$db_user -p$db_pass -h $PW_TEST_DB_HOST << EOF
> +DROP DATABASE IF EXISTS patchwork;
> +CREATE DATABASE patchwork CHARACTER SET utf8;
> +GRANT ALL ON patchwork.* TO 'patchwork' IDENTIFIED BY 'password';
> +FLUSH PRIVILEGES;
> +EOF
> +
> + # load initial data
> + python3 $PROJECT_HOME/manage.py migrate #> /dev/null
> + python3 $PROJECT_HOME/manage.py loaddata default_tags #> /dev/null
> + python3 $PROJECT_HOME/manage.py loaddata default_states #> /dev/null
> + python3 $PROJECT_HOME/manage.py loaddata default_projects #> /dev/null
> +}
> +
> +
> +# The script begins!
> +
> +# check if patchwork is mounted. Checking if we exist is a
> +# very good start!
> +if [ ! -f ~patchwork/patchwork/docker/entrypoint.sh ]; then
> + echo "The patchwork directory doesn't seem to be mounted!"
> + echo "Are you using docker-compose?"
> + echo "If not, you need -v PATH_TO_PATCHWORK:/home/patchwork/patchwork"
> + exit 1
> +fi
> +
> +# check if we need to rebuild because requirements changed
> +for x in /tmp/requirements-*.txt; do
> + if ! cmp $x ~/patchwork/$(basename $x); then
> + echo "A requirements file has changed."
> + echo "Please rebuild the patchwork image:"
> + echo " docker-compose build web"
> + exit 1
> + fi
> +done
> +
> +# check if mysql is connected
> +if ! test_db_connection; then
> + echo "MySQL seems not to be connected, or the patchwork user is broken"
> + echo "MySQL may still be starting. Waiting 5 seconds."
> + sleep 5
> + if ! test_db_connection; then
> + echo "Still cannot connect to MySQL."
> + echo "Are you using docker-compose? If not, have you set up the link correctly?"
> + exit 1
> + fi
> +fi
> +
> +# rebuild mysql db
> +# do this on --reset or if the db doesn't exist
> +if [[ "$1" == "--reset" ]]; then
> + shift
> + reset_data
> +elif ! ( echo ';' | mysql -h db -u patchwork -ppassword patchwork 2> /dev/null ); then
> + reset_data
> +fi
> +
> +if [ $# -eq 0 ]; then
> + # we probably ran with --reset and nothing else
> + # just exit cleanly
> + exit 0
> +elif [ "$1" == "--shell" ]; then
> + exec bash
> +elif [ "$1" == "--quick-test" ]; then
> + export PW_SKIP_BROWSER_TESTS=yes
> + python3 manage.py test
> +elif [ "$1" == "--test" ]; then
> + xvfb-run --server-args='-screen 0, 1024x768x16' python3 manage.py test
> +else # run whatever CMD is set to
> + $@
> +fi
>
--
Andrew Donnellan OzLabs, ADL Canberra
andrew.donnellan at au1.ibm.com IBM Australia Limited
More information about the Patchwork
mailing list