[PATCH v3 4/5] tox: Integrate tox-docker

Wed Aug 21 16:07:50 AEST 2019

Stephen Finucane <stephen at that.guru> writes:

> This eliminates the need to use docker-compose for most use cases.
> Instead, we can now do:
>
>     tox -e py27-django111-postgres
>
> If you're using a locally configured PostgreSQL or MySQL instance, you
> simply omit the last factor and things behave as before:
>
>     tox -e py27-django111
>
> We removed the 'venv' environment, since it was never actually that
> useful and is even less so now (you'd need to have a local DB set up)
> and add the 'skip_missing_interpreters' flag for folks on an OS that
> doesn't provide all the Python versions under the sun like Fedora does.
>
> Signed-off-by: Stephen Finucane <stephen at that.guru>
> Cc: Daniel Axtens <dja at axtens.net>
> ---
> v3:
> - Bump the mysql container version to 8.0 to work around an issue with
>   tox-docker (https://github.com/tox-dev/tox-docker/issues/35)
> - Use the 'root' user for the mysql container since it simplifies things
>   greatly (we do this for Travis already)
> - Revert changes to requirements.txt which turned out to be unnecessary
>   and broke Travis
> - Remove the venv environment instead of attempting to hack it into
>   working
> ---
>  docs/development/contributing.rst | 28 +++++++---------------------
>  docs/development/installation.rst |  6 ------
>  patchwork/settings/dev.py         | 12 ++++++++++++
>  tox.ini                           | 25 ++++++++++++++++++++-----
>  4 files changed, 39 insertions(+), 32 deletions(-)
>
> diff --git a/docs/development/contributing.rst b/docs/development/contributing.rst
> index 5089bba8..f713f872 100644
> --- a/docs/development/contributing.rst
> +++ b/docs/development/contributing.rst
> @@ -31,29 +31,15 @@ Testing
>  -------
>  
>  Patchwork includes a `tox`_ script to automate testing. This requires a
> -functional database and some Python requirements like *tox*. Refer to
> -:doc:`installation` for information on how to configure these.
> -
> -You may also need to install *tox*. If so, do this now:
> +functional database and some Python requirements like *tox*. These can be
> +installed using :command:`pip`:
>  
>  .. code-block:: shell
>  
> -   $ pip install --user tox
> -
> -.. tip::
> -
> -   If you're using Docker, you may not need to install *tox*
> -   locally. Instead, it will already be installed inside the
> -   container. For Docker, you can run *tox* like so:
> -
> -   .. code-block:: shell
> -
> -      $ docker-compose run --rm web tox [ARGS...]
> -
> -   For more information, refer to :ref:`installation-docker`.
> +   $ pip install --user tox tox-docker
>  
> -Assuming these requirements are met, actually testing Patchwork is quite easy
> -to do. To start, you can show the default targets like so:
> +Once installed, actually testing Patchwork is quite easy to do. To start, you
> +can show the default targets like so:
>  
>  .. code-block:: shell
>  
> @@ -66,7 +52,7 @@ parameter:
>  
>  .. code-block:: shell
>  
> -   $ tox -e py27-django18
> +   $ tox -e py36-django21-mysql

So I'm trying this out (finally!) and it seems to want to install all
the dependencies locally before starting a container. I don't have the
mysql bits installed, so it fails looking for `mysql_config`. Is this
supposed to happen or am I Doing It Wrong?

Regards,
Daniel
>  
>  In the case of the unit tests targets, you can also run specific tests by
>  passing the fully qualified test name as an additional argument to this
> @@ -74,7 +60,7 @@ command:
>  
>  .. code-block:: shell
>  
> -   $ tox -e py27-django18 patchwork.tests.SubjectCleanUpTest
> +   $ tox -e py36-django21-mysql patchwork.tests.SubjectCleanUpTest
>  
>  Because Patchwork support multiple versions of Django, it's very important that
>  you test against all supported versions. When run without argument, tox will do
> diff --git a/docs/development/installation.rst b/docs/development/installation.rst
> index 0ab755f4..433c3a41 100644
> --- a/docs/development/installation.rst
> +++ b/docs/development/installation.rst
> @@ -86,12 +86,6 @@ To run unit tests against the system Python packages, run:
>  
>     $ docker-compose run --rm web python manage.py test
>  
> -To run unit tests for multiple versions using ``tox``, run:
> -
> -.. code-block:: shell
> -
> -   $ docker-compose run --rm web tox
> -
>  To reset the database before any of these commands, add ``--reset`` to the
>  command line after ``web`` and before any other arguments:
>  
> diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py
> index 53fa58f6..cfb9256c 100644
> --- a/patchwork/settings/dev.py
> +++ b/patchwork/settings/dev.py
> @@ -19,6 +19,18 @@ try:
>  except ImportError:
>      debug_toolbar = None
>  
> +#
> +# tox-docker settings
> +#
> +
> +if 'POSTGRES_5432_TCP' in os.environ:
> +    os.environ['PW_TEST_DB_HOST'] = os.environ['POSTGRES_HOST']
> +    os.environ['PW_TEST_DB_PORT'] = os.environ['POSTGRES_5432_TCP']
> +elif 'MYSQL_3306_TCP' in os.environ:
> +    os.environ['PW_TEST_DB_HOST'] = os.environ['MYSQL_HOST']
> +    os.environ['PW_TEST_DB_PORT'] = os.environ['MYSQL_3306_TCP']
> +
> +
>  #
>  # Core settings
>  # https://docs.djangoproject.com/en/1.11/ref/settings/#core-settings
> diff --git a/tox.ini b/tox.ini
> index d4c34e1c..3c71dafd 100644
> --- a/tox.ini
> +++ b/tox.ini
> @@ -1,6 +1,7 @@
>  [tox]
>  minversion = 2.0
> -envlist = pep8,docs,py{27,34}-django111,py{35,36}-django{111,20,21,22}
> +envlist = pep8,docs,py{27,34}-django111-{mysql,postgres},py{35,36}-django{111,20,21,22}-{mysql,postgres}
> +skip_missing_interpreters = True
>  skipsdist = True
>  
>  [testenv]
> @@ -17,6 +18,9 @@ deps =
>      django22: django>=2.2,<2.3
>      django22: djangorestframework>=3.9.2
>      django22: django-filter>=2.1,<3.0
> +docker =
> +    postgres: postgres:9.6
> +    mysql: mysql:8.0
>  setenv =
>      DJANGO_SETTINGS_MODULE = patchwork.settings.dev
>      PYTHONDONTWRITEBYTECODE = 1
> @@ -24,15 +28,25 @@ setenv =
>      py27: PYTHONWARNINGS = once
>      py{34,36}:PYTHONWARNINGS = once,ignore::ImportWarning:backports
>      py35:PYTHONWARNINGS = once,ignore::ResourceWarning:unittest.suite,ignore::ImportWarning:backports
> +    postgres: PW_TEST_DB_TYPE = postgres
> +    postgres: PW_TEST_DB_USER = postgres
> +    postgres: PW_TEST_DB_PASS = password
> +    mysql: PW_TEST_DB_USER = root
> +    mysql: PW_TEST_DB_PASS = password
>  passenv =
>      http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY
>      PW_TEST_DB_TYPE PW_TEST_DB_USER PW_TEST_DB_PASS PW_TEST_DB_HOST
>      PW_TEST_DB_PORT
> +dockerenv =
> +    postgres: POSTGRES_PASSWORD=password
> +    mysql: MYSQL_ROOT_PASSWORD=password
>  commands =
> -    python {toxinidir}/manage.py test --noinput '{posargs:patchwork}'
> +    python {toxinidir}/manage.py test --noinput {posargs:patchwork}
>  
>  [testenv:bashate]
>  deps = bashate
> +docker =
> +dockerenv =
>  whitelist_externals = bash
>  commands =
>      bash -c "find {toxinidir} \
> @@ -43,6 +57,8 @@ commands =
>  [testenv:pep8]
>  basepython = python2.7
>  deps = flake8
> +docker =
> +dockerenv =
>  commands = flake8 {posargs} patchwork patchwork/bin/pwclient
>  
>  [flake8]
> @@ -57,6 +73,8 @@ exclude = ./patchwork/migrations
>  [testenv:docs]
>  deps =
>      -r{toxinidir}/docs/requirements.txt
> +docker =
> +dockerenv =
>  commands =
>      sphinx-build -E -W -b dirhtml -d docs/_build/doctrees docs docs/_build/html
>  
> @@ -67,9 +85,6 @@ deps =
>      -r{toxinidir}/requirements-prod.txt
>  commands = pylint patchwork --rcfile=pylint.rc
>  
> -[testenv:venv]
> -commands = {posargs}
> -
>  [testenv:coverage]
>  basepython = python2.7
>  deps =
> -- 
> 2.21.0