[PATCH v3 00/10] Convert 'parse' scripts to management commands

[Stephen Finucane]

The parsemail and parsearchive scripts are currently the main way of
loading content into Patchwork. However, running them requires modifying
the PYTHONPATH environment variable, along with calls to configure
Django before doing anything.

Django describes management commands as a good option for standalone
scripts, which make it a good fit for these two scripts. Migrate these
to management commands, moving the main bulk of the parsing code into a
dedicated parsing module. As part of this, we fix some bugs and
refactor some of the cruftier code.

This sets us up nicely for adding an '/upload' endpoint at some point in
the future, which could allow Patchwork to be deployed on PaaS systems.
These systems don't have access to the underlying environment, meaning
something like a Unix mailbox might not be available. An API may well be
the only way to submit content in this area.

Changes since v2:
- Allow additional parameters to parsemail management command

Stephen Finucane (10):
  trivial: Cleanup of 'parser'
  parser: Remove 'main' function
  parser: Move hash_diff and extract_tags
  parsemail: Move parsing to 'parser'
  parser: Put more specific 'From' regex first
  parser: Ensure consistent use of 'parse_'
  tests: Additional rework of 'test_parser'
  parser: Correct empty email value check
  parsemail: Convert to a management command
  parsearchive: Convert to a management command

 docs/development.md                           |   8 +-
 patchwork/bin/parsearchive.py                 | 106 -----
 patchwork/bin/parsemail.py                    | 637 -------------------------
 patchwork/bin/parsemail.sh                    |   4 +-
 patchwork/management/commands/parsearchive.py |  87 ++++
 patchwork/management/commands/parsemail.py    |  61 +++
 patchwork/models.py                           |  66 ++-
 patchwork/parser.py                           | 651 +++++++++++++++++++++-----
 patchwork/settings/base.py                    |  60 ++-
 patchwork/tests/test_parser.py                | 198 ++++----
 patchwork/tests/test_tags.py                  |   3 +-
 11 files changed, 923 insertions(+), 958 deletions(-)
 delete mode 100755 patchwork/bin/parsearchive.py
 delete mode 100755 patchwork/bin/parsemail.py
 create mode 100644 patchwork/management/commands/parsearchive.py
 create mode 100644 patchwork/management/commands/parsemail.py

-- 
2.7.4