[PULL] Improvements to message parsing

[Daniel Axtens]

Hi Stephen,

Please consider pulling from the signed tag below for any patches that got
lost on the mailing list.

Regards,
Daniel

The following changes since commit c0dae9bac11644bd98c60e7c817f5047b8440d08:

  ui: Shrink vertical space (2016-09-25 22:34:08 +0100)

are available in the git repository at:

  https://github.com/daxtens/patchwork tags/parse-improvements-20160928

for you to fetch changes up to bb0627207ff5435ff9bc6861c4c85ffa702adab9:

  parsemail: Convert to a management command (2016-09-28 15:20:09 +1000)

----------------------------------------------------------------
Improvements to message parsing

This includes:
 - some fixes to make testing easier by passing around command-line options
 - fixes for headers with invalid characters
 - A modified version of Stephen's parsemail managment command migration
 - more tests

This is version 2.

----------------------------------------------------------------
Daniel Axtens (7):
      parsemail.sh: make testing easier
      parsemail-batch.sh: pass command-line options to parsemail
      Test for header preservation
      parser: parse headers containing invalid characters or codings
      parser: subject_check needs to call clean_header
      parser tests: test parsing of interesting mail
      parsemail: Convert to a management command

 patchwork/bin/parsemail-batch.sh                   |   8 +-
 patchwork/bin/parsemail.py                         | 114 ------------------
 patchwork/bin/parsemail.sh                         |  17 ++-
 patchwork/management/commands/parsemail.py         |  84 +++++++++++++
 patchwork/parser.py                                | 107 +++++++++++++----
 patchwork/settings/base.py                         |  60 +++++++++-
 patchwork/tests/__init__.py                        |  23 ++++
 patchwork/tests/mail/0012-invalid-header-char.mbox |  72 +++++++++++
 patchwork/tests/mail/0013-with-utf8-body.mbox      |  46 +++++++
 .../mail/0014-with-unencoded-utf8-headers.mbox     |  46 +++++++
 .../tests/mail/0015-with-invalid-utf8-headers.mbox |  46 +++++++
 patchwork/tests/test_management.py                 | 112 +++++++++++++++++
 patchwork/tests/test_parser.py                     | 133 ++++++++++++++++++++-
 patchwork/tests/utils.py                           |   2 +-
 14 files changed, 719 insertions(+), 151 deletions(-)
 delete mode 100755 patchwork/bin/parsemail.py
 create mode 100644 patchwork/management/commands/parsemail.py
 create mode 100644 patchwork/tests/mail/0012-invalid-header-char.mbox
 create mode 100644 patchwork/tests/mail/0013-with-utf8-body.mbox
 create mode 100644 patchwork/tests/mail/0014-with-unencoded-utf8-headers.mbox
 create mode 100644 patchwork/tests/mail/0015-with-invalid-utf8-headers.mbox
 create mode 100644 patchwork/tests/test_management.py
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 859 bytes
Desc: not available
URL: <http://lists.ozlabs.org/pipermail/patchwork/attachments/20160928/42f706b4/attachment.sig>