[PATCH 00/10] Fuzzing the parser: fixes

[Daniel Axtens]

Hi all,

I fuzzed parsemail with python-afl/afl-fuzz.

Predictably, it did not hold up well. Nothing major, just a bunch of
edge cases. Date parsing held up particularly poorly.

Currently, if we hit any of the errors this patch set fixes, the
entire mail will be rejected. With this patch set, the emails have a
much better chance of surviving. This is helpful for us - we have
previously had bug reports where a stray mail has been rejected due to
a corrupt header, which is suboptimal.

Test cases are included. They may not email particularly well so I
have also put them up on my github as a signed tag:
https://github.com/daxtens/patchwork/releases/tag/fuzz-testing

For the interested, the bulk of the fuzzing was done on an AWS cloud
instance, using something quite similar to the standard docker-compose
setup. Several hundered thousand executions were done - nowhere near
what I'd like but I can't seem to get execution speed above about 1
execution per second. If I was able to confidently mock out the db or
extract the parser out of the django framework it would go faster, but
both of those were a bit more than I was easily able to do. I also
picked up some bugs when interfacing with the db (patch 7) so I feel
like this approach was vindicated.

I also included a patch to allow people to replicate my setup. It's
not really ready to merge and I'm not convinced it's really necessary
as it's not a lot of work to replicate.

Regards,
Daniel

Daniel Axtens (10):
  parser: fix charset 'guessing' algorithm
  parser: don't assume headers are strings
  parser: codec lookup fails when a NUL (\x00) is in the name
  parser: catch failures in decoding headers
  parser: deal with headers entirely failing to parse
  parser: better date parsing
  parser: Don't pass a message-id longer than 255 chars to the db
  parse(mail|archive): handle early fail within email module
  Add fuzzer-generated tests
  [RFC] Fuzzing harness

 patchwork/management/commands/fuzz.py            |  88 +++++++++++++++
 patchwork/management/commands/parsearchive.py    |   9 ++
 patchwork/management/commands/parsemail.py       |  31 +++---
 patchwork/parser.py                              | 126 +++++++++++++++++-----
 patchwork/tests/__init__.py                      |   1 +
 patchwork/tests/fuzztests/base64err.mbox         |  46 ++++++++
 patchwork/tests/fuzztests/charset.mbox           | 131 +++++++++++++++++++++++
 patchwork/tests/fuzztests/codec-null.mbox        | Bin 0 -> 8192 bytes
 patchwork/tests/fuzztests/date-oserror.mbox      | Bin 0 -> 8209 bytes
 patchwork/tests/fuzztests/date-too-long.mbox     | Bin 0 -> 1828 bytes
 patchwork/tests/fuzztests/date.mbox              |  44 ++++++++
 patchwork/tests/fuzztests/dateheader.mbox        | Bin 0 -> 580 bytes
 patchwork/tests/fuzztests/earlyfail.mbox         | Bin 0 -> 1712 bytes
 patchwork/tests/fuzztests/msgid-len.mbox         | Bin 0 -> 1809 bytes
 patchwork/tests/fuzztests/msgid-len2.mbox        |  37 +++++++
 patchwork/tests/fuzztests/msgidheader.mbox       | 131 +++++++++++++++++++++++
 patchwork/tests/fuzztests/refshdr.mbox           | Bin 0 -> 816 bytes
 patchwork/tests/fuzztests/unknown-encoding.mbox  | Bin 0 -> 1751 bytes
 patchwork/tests/fuzztests/value2.mbox            | Bin 0 -> 1598 bytes
 patchwork/tests/fuzztests/year-out-of-range.mbox | Bin 0 -> 1660 bytes
 patchwork/tests/test_parser.py                   |  60 ++++++++++-
 tools/docker/Dockerfile                          |   2 +
 tools/fuzzer_dict                                |  52 +++++++++
 23 files changed, 714 insertions(+), 44 deletions(-)
 create mode 100644 patchwork/management/commands/fuzz.py
 create mode 100644 patchwork/tests/fuzztests/base64err.mbox
 create mode 100644 patchwork/tests/fuzztests/charset.mbox
 create mode 100644 patchwork/tests/fuzztests/codec-null.mbox
 create mode 100644 patchwork/tests/fuzztests/date-oserror.mbox
 create mode 100644 patchwork/tests/fuzztests/date-too-long.mbox
 create mode 100644 patchwork/tests/fuzztests/date.mbox
 create mode 100644 patchwork/tests/fuzztests/dateheader.mbox
 create mode 100644 patchwork/tests/fuzztests/earlyfail.mbox
 create mode 100644 patchwork/tests/fuzztests/msgid-len.mbox
 create mode 100644 patchwork/tests/fuzztests/msgid-len2.mbox
 create mode 100644 patchwork/tests/fuzztests/msgidheader.mbox
 create mode 100644 patchwork/tests/fuzztests/refshdr.mbox
 create mode 100644 patchwork/tests/fuzztests/unknown-encoding.mbox
 create mode 100644 patchwork/tests/fuzztests/value2.mbox
 create mode 100644 patchwork/tests/fuzztests/year-out-of-range.mbox
 create mode 100644 tools/fuzzer_dict

-- 
2.11.0