[PATCH v2 5/9] parser: Handle even more exotically broken headers
Stephen Finucane
stephen at that.guru
Sun Feb 25 23:18:11 AEDT 2018
On Sun, 2018-02-25 at 01:50 +1100, Daniel Axtens wrote:
> An archive of the Ubuntu kernel team mailing list contains a
> fascinating email that causes the following parse error:
>
> email.errors.HeaderParseError: header value appears to contain an embedded header:
> '4Mf^tnii7k\\_EnR5aobBm6Di[DZ9 at AX1wJ"okBdX-UoJ>:SRn]c6DDU"qUIwfs98vF>...
>
> The broken bit seem related to a UTF-8 quoted-printable encoded
> section and to be from an internal attempt to break it over multiple
> lines: here's a snippet from the error message:
> '\n\t=?utf-8?q?Tnf?=\n'
> but interesting the header itself does not contain the new lines, so
> clearly something quite weird is happening behind the scenes!
>
> This only throws on header.encode(): it actually makes it through
> sanitise_header and into find_headers before throwing the assertion.
>
> So, try to encode in sanitize_header as a final step.
>
> Also, fix a hilarious* python bug that this exposes: whitespace-only
> headers cause an index error!
Looks good to me.
Reviewed-by: Stephen Finucane <stephen at that.guru>
> Reviewed-by: Andrew Donnellan <andrew.donnellan at au1.ibm.com>
> Signed-off-by: Daniel Axtens <dja at axtens.net>
> ---
> patchwork/parser.py | 8 +++++
> patchwork/tests/fuzztests/x-face.mbox | 58 +++++++++++++++++++++++++++++++++++
> patchwork/tests/test_parser.py | 7 +++--
> 3 files changed, 71 insertions(+), 2 deletions(-)
> create mode 100644 patchwork/tests/fuzztests/x-face.mbox
>
> diff --git a/patchwork/parser.py b/patchwork/parser.py
> index 2cabb3cbc299..cbf88fe4e464 100644
> --- a/patchwork/parser.py
> +++ b/patchwork/parser.py
> @@ -129,6 +129,14 @@ def sanitise_header(header_contents, header_name=None):
> header_name=header_name,
> continuation_ws='\t')
>
> + try:
> + header.encode()
> + except (HeaderParseError, IndexError):
> + # despite our best efforts, the header is stuffed
> + # HeaderParseError: some very weird multi-line headers
> + # IndexError: bug, thrown by make_header(decode_header(' ')).encode()
> + return None
> +
> return header
>
>
> diff --git a/patchwork/tests/fuzztests/x-face.mbox b/patchwork/tests/fuzztests/x-face.mbox
> new file mode 100644
> index 000000000000..98019cff8250
> --- /dev/null
> +++ b/patchwork/tests/fuzztests/x-face.mbox
> @@ -0,0 +1,58 @@
> +From laurent.pinchart at skynet.be Thu Nov 13 15:54:10 2008
> +Received: from mailrelay005.isp.belgacom.be ([195.238.6.171])
> + by chlorine.canonical.com with esmtp (Exim 4.60)
> + (envelope-from <laurent.pinchart at skynet.be>) id 1L0eWI-0007oB-7K
> + for kernel-team at lists.ubuntu.com; Thu, 13 Nov 2008 15:54:10 +0000
> +X-IronPort-Anti-Spam-Filtered: true
> +X-IronPort-Anti-Spam-Result: ApsEAP/aG0nCTsYx/2dsb2JhbACBds9Hg1c
> +Received: from 49.198-78-194.adsl-static.isp.belgacom.be (HELO
> + laptop-laurent.belgium.cse-semaphore.com) ([194.78.198.49])
> + by relay.skynet.be with ESMTP; 13 Nov 2008 16:54:09 +0100
> +From: Laurent Pinchart <laurent.pinchart at skynet.be>
> +To: kernel-team at lists.ubuntu.com
> +Subject: uvcvideo (webcam) support for COMPAL JHL90 based laptops
> +Date: Thu, 13 Nov 2008 16:54:22 +0100
> +User-Agent: KMail/1.9.9
> +X-Face: 4Mf^tnii7k\_EnR5aobBm6Di[DZ9 at AX1wJ"okBdX-UoJ>:SRn]c6DDU"qUIwfs98vF>=?utf-8?q?Tnf=0A=09SacR=7B?=(0Du"N%_.#X]"TXx)A'gKB1i7SK$CTLuy{h})c=g:'w3
> +MIME-Version: 1.0
> +Content-Type: text/plain;
> + charset="us-ascii"
> +Content-Transfer-Encoding: 7bit
> +Content-Disposition: inline
> +Message-Id: <200811131654.22389.laurent.pinchart at skynet.be>
> +X-Mailman-Approved-At: Fri, 14 Nov 2008 14:54:06 +0000
> +Cc: amit at ubuntu.com
> +X-BeenThere: kernel-team at lists.ubuntu.com
> +X-Mailman-Version: 2.1.8
> +Precedence: list
> +List-Id: Kernel team discussions <kernel-team.lists.ubuntu.com>
> +List-Unsubscribe: <https://lists.ubuntu.com/mailman/listinfo/kernel-team>,
> + <mailto:kernel-team-request at lists.ubuntu.com?subject=unsubscribe>
> +List-Archive: <https://lists.ubuntu.com/archives/kernel-team>
> +List-Post: <mailto:kernel-team at lists.ubuntu.com>
> +List-Help: <mailto:kernel-team-request at lists.ubuntu.com?subject=help>
> +List-Subscribe: <https://lists.ubuntu.com/mailman/listinfo/kernel-team>,
> + <mailto:kernel-team-request at lists.ubuntu.com?subject=subscribe>
> +X-List-Received-Date: Thu, 13 Nov 2008 15:54:10 -0000
> +
> +Hi Amit,
> +
> +I've noticed by sheer luck that the Ubuntu 8.10 linux-image-2.6.27-7 packages
> +include a patch to the uvcvideo driver to support webcam modules integrated
> +into Compal JHL90 laptops.
> +
> +Is there a reason why the patch hasn't been pushed upstream ? Being the
> +uvcvideo author and maintainer, I'd appreciate if you could forward patches
> +related to the driver in the future.
> +
> +On a pure technical note, the patch might not be required with the current
> +uvcvideo driver version. There is no way to confirm this without testing the
> +driver with that particular webcam model, so I'd appreciate if you could
> +contact the patch author and check with him if his camera works with the
> +latest uvcvideo driver (available from http://linux-uvc.berlios.de/).
> +
> +Best regards,
> +
> +Laurent Pinchart
> +
> +
> diff --git a/patchwork/tests/test_parser.py b/patchwork/tests/test_parser.py
> index 20d70af12120..68bcb937b273 100644
> --- a/patchwork/tests/test_parser.py
> +++ b/patchwork/tests/test_parser.py
> @@ -896,8 +896,8 @@ class SubjectTest(TestCase):
> self.assertEqual(parse_version('Hello, world (V6)', []), 6)
>
>
> -class FuzzTest(TransactionTestCase):
> - """Test fuzzed patches."""
> +class WeirdMailTest(TransactionTestCase):
Heh :)
> + """Test fuzzed or otherwise weird patches."""
> def setUp(self):
> create_project(listid='patchwork.ozlabs.org')
>
> @@ -940,3 +940,6 @@ class FuzzTest(TransactionTestCase):
> self._test_patch('refshdr.mbox')
> self._test_patch('dateheader.mbox')
> self._test_patch('msgidheader.mbox')
> +
> + def test_x_face(self):
> + self._test_patch('x-face.mbox')
More information about the Patchwork
mailing list