[PATCH v6 1/2] parsemail: Convert to a management command

[Daniel Axtens]

So, umm, I went ahead and had a crack at this.

It turns out this is hideously difficult to get right. But this plus my
other patch to fix Thomas' problem should have things working on Py2 and
Py3 with this series.

It's a bit of a work in progress: I need to close the file at the end
of the function, the logging needs to be added again, etc.

Tests to come.

Stephen: I can do this up into a proper patch if you like or you can
fold it into your series.

Regards,
Daniel

diff --git a/patchwork/management/commands/parsemail.py b/patchwork/management/commands/parsemail.py
index a60e2ad11f06..2b957473167f 100644
--- a/patchwork/management/commands/parsemail.py
+++ b/patchwork/management/commands/parsemail.py
@@ -18,10 +18,11 @@
 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 
 import argparse
-from email import message_from_file
+import email
 import logging
 from optparse import make_option
 import sys
+import codecs
 
 import django
 from django.core.management import base
@@ -48,9 +49,9 @@ class Command(base.BaseCommand):
             parser.add_argument(
                 'infile',
                 nargs='?',
-                type=argparse.FileType('r'),
-                default=sys.stdin,
-                help='input mbox file (a filename or stdin)')
+                type=str,
+                default=None,
+                help='input mbox file (stdin if not provided)')
             parser.add_argument(
                 '--list-id',
                 help='mailing list ID. If not supplied, this will be '
@@ -59,17 +60,19 @@ class Command(base.BaseCommand):
     def handle(self, *args, **options):
         infile = args[0] if args else options['infile']
 
-        # Attempt to parse the path if provided, and fallback to stdin if not
-        if infile and isinstance(infile, six.string_types):  # Django < 1.8
-            logger.info('Parsing mail loaded by filename')
-            with open(infile, 'r+') as file_:
-                mail = message_from_file(file_)
+        if six.PY3:
+            if infile:
+                file_ = open(infile, 'r', encoding='utf-8', errors='replace')
+            else:
+                file_ = codecs.getreader('utf-8')(sys.stdin.buffer, errors='replace')
         else:
-            if infile == sys.stdin:
-                logger.info('Parsing mail loaded from stdin')
-            else:  # Djano >= 1.8
-                logger.info('Parsing mail loaded by filename')
-            mail = message_from_file(infile)
+            if infile:
+                file_ = open(infile, 'r')
+            else:
+                file_ = codecs.getreader('utf-8')(sys.stdin, errors='replace')
+
+        mail = email.message_from_file(file_)
+
         try:
             result = parse_mail(mail, options['list_id'])
             if result:
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 859 bytes
Desc: not available
URL: <http://lists.ozlabs.org/pipermail/patchwork/attachments/20160920/bab06a7a/attachment.sig>