[PATCH 1/3] Add export project as mbox management command

[Mete Polat]

On 15.07.19 17:39, Stephen Finucane wrote:
> On Mon, 2019-07-15 at 16:50 +0200, Mete Polat wrote:
>> Introduces a new management command which can export all patches in a
>> project as one mbox file. Export of multiple projects is supported.
>> Additionaly allows to compress the output.
> This looks good. Just two small comments below.
>
>> Signed-off-by: Mete Polat <metepolat2000 at gmail.com>
>> ---
>> Also supports python2 with the force_bytes() function.
>>
>>   .../management/commands/exportproject.py      | 71 +++++++++++++++++++
>>   1 file changed, 71 insertions(+)
>>   create mode 100644 patchwork/management/commands/exportproject.py
>>
>> diff --git a/patchwork/management/commands/exportproject.py b/patchwork/management/commands/exportproject.py
>> new file mode 100644
>> index 0000000..7e18234
>> --- /dev/null
>> +++ b/patchwork/management/commands/exportproject.py
>> @@ -0,0 +1,71 @@
>> +# Patchwork - automated patch tracking system
>> +# Copyright (C) 2019, Bayerische Motoren Werke Aktiengesellschaft (BMW AG)
>> +#
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +
>> +import os
>> +import tarfile
>> +from uuid import uuid4
>> +
>> +from django.core.management import BaseCommand, CommandError
>> +from django.utils.encoding import force_bytes
>> +
>> +from patchwork.models import Project, Patch
>> +from patchwork.views.utils import patch_to_mbox
>> +
>> +
>> +class Command(BaseCommand):
>> +    help = 'Export patchwork projects as mbox files and optionally compress '
>> +    'the result.'
>> +
>> +    def add_arguments(self, parser):
>> +        parser.add_argument(
>> +            '-c', '--compress', action='store_true',
>> +            help='Bundle and compress projects.'
>> +        )
>> +        parser.add_argument(
>> +            '-l', '--level', action='store', type=int, default=9,
>> +            help='Set a compression level between 0 and 9 (default). 0 is no '
>> +                 'compression. '
>> +        )
> Do we need this knob? It seems defaulting to 9 if the '-c' flag is
> provided would be fine for 90% of users?

Indeed, fine-tuning the compression level will be hardly used but 
especially for
large projects turning off compression with '-l 0' can save time when 
there is
no need for sharing the output.

>> +        parser.add_argument(
>> +            'project_linkname', nargs='*',
>> +            help='Project linknames. Export all projects if none specified.'
>> +        )
>> +
>> +    def handle(self, *args, **options):
>> +        if options['project_linkname']:
>> +            projects = []
>> +            for p_linkname in options['project_linkname']:
>> +                try:
>> +                    projects.append(Project.objects.get(linkname=p_linkname))
>> +                except Project.DoesNotExist:
>> +                    raise CommandError('%s: Project not found' % p_linkname)
>> +        else:
>> +            projects = Project.objects.all()
>> +
>> +        compress = options['compress']
>> +        level = options['level']
>> +
>> +        tar = None
>> +        if compress:
>> +            name = projects[0].linkname if len(projects) == 1 else 'patchwork'
> Any reason not to use 'projects[0].linkname' if len(projects) == 1 too?

Do you mean 'len(projects) != 1'? It can be confusing if we use the name of
the first project as the archive name when there is more than one 
project in it.

>> +            name += '.tar' if level == 0 else '.tar.gz'
>> +            tar = tarfile.open(name, 'w:gz', compresslevel=level)
>> +
>> +        try:
>> +            for project in projects:
>> +                name = project.linkname + '.mbox'
>> +                tmp_name = '%s_%s' % (project.linkname, uuid4().hex)
>> +                with open(tmp_name, 'wb') as mbox:
>> +                    for patch in Patch.objects.filter(patch_project=project):
>> +                        mbox.write(force_bytes(patch_to_mbox(patch) + '\n'))
>> +                    mbox.close()
>> +                    if compress:
>> +                        tar.add(tmp_name, arcname=name)
>> +                        os.remove(tmp_name)
>> +                    else:
>> +                        os.rename(tmp_name, name)
>> +        finally:
>> +            if tar is not None:
>> +                tar.close()