[PATCH 1/3] Add export project as mbox management command

[Daniel Axtens]

Stephen Finucane <stephen at that.guru> writes:

> On Thu, 2019-08-22 at 21:21 +1000, Daniel Axtens wrote:
>> Hi Stephen,
>> 
>> It seems you've merged just patch 1 - was this intentional?
>> 
>> https://github.com/getpatchwork/patchwork/commit/b2d22fc59ba01a4d21fbc0e038739252aa7dfc3b
>
> It was. I thought I'd replied, but clearly not. I squashed these three
> patches together, renamed the command to 'dumparchive', and removed the
> knob to configure the level of compression in favour of a simple '
> --compress' flag. I'll update patches in Patchwork now.

Cool :)

Thanks,
Daniel
>
> Stephen
>
>> Regards,
>> Daniel
>> 
>> > On Mon, 2019-07-15 at 16:50 +0200, Mete Polat wrote:
>> > > Introduces a new management command which can export all patches in a
>> > > project as one mbox file. Export of multiple projects is supported.
>> > > Additionaly allows to compress the output.
>> > 
>> > This looks good. Just two small comments below.
>> > 
>> > > Signed-off-by: Mete Polat <metepolat2000 at gmail.com>
>> > > ---
>> > > Also supports python2 with the force_bytes() function.
>> > > 
>> > >  .../management/commands/exportproject.py      | 71 +++++++++++++++++++
>> > >  1 file changed, 71 insertions(+)
>> > >  create mode 100644 patchwork/management/commands/exportproject.py
>> > > 
>> > > diff --git a/patchwork/management/commands/exportproject.py b/patchwork/management/commands/exportproject.py
>> > > new file mode 100644
>> > > index 0000000..7e18234
>> > > --- /dev/null
>> > > +++ b/patchwork/management/commands/exportproject.py
>> > > @@ -0,0 +1,71 @@
>> > > +# Patchwork - automated patch tracking system
>> > > +# Copyright (C) 2019, Bayerische Motoren Werke Aktiengesellschaft (BMW AG) 
>> > > +#
>> > > +# SPDX-License-Identifier: GPL-2.0-or-later
>> > > +
>> > > +import os
>> > > +import tarfile
>> > > +from uuid import uuid4
>> > > +
>> > > +from django.core.management import BaseCommand, CommandError
>> > > +from django.utils.encoding import force_bytes
>> > > +
>> > > +from patchwork.models import Project, Patch
>> > > +from patchwork.views.utils import patch_to_mbox
>> > > +
>> > > +
>> > > +class Command(BaseCommand):
>> > > +    help = 'Export patchwork projects as mbox files and optionally compress '
>> > > +    'the result.'
>> > > +
>> > > +    def add_arguments(self, parser):
>> > > +        parser.add_argument(
>> > > +            '-c', '--compress', action='store_true',
>> > > +            help='Bundle and compress projects.'
>> > > +        )
>> > > +        parser.add_argument(
>> > > +            '-l', '--level', action='store', type=int, default=9,
>> > > +            help='Set a compression level between 0 and 9 (default). 0 is no '
>> > > +                 'compression. '
>> > > +        )
>> > 
>> > Do we need this knob? It seems defaulting to 9 if the '-c' flag is
>> > provided would be fine for 90% of users?
>> > 
>> > > +        parser.add_argument(
>> > > +            'project_linkname', nargs='*',
>> > > +            help='Project linknames. Export all projects if none specified.'
>> > > +        )
>> > > +
>> > > +    def handle(self, *args, **options):
>> > > +        if options['project_linkname']:
>> > > +            projects = []
>> > > +            for p_linkname in options['project_linkname']:
>> > > +                try:
>> > > +                    projects.append(Project.objects.get(linkname=p_linkname))
>> > > +                except Project.DoesNotExist:
>> > > +                    raise CommandError('%s: Project not found' % p_linkname)
>> > > +        else:
>> > > +            projects = Project.objects.all()
>> > > +
>> > > +        compress = options['compress']
>> > > +        level = options['level']
>> > > +
>> > > +        tar = None
>> > > +        if compress:
>> > > +            name = projects[0].linkname if len(projects) == 1 else 'patchwork'
>> > 
>> > Any reason not to use 'projects[0].linkname' if len(projects) == 1 too?
>> > 
>> > > +            name += '.tar' if level == 0 else '.tar.gz'
>> > > +            tar = tarfile.open(name, 'w:gz', compresslevel=level)
>> > > +
>> > > +        try:
>> > > +            for project in projects:
>> > > +                name = project.linkname + '.mbox'
>> > > +                tmp_name = '%s_%s' % (project.linkname, uuid4().hex)
>> > > +                with open(tmp_name, 'wb') as mbox:
>> > > +                    for patch in Patch.objects.filter(patch_project=project):
>> > > +                        mbox.write(force_bytes(patch_to_mbox(patch) + '\n'))
>> > > +                    mbox.close()
>> > > +                    if compress:
>> > > +                        tar.add(tmp_name, arcname=name)
>> > > +                        os.remove(tmp_name)
>> > > +                    else:
>> > > +                        os.rename(tmp_name, name)
>> > > +        finally:
>> > > +            if tar is not None:
>> > > +                tar.close()
>> > 
>> > _______________________________________________
>> > Patchwork mailing list
>> > Patchwork at lists.ozlabs.org
>> > https://lists.ozlabs.org/listinfo/patchwork