[PATCH 1/3] Add export project as mbox management command

[Stephen Finucane]

On Thu, 2019-08-22 at 21:21 +1000, Daniel Axtens wrote:
> Hi Stephen,
> 
> It seems you've merged just patch 1 - was this intentional?
> 
> https://github.com/getpatchwork/patchwork/commit/b2d22fc59ba01a4d21fbc0e038739252aa7dfc3b

It was. I thought I'd replied, but clearly not. I squashed these three
patches together, renamed the command to 'dumparchive', and removed the
knob to configure the level of compression in favour of a simple '
--compress' flag. I'll update patches in Patchwork now.

Stephen

> Regards,
> Daniel
> 
> > On Mon, 2019-07-15 at 16:50 +0200, Mete Polat wrote:
> > > Introduces a new management command which can export all patches in a
> > > project as one mbox file. Export of multiple projects is supported.
> > > Additionaly allows to compress the output.
> > 
> > This looks good. Just two small comments below.
> > 
> > > Signed-off-by: Mete Polat <metepolat2000 at gmail.com>
> > > ---
> > > Also supports python2 with the force_bytes() function.
> > > 
> > >  .../management/commands/exportproject.py      | 71 +++++++++++++++++++
> > >  1 file changed, 71 insertions(+)
> > >  create mode 100644 patchwork/management/commands/exportproject.py
> > > 
> > > diff --git a/patchwork/management/commands/exportproject.py b/patchwork/management/commands/exportproject.py
> > > new file mode 100644
> > > index 0000000..7e18234
> > > --- /dev/null
> > > +++ b/patchwork/management/commands/exportproject.py
> > > @@ -0,0 +1,71 @@
> > > +# Patchwork - automated patch tracking system
> > > +# Copyright (C) 2019, Bayerische Motoren Werke Aktiengesellschaft (BMW AG) 
> > > +#
> > > +# SPDX-License-Identifier: GPL-2.0-or-later
> > > +
> > > +import os
> > > +import tarfile
> > > +from uuid import uuid4
> > > +
> > > +from django.core.management import BaseCommand, CommandError
> > > +from django.utils.encoding import force_bytes
> > > +
> > > +from patchwork.models import Project, Patch
> > > +from patchwork.views.utils import patch_to_mbox
> > > +
> > > +
> > > +class Command(BaseCommand):
> > > +    help = 'Export patchwork projects as mbox files and optionally compress '
> > > +    'the result.'
> > > +
> > > +    def add_arguments(self, parser):
> > > +        parser.add_argument(
> > > +            '-c', '--compress', action='store_true',
> > > +            help='Bundle and compress projects.'
> > > +        )
> > > +        parser.add_argument(
> > > +            '-l', '--level', action='store', type=int, default=9,
> > > +            help='Set a compression level between 0 and 9 (default). 0 is no '
> > > +                 'compression. '
> > > +        )
> > 
> > Do we need this knob? It seems defaulting to 9 if the '-c' flag is
> > provided would be fine for 90% of users?
> > 
> > > +        parser.add_argument(
> > > +            'project_linkname', nargs='*',
> > > +            help='Project linknames. Export all projects if none specified.'
> > > +        )
> > > +
> > > +    def handle(self, *args, **options):
> > > +        if options['project_linkname']:
> > > +            projects = []
> > > +            for p_linkname in options['project_linkname']:
> > > +                try:
> > > +                    projects.append(Project.objects.get(linkname=p_linkname))
> > > +                except Project.DoesNotExist:
> > > +                    raise CommandError('%s: Project not found' % p_linkname)
> > > +        else:
> > > +            projects = Project.objects.all()
> > > +
> > > +        compress = options['compress']
> > > +        level = options['level']
> > > +
> > > +        tar = None
> > > +        if compress:
> > > +            name = projects[0].linkname if len(projects) == 1 else 'patchwork'
> > 
> > Any reason not to use 'projects[0].linkname' if len(projects) == 1 too?
> > 
> > > +            name += '.tar' if level == 0 else '.tar.gz'
> > > +            tar = tarfile.open(name, 'w:gz', compresslevel=level)
> > > +
> > > +        try:
> > > +            for project in projects:
> > > +                name = project.linkname + '.mbox'
> > > +                tmp_name = '%s_%s' % (project.linkname, uuid4().hex)
> > > +                with open(tmp_name, 'wb') as mbox:
> > > +                    for patch in Patch.objects.filter(patch_project=project):
> > > +                        mbox.write(force_bytes(patch_to_mbox(patch) + '\n'))
> > > +                    mbox.close()
> > > +                    if compress:
> > > +                        tar.add(tmp_name, arcname=name)
> > > +                        os.remove(tmp_name)
> > > +                    else:
> > > +                        os.rename(tmp_name, name)
> > > +        finally:
> > > +            if tar is not None:
> > > +                tar.close()
> > 
> > _______________________________________________
> > Patchwork mailing list
> > Patchwork at lists.ozlabs.org
> > https://lists.ozlabs.org/listinfo/patchwork