[PATCH 4/9] tools/scripts: parallel_parsearchive - load archives in parallel

[Andrew Donnellan]

On 22/02/18 01:17, Daniel Axtens wrote:
> If you have multiple archives, you quickly tire of typing stuff like
> python3 manage.py parsearchive --list-id=patchwork.ozlabs.org foo-1 &
> python3 manage.py parsearchive --list-id=patchwork.ozlabs.org foo-2 &
> python3 manage.py parsearchive --list-id=patchwork.ozlabs.org foo-3 &
> python3 manage.py parsearchive --list-id=patchwork.ozlabs.org foo-4 &
> and having to copy and paste it - or retype it! - each time you reset
> the database.
> 
> Instead, this patch allows you to do
> tools/scripts/parallel_parsearchive.sh --list-id=patchwork.ozlabs.org -- foo-*
> 
> Much easier, especially when you are doing it a dozen times.
> 
> Signed-off-by: Daniel Axtens <dja at axtens.net>

The overengineering approach to this would be to make the parsearchive 
command support -j and work around python's broken threading model and 
all ;) For the purposes, this solution will work fine though...

(Also, who was it that said Greenspun's tenth rule, but with GNU parallel?)

One comment below, with that fixed:

Reviewed-by: Andrew Donnellan <andrew.donnellan at au1.ibm.com>

> ---
>   tools/scripts/parallel_parsearchive.sh | 55 ++++++++++++++++++++++++++++++++++
>   1 file changed, 55 insertions(+)
>   create mode 100755 tools/scripts/parallel_parsearchive.sh
> 
> diff --git a/tools/scripts/parallel_parsearchive.sh b/tools/scripts/parallel_parsearchive.sh
> new file mode 100755
> index 000000000000..a5197f6c2c5f
> --- /dev/null
> +++ b/tools/scripts/parallel_parsearchive.sh
> @@ -0,0 +1,55 @@
> +#!/bin/bash
> +# Patchwork - automated patch tracking system
> +# Copyright (C) 2018 Daniel Axtens <dja at axtens.net>
> +#
> +# This file is part of the Patchwork package.
> +#
> +# Patchwork is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# Patchwork is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +
> +set -euo pipefail
> +
> +usage() {
> +    cat <<EOF
> +parallel_parsearchive.sh - load archives in parallel
> +Usage:
> +  parallel_parsearchive.sh [parsearchive options] -- <archives>
> +  The -- is mandatory.
> +  As many processes as there are archives will be spun up.
> +
> +Example:
> +EOF

Did you mean to put an example here?

-- 
Andrew Donnellan              OzLabs, ADL Canberra
andrew.donnellan at au1.ibm.com  IBM Australia Limited