[PATCH v2 0/5] Update REST API: Add 'project patches as mbox' field

[Andrew Donnellan]

On 2/7/19 10:26 am, Daniel Axtens wrote:
> So there are two possible complimenatry approaches I can think of:
> 
>   - gather the data from a download of the mailing list that patchwork
>     injests. For LKML you can get this from
>     https://www.kernel.org/lore.html, for example.
>     You could then pass this through a local patchwork instance. (Let me
>     know if you want my scripts for importing a public-archive git repo
>     into patchwork.)
> 
>   - add a management command to export a project as an mbox and then
>     coordinate with patchwork admins at the instance you're interested in
>     to run the export at a time that suits them and provide you with a
>     heavily compressed copy of the output.
> 
If we were to add an API for this kind of bulk mbox export, I think it 
would need to export a fixed number of emails at a time (100 or 250 or 
something like that).

For patchwork setups where the raw mailing list data isn't easily 
retrievable, a management command to export the whole project as mbox 
could be extended fairly easily to "export the past N days of the 
project as mbox". Then just put that in your crontab and have it export 
a new archive every so often, which you compress and then serve up 
statically.

> Regards,
> Daniel
> 
> 
>>
>> Regards,
>>
>> Mete
>>>> Updated REST API version to 1.2
>>>>
>>>> New REST API url: api/1.2/
>>>> New internal url: project/<project_id>/list/mbox/
>>>> New project rest api field: patches_mbox
>>>>
>>>> Like patchwork.example.com/project/<project_id>/list/ the final project.mbox
>>>> only includes patches. Careful naming has been made in case the api should
>>>> support export of a whole project in the future as well.
>>>>
>>>> Changes since v1:
>>>> - Add missing url for api version 1.2
>>>>
>>>> Mete Polat (5):
>>>>    Add option to get all project patches in one mbox
>>>>    Add urls to get all project patches in one mbox
>>>>    Add api endpoint for project patches as mbox
>>>>    Update api documentation for v1.2
>>>>    Add release notes: project patches as mbox
>>>>
>>>>   docs/api/rest/index.rst                       |   50 +-
>>>>   docs/api/rest/schemas/v1.1.rst                |    4 +-
>>>>   docs/api/rest/schemas/v1.2.rst                |    5 +
>>>>   docs/api/schemas/generate_schema.py           |    4 +-
>>>>   docs/api/schemas/latest/patchwork.yaml        |    7 +-
>>>>   docs/api/schemas/patchwork.j2                 |    7 +
>>>>   docs/api/schemas/v1.2/patchwork.yaml          | 2319 +++++++++++++++++
>>>>   patchwork/api/project.py                      |   11 +-
>>>>   patchwork/models.py                           |   25 +-
>>>>   patchwork/urls.py                             |    6 +-
>>>>   patchwork/views/patch.py                      |   12 +
>>>>   patchwork/views/utils.py                      |   13 +
>>>>   ...project-patches-mbox-623f8c9d4cf6a952.yaml |    6 +
>>>>   13 files changed, 2428 insertions(+), 41 deletions(-)
>>>>   create mode 100644 docs/api/rest/schemas/v1.2.rst
>>>>   create mode 100644 docs/api/schemas/v1.2/patchwork.yaml
>>>>   create mode 100644 releasenotes/notes/project-patches-mbox-623f8c9d4cf6a952.yaml
>>>>
>>>> -- 
>>>> 2.22.0
>>>>
>>>> _______________________________________________
>>>> Patchwork mailing list
>>>> Patchwork at lists.ozlabs.org
>>>> https://lists.ozlabs.org/listinfo/patchwork
> _______________________________________________
> Patchwork mailing list
> Patchwork at lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/patchwork
> 

-- 
Andrew Donnellan              OzLabs, ADL Canberra
ajd at linux.ibm.com             IBM Australia Limited