Downloading archives no longer possible

[Ralf Ramsauer]

Hi Stephen,

On 6/2/20 11:14 AM, Stephen Finucane wrote:
> Our documentation [1] suggests people initialize their development
> environments by download the archives of the Patchwork mailing lists
> from [2] and load them with the 'parsearchive' command. However, it's
> been pointed out to me that at request to [1] now return only a handful
> of recent messages. Is this change intentional?

we intensively used parsearchive over the last few weeks to simulate
incoming patches to patchwork. (We're working on the PaStA integration)

However, we experienced similar behavior with parsearchive and realised
that the order of messages in the mbox-file does matter: We get
different stats if we reverse the mbox, while order of incoming messages
shouldn't matter at all - you never know which mail you receive first.

Besides that, we also realised that 'failed' messages somehow get marked
by patchwork. Consider the following scenario: You want to parse a mbox
for mailing list a at b.com, but you didn't configure the corresponding
project yet. parsearchive will report that it didn't find a single
patch. That's true, because it's simply missing the configuration. Then
you create and configure the project, and re-run parsearchive. Again,
parsearchive will report that it couldn't find a single patch. I assume
that during the first run, message-ids that contain non-valid mails
somehow get stored. In order to import that archive, we had to wipe the
whole database.

I thought this behavior would be intentional, but I think it's worth
mentioning our findings as you seem to have related issues.

> 
> Stephen
> 
> [1] https://patchwork.readthedocs.io/en/latest/development/installation/#import-mailing-list-archives
> [2] https://lists.ozlabs.org/private/patchwork.mbox/patchwork.mbox

Didn't test this archive, though. Downloading it requires authentication.

  Ralf