[PATCH v3 3/5] models: Convert Series-Patch relationship to 1:N

[Daniel Axtens]

Daniel Axtens <dja at axtens.net> writes:

> Hi Stephen,
>
>>> I'm cloning the lkml public-inbox git repos and attempting to import
>>> them into patchwork (which requires a new helper script). That'll be
>>> over a million messages. Then I will time the migration and watch the
>>> memory usage. That's the one thing I want to check; apart from that I
>>> have no qualms with the series.
>>> 
>>> If you want to do this yourself and report back I'm very happy to take
>>> your word for the results, otherwise it should be no more than a day or
>>> two given that I've now finished with the URL schema mangling.
>>
>> If you're able to do it, I'm happy to wait. There's also the Bootstrap
>> 4 thing but that's more subjective and not as crucial to have eyes on
>> (it's supposed to look almost the same for now).
>
> I have imported 2 of the 7 repos - each import takes 5-6 hours :(
> In total I now have 340k patches and I think roughly the same number of
> comments. On my laptop, the db queries to display the list page takes a
> terrifying 12 seconds, so I have no idea how kernel.org's lkml pw is so
> fast. (It has almost a million patches!)
>
> I then tried to apply the patches but they no longer apply to master. So
> I pulled out a branch where I had applied some before, applied the
> remainder, and rebased that. 
>
> I then ran 'time python3 manage.py migrate'.
>
> The migration took 6 and a half minutes on my laptop and nothing was
> oomkilled. So even for people with significantly more patches, the
> migration should be doable.
>
> Tests pass.
>
> Tested-by: Daniel Axtens <dja at axtens.net>
> (you'll have to add that manually as the patch isn't showing up in pw)

It seems I forgot to check this (d'oh!), but it now takes 112 db queries
for me to load the patch list page. They're almost all of the form

SELECT
`patchwork_patch`.`submission_ptr_id`, `patchwork_patch`.`series_id`
FROM `patchwork_patch` WHERE `patchwork_patch`.`submission_ptr_id` = NNNN

Stephen, could you check this out please?

Regards,
Daniel


>
> Regards,
> Daniel
>>
>> I'll stick the URL patch on my review queue for sometime this week.
>>
>> Cheers,
>> Stephen