[PATCH 01/11] Improve patch listing performance (~3x)

[Stewart Smith]

Stephen Finucane <stephen at that.guru> writes:
> On Sat, 2018-08-11 at 04:28 +1000, Daniel Axtens wrote:
>> Stewart Smith <stewart at linux.ibm.com> writes:
>> So, further to our conversation with Konstantin, I tested this against
>> Django 2.0. It still saves us some time - it means we no longer load the
>> following fields:
>> 
>> `patchwork_submission`.`id`, `patchwork_submission`.`msgid`,
>> `patchwork_patch`.`commit_ref`, `patchwork_patch`.`pull_url`,
>> `patchwork_patch`.`archived`, `patchwork_patch`.`hash`,
>> `patchwork_patch`.`patch_project_id`, 
>
> Out of curiosity, does this allow us to drop the patch_project_id field
> entirely or not (bear in mind, I haven't read through the rest of
> this).

I'm not too sure... Umm... maybe not as I think we can go from patch to
project when viewing a patch?

>> This obviously saves the db some work and communication overhead.
>> 
>> I'm a little nervous that this will slightly complicate some of the
>> further denormalisation but I think that's probably a price worth paying
>> unless Stephen objects.
>
> To be honest, it could be 6 months before we get any further on big
> efforts like this. A 3x performance boost right now beats an unknown
> boost N months down the line, IMO. My 2c anyway.

I'd tend to agree, we may as well get the maximum performance we can
today, and then if it's *really* needed later down the track, we can
change the schema a bunch to denormalise.

>From my benchmarks, the biggest boost currently would be to precompute
and cache patch counts for most of the common views, as a *lot* of page
time is for that.

>> I do still want to test the 'ordering' change a bit more before
>> committing it though.
>
> I'm going to attempt to do my own testing of this (with a much larger
> dataset) some evening this week. I'll hold off applying anything
> pending this though.

Nice, let me know how it goes. I'll be off somewhere in a national park
for a bunch of next week, which means I won't look at email, so it's the
*perfect* time to merge and deploy all of my patches :)

-- 
Stewart Smith
OPAL Architect, IBM.