[Patchwork-maintainers] patchwork.ozlabs.org downtime for maintenance - 15/16 August

[Jeremy Kerr]

Hi all,
> I'll try and get to that over the weekend.

Looks like the heaviest database load is due to API request to the
global patches view, which is a bit of an odd use-case; that all
appears to be mostly spider traffic.

Konstantin: I'm not sure your new index would help in that case, we're
not looking up delegates for those views.

Looking through the access logs, there seem to be three clients that
are causing around 40-50% of patchwork load:

 - one IP from an "Alibaba Cloud HK" AS, various UAs
 - one IP from a Red Hat AS, curl/7.61.1 UA
 - the Bytedance "Bytespider" UA

All three seem to be scraping the patchwork site.

I have blocked all three for now, but it would be worthwhile setting up
a more fair robots.txt and/or a reasonable ratelimit for the latter
case.

If anyone knows what might be up with that Red Hat crawler, please get
in touch with me.

I'll keep an eye on things here; there's still likely a bunch of
potential configuration optimisation we can do too. Let me know if your
observations change though.

Cheers,


Jeremy