Patchwork v2 and beyond

[Daniel Axtens]

Hi Stephen,

> Yes and no. We're on RC4 already, which is kind of mad. If you look at the RCs
> we've been releasing, most of them have been more about adding features to fill
> holes or oversights than fixing bugs. I'm concerned that if we continue on this
> trajectory, new features will keep coming in and we'll never get the release
> out. To be honest, at this point I want to just merge the "single patch as
> series feature" and, barring any serious issues from the below, release a final
> version. We're not Debian, but we've done a lot of real world testing with
> popular mailing lists and found very few issues. If we do encounter some later,
> I'm more than happy to make PATCH releases as required. I'll hang on for the
> fuzzing results, but after that I really want to cut this and move on to
> getting it deployed and used in the real world.

Yeah, I understand that and I think we'll probably get away with it this
time without any major issues. In hindsight, I think we probably needed
to call a feature-freeze explicitly. I kind of assumed the rc process
would do that because that is how it works in kernel land, but that
hasn't been what ended up happening. In hindsight, I probably should
have kicked off this discussion before the start of the rc process.

(I will admit to wanting Debian-esque levels of stability for what has
arguably become core infrastructure for several kernel sub-systems, but
again, I should have kicked off this discussion earlier.)

>> ** Fuzzing the parser
>> 
>>  The hope here is to find any as-yet-undiscovered failure modes in the
>>  parser. I am hoping to do this in the next couple of days.
>
> +1. This is the one other thing I'm willing to hold 2.0.0-final for, for better
> or worse :)

I'm cleaning up the patches now. Nothing especially critical, but just
lots of unexpected ways things can fail - especially dates.

>> * For v2.0.1
>> ** Django 1.11 support
>> 
>>  We need to keep up with Django, so this would be important.
>
> I agree that we want Django 1.11, but it's a 2.1 target. PATCH releases are
> only used to fix bugs, but Django 1.11 support is a feature. Given that no LTS
> distro is packaging 1.11 yet and that 1.8 is still supported for the
> foreseeable future, I think we're fine to wait a few months for 2.1.

OK.

>
>> * For version 2.n:
>> ** Version support
>> 
>>  We currently have the ability to record the version of patches/series -
>>  we'd like to be able to integrate this in the UI. 
>
> I assume you mean some way to switch between series? If so, that sounds good. I
> think Andy Doan did some work on this?

Yes, that's what I wanted.

>
>> ** Check permissions
>> 
>>  A more sophisticated model for granting permissions for checks may be
>>  helpful.
>
> +1
>
>> ** Events/ETags/Push API
>> 
>>  We've been talking about good ways to provide this. We should do it.
>
> +1 - issue #2 is tracking this, and I'm planning to make use of django-channels 
> to implement this. This will be optional, of course, to avoid deployment issues
> for folks on distros without this package.
>
>> ** Port pwclient to REST
>> 
>>  What it says on the box.
>
> Getting support for REST is definitely a must. Ideally though, I'd like
> pwclient to handle both APIs and gracefully fallback for unsupported operations
> (series stuff) on XML-RPC. There are a lot of pre-0.9 Patchwork instances out
> there and I'd like to keep supporting these for the next two to three years.
>
>> * Version 3.0+
>> 
>> ** Drop Python 2 (and anything < Django 2)
>
> Unless 3.0 isn't coming out until 2020 (when Python 2.7 reaches EOL), I'd hold
> off on this. Python 2.7 support basically costs us nothing, and we want to
> support both Django 1.8 and 1.11 for the full duration of their lifecycle.
>

The only thing I really want to add that is much harder with Py2 is MyPy
type annotations. I also think we could switch whenever py3 is the
default in the majority of stable distros, which should be the next
couple of years. But yes, this is a long term thing and doesn't need to
happen at any particular version number.

>> ** Extend the REST API so that we can separate the front and back end
>
> This needs further discussion. I get that Single Page Applications and the
> likes are all the rage these days, but implementing something like this is
> likely to take a lot of time and deliver at best very minor benefits. Unless we
> can deliver something tangible that hardcore Patchwork users will appreciate,
> I'd rather expend that effort elsewhere.
>
I'm not necessarily wanting to make patchwork a SPA: I know lots of our
users have a strong attachment to the UI and the classic web experience
so I don't want to make them unhappy.

There are a few other things I want to be able to do with this:

 - full CLI support. Lots of our users basically live in the terminal,
   I'd like to give them a first-class experience.
   
 - ability to do much better testing of both ends.

 - maintaining smaller, more isolated codebases.

I also have some other very long term plans to experiment on the internals
that are much easier with a split. The hopes here are to significantly
speed up all interactions with patchwork.

>
>> Thoughts?
>
> My few points aside, this looks pretty good. I do assume that there's an
> implicit promise to contribute some of these full features yourselves? My own
> priorities for Patchwork are slightly different, as seen below.
>

Yes, especially the REST stuff.

> ---
>
> * 2.1
>
> This will be a small finetuning release which will work on some of the stuff we
> missed in 2.0.
>
> ** Bootstrap-ify the rest of the web UI
>
>    Our web UI is very mismash and hasn't got any real love in 2.0. I'd like to
> do some work on making it more consistent and approachable.
>
Sure, sounds good.

> ** REST support for pwclient
>
>    See notes above. It's worth mentioning that I've already started work on
> separating out pwclient into its own Git repo, but I plan to continue to use
> the same mailing list for patch, bugs, etc. This leads us to...

It would be nice to make it pip install capable and/or putting it in a
PPA on launchpad so people can apt-get install it. Happy to help with
that.

>
> ** Multiple projects per mailing list
>
>    We saw patches related to this recently. I'd like to support these.
>
> ** Remove Django 1.6, 1.7 support, add Django 1.11 support
>
>    Removing these old versions is long overdue. With the new Debian out, we
> need to do this.

Very keen to drop those, it will remove a bunch of special cases.

>
> ** ETags
>
>    Minimize hits to the API as much as possible
>
> * 2.2
>
> This will be a larger release which will add a couple of useful new features
>
> ** Proper series versioning
>
>    We have multiple series versions. Now we need a way to show the link between
> these in both the web UI and REST API.
>
> ** Check permissions
>
>    As above. This might end up in 2.1 if it's small enough.
>
>    See
>
> ** Labels
>
>    See https://github.com/getpatchwork/patchwork/issues/22
>
> * 2.3
>
> ** Web hooks/"push API"
>
>    See above. This is out here because I really want to see if it's necessary
> first (and not just a "nice to have"). It's going to result in significant
> changes to the deployment method, so we need to make sure of this.
>
>    See https://github.com/getpatchwork/patchwork/issues/2
>
> * 2.4+
>
> Who knows???

Looks good.

Regards,
Daniel

>
> ---
>
> Hope this helps,
> Stephen