[PATCH 00/10] Add series support

[Russell Currey]

On Thu, 2016-06-16 at 09:21 +1000, Andrew Donnellan wrote:
> On 15/06/16 19:07, Finucane, Stephen wrote:

<snip>

> > The best option we might have, if per-series reporting is really
> > necessary, is to allow Check uploading against a Series endpoint. This
> > would actually cause N Checks to be created - one for each Patch in the
> > series - meaning each Patch could still be individually queried. It
> > would be a bit of a lie (we didn't actually test the patch by itself,
> > therefore it might be broken) and I wouldn't promote this workflow
> > myself (bisectability FTW), but it could be a good way of dealing with
> > extremely long-running or resource-intensive test suites, where
> > per-patch validation would be too expensive.
> 
> I suppose this would be better than nothing, though apart from being 
> mildly misleading it would also generate a lot of spam. If I'm 
> submitting 5 test results against the entire series and 1-2 test results 
> against each individual patch, and we have all of those results 
> appearing on every patch, things could get more than a little bit 
> confusing. Personally, I'd be inclined to submit series-wide test 
> results as checks on the very last patch in the series instead of using 
> this.
> 
I don't think adding checks on the very last patch is particularly intuitive.
Having a Series endpoint that creates checks on every patch in the series would
be fine so long as there was some obvious way of distinguishing what was run on
the series as a whole and what was run on the individual patch, whether it's
something arbitrary like "[series]" in the check name or if there was some flag
that triggered a visual differentiation.

The reason I brought this up is that Andrew and I wrote a tool to automatically
test series as they're indexed in Patchwork that was built on the Series support
in the fdo fork, I'll play around with this series and see if anything major is
missing that would prevent us from doing the same with this implementation.