[PULL] github.com/stephenfin/development

[Damien Lespiau]

On Fri, Oct 16, 2015 at 04:11:50PM +0100, Finucane, Stephen wrote:
> You appear to be insisting there should be a more static configuration
> of "runnable tests". However, I don't think this would scale or allow
> the fluidity of Thomas' proposal.

I'm insisting it's *possible* to have a list of pre-defined tests, not
that it would be the only way to report test results. I can see merits
in both policies. I actually don't care how it's implemented as long as
it's reasonable and integrated into patchwork.

> In the normalized case, allowing a new CI system to start reporting
> test statuses would require (a) allowing them access to the mailing
> list

Nop. The test results are sent to patchwork, which then replies on
behalf of the tester. Several configurable policies could be possible,
depending on what project want to do:

  - send a consolidated email if we have a list of pre-defined tests
  - send per-test mails otherwise?
  - send consolidated mails on full series (remember the discussion
    about testing full series because it can impractical to do it on
    every patch, not just a corner case, we need it for the project I
    work on)
  - ...

No need for the test system to be on the mailing list, no mail setup on
the client side is all win to me.

> and (b) configuring patchwork to enter this new tests. This, to me, is
> a barrier.

This doesn't have to be the case. Configurable behaviours are totally
fine.

> I hadn't planned on letting patchwork send any more emails, if I'm
> being totally honest. If you think an email would be useful, could we
> develop a script that checks for a given list of 'contexts' on a patch
> and asserts that those contexts all exist and have a 'passed' state.

Given that patchwork is there to augment mails & mailing-lists, it
seemed natural to me to have it replying to patches with test results.

An external script could mostly work. However I'd like it to be
integrated so we can do things like replying to the cover letter of the
series, which needs full access to the patchwork.

-- 
Damien