[Automated-testing] Structured feeds

[Don Zickus]

On Fri, Nov 08, 2019 at 10:44:37PM +1100, Daniel Axtens wrote:
> Don Zickus <dzickus at redhat.com> writes:
> 
> > On Thu, Nov 07, 2019 at 02:35:08AM +1100, Daniel Axtens wrote:
> >> > As soon as we have a bridge from plain-text emails into the structured
> >> > form, we can start building everything else in the structured world.
> >> > Such bridge needs to parse new incoming emails, try to make sense out
> >> > of them (new patch, new patch version, comment, etc) and then push the
> >> > information in structured form. Then e.g. CIs can fetch info about
> >> 
> >> This is an non-trivial problem, fwiw. Patchwork's email parser clocks in
> >> at almost thirteen hundred lines, and that's with the benefit of the
> >> Python standard library. It also regularly gets patched to handle
> >> changes to email systems (e.g. DMARC), changes to git (git request-pull
> >> format changed subtly in 2.14.3), the bizzare ways people send email,
> >> and so on.
> >
> > Does it ever make sense to just use git to do the translation to structured
> > json?  Git has similar logic and can easily handle its own changes.  Tools
> > like git-mailinfo and git-mailsplit probably do a good chunk of the
> > work today.
> >
> +patchwork@
> 
> So patchwork, in theory at least, is VCS-agnostic: if a mail contains a
> unified-diff, we can treat it as a patch. We do have some special
> handling for git pull requests, but we also have tests for parsing of
> CVS and if memory serves Mercurial too. So we haven't wanted to depend
> on git-specific tools. Maybe in future we will give up on that, but we
> haven't yet.

Fair point.  Thanks!

Cheers,
Don

> 
> Regards,
> Daniel
> 
> > It wouldn't pull together series info.
> >
> > Just a thought.
> >
> > Cheers,
> > Don
> >
> >
> >
> >> 
> >> Patchwork does expose much of this as an API, for example for patches:
> >> https://patchwork.ozlabs.org/api/patches/?order=-id so if you want to
> >> build on that feel free. We can possibly add data to the API if that
> >> would be helpful. (Patches are always welcome too, if you don't want to
> >> wait an indeterminate amount of time.)
> >> 
> >> Regards,
> >> Daniel
> >> 
> >> 
> >> -- 
> >> _______________________________________________
> >> automated-testing mailing list
> >> automated-testing at yoctoproject.org
> >> https://lists.yoctoproject.org/listinfo/automated-testing