Structured feeds

[Stephen Finucane]

On Sat, 2019-11-09 at 08:41 +0100, Johannes Berg wrote:
> On Sat, 2019-11-09 at 01:18 +1100, Daniel Axtens wrote:
> >  - code that efficiently reads a public-inbox git repository/folder of
> >    git repositories and feeds it into the existing parser. I have very
> >    inefficient code that converts public-inbox to an mbox and then
> >    parses that, but I'm sure you can do better with a git library.
> 
> Somebody (Daniel Borkmann?) posted a (very fast) public-inbox git to
> maildir converter, with procmail support. I assume that would actually
> satisfy this step already, since you can just substitute the patchwork
> parser for procmail.

What do you mean "substitute the patchwork parser for procmail"? From
reading this thread, I got the impression that we'd be changing what
feeds things into the 'parsemail' management command, right? Why would
we be dropping or substituting anything from Patchwork itself? We still
need that to do things like identify "is this a patch, a cover letter,
or something else entirely" surely?

Stephen

> >  - careful thought about how to do this incrementally. It's obvious how
> >    to do email incrementally, but I think you need to keep an extra bit
> >    of state around to incrementally parse the git archive. I think.
> 
> Not sure he had an incremental mode figured out there, but that can't
> really be all *that* hard, just store the last-successfully-parsed git
> sha1?
> 
> johannes
> 
> _______________________________________________
> Patchwork mailing list
> Patchwork at lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/patchwork