[ccan] CCAN status update

[Rusty Russell]

On Friday 21 March 2008 17:39:19 Adam Kennedy wrote:
> Rusty Russell wrote:
> >
> > So, we're talking about individual modules having releases, rather than
> > a "CCAN" release?  Even so, I'd prefer to keep a "release" tree/branch
> > and have people commit to that.
>
> Correct.
>
> The single most common failure made by library developers in every
> language is that they treat their entire collection of modules as a
> single entity.

A very good point.  Yes, we should avoid this; organic independent development 
is what we're after.

> > I pretty much assumed that most developers (ie ones that do more than
> > throw in code and walk away) will hold the whole repo.  But maybe that's
> > just me.
>
> Unfortunately, that doesn't scale very far... the "entire" CPAN
> repository is around 1 gigabyte compressed.
>
> So that would be something like a 10-20 gig checkout, uncompressed, with
> god only knows how many directories and files.

Well, it seems that bazaar supports partial checkouts.  Mercurial does not yet 
(http://www.selenic.com/mercurial/wiki/index.cgi/PartialClone).  I don't 
think it's a showstopper, as long as it's coming within a few years.

It'd be nice to have the entirity of CCAN somewhere we can tar it up and 
transfer it or whatever.  Keeping history is definitely also a feature, but 
it's less clear that atomic updates across multiple packages are required 
(put this as a 'nice to have' tho).

> > Erk, I'd prefer to have something more sophisticated, but standard-tool
> > friendly.  At least mercurial has a 'download .tar.gz' button which takes
> > care of challenged platforms.  We should certainly have the same thing
> > for ccan.
>
> This somewhat depends on what exactly we're talking about as the
> "repository".
>
> I am mostly referring to a version control repo that will hold the
> website content, the indexer and mirroring and etc code, and so on.

Sure, that's currently the ccan_tools directory.  It should all be in its own 
repo for sure: it's pretty independent of ccan itself.  I'm just lazy :)

> The master CCAN repository should be a simple FTP-mirrorable directory
> structure containing the release tarballs, with a database of some sort
> to hold all the metadata and master indexes.

My experience so far is that most modules will have at least one dependency, 
so (at least one oversion of) the tarball should include the dependencies.

I thought we'd generate this from whatever repo(s) we use, along with the 
metadata and index web pages.

To crystalise this further, I was thinking something like:

	ccan: code with some minimal metadata (ie. _info file, maybe tests?)
	junkcode: code snippets with no metadata

ie. you can submit anything, if it doesn't have some minimal metadata it gets 
dropped into "junkcode".  Otherwise it gets put into ccan proper.

Maybe also a "core" of stuff that we choose as good examples of code which is 
popular and useful.  We can do that later tho.

> >> Where each individual author, or collection of authors, choose to store
> >> their modules should not need to be important.
> >
> > It's a good point, but I'd like to do this without dropping back to svn
> > on the main server.
>
> As I understand decentralized version control (bzr, mercurial, etc)
> there's no way I can commit to the project?
>
> I have to send every single change to you by email as a patch file?

God no, that would be awful!  That was just my suggestion for starting.

Mercurial uses ssh, in the past I've created an account and used 
authorized_keys to control who gets access to what repos.  Bzr can do a 
similar thing and I'm setting it up now: please send your ssh key.

Cheers,
Rusty.