[ccan] CCAN issues

Thu Dec 20 18:20:19 EST 2007

On 19/12/2007, Michael Cohen <scudette at gmail.com> wrote:
>
> Hi Rusty,
>    Thanks for taking the initiative...
>
> > 1) Domain name.  ccan.org would be ideal: seems to point to a generic
> ISP
> >    page, whois says Community Care Ambulance Network in Ohio, which
> seems
> >    legit at first glance.  ccan.com seems to be for sale by a squatter,
> >    perhaps we could do some deal where we buy ccan.com for the ambos and
> get
> >    ccan.org?
>
> What about ccan.com.au? or maybe a hyphenated version (ccan-repo.org
> for example?)
>
> > 2) Namespacization.  If someone can't use (say) container_of because
> their
> >    existing project uses that name, it'd be nice to rename it to
> >    ccan_container_of.  Of course, this means any other ccan modules
> which use
> >    this need changing too.
> I wonder if that is an integration issue? Clearly linking in C will
> become an issue due to name clashes. We should aim to have all ccan
> modules linked internally with strict symbol exportation controls
> (most things static etc). Things like macros and exported functions
> will need to be prefixed by something. (maybe ccan_ ).

I suspect this will be one of the trickiest elements of the design.

Keep in mind that one of the nicest features of this sort of repository is
that once it becomes useful, network effects kick in and you often suck in
other existing code.

So what if libpng needs to be a CCAN package?

As package numbers increase, so does the length of the package names.

http://search.cpan.org/recent

That is the upload stream for CPAN modules, note that some of them have
named that are very very long. Will it be feasible to have names like

foo__bar__baz__openoffice__plugins__etc

But my C knowledge lacks here, so keeping the namespaces and amount of
typing under control is somewhere C experts need to come up with ideas.

My naive solution would be potentially something like an implicit macro that
maps "namespace.variable" to ccan__foo__bar__baz_.....__variable or
something.

Code outside of the "current namespace" would use the explicit version...

>    I thought about making the ccan_ versions canonical and providing
> macros
> >    to map to the normal names, but it breaks the golden rule:
> >
> >         This code must not be ugly.
> >
> >    So I chose the harder road of supplying a C program to actually
> rewrite the
> >    module and anything which depends on it.  Not quite finished yet, but
> it
> >    has the merit of placing the burden of ugliness where it belongs.
> This might produce un-ugly code but would make it difficult to
> integrate future versions of ccan into your project. This is because
> the version in your project would be different than the version in
> ccan and its hard to track these differences. I think it would be less
> ugly if the user ultimately drops in newer versions of ccan files
> instead of having to re-generate them.

CXAN projects tend to lead to fairly rich installation clients, due to
issues like mirror selection, dependency resolution and auto-mated testing.

So having some amount of code auto-munging is not too big a deal.

For example, the JSAN strips out all the documentation at install time to
compile into separate files, because the code will need to go over the
internet and the docs add a lot of bloat.

> 3) Minimum compiler requirements.  Seems like Microsoft's C compiler
> doesn't
> >    do vararg macros, let alone any other C99 stuff.  This makes it
> impossible
> >    to have nice code for some things, so for the moment I'm still
> sticking
> >    with my intention of requiring (most of) C99, and Windows coders will
> have
> >    to hope they catch up, or we do some horrible mangling at the user
> end
> >    (like namespacization).
> I dont think thats a great issue since gcc is also available for
> windows - I somehow dont expect hard core windows coders to be using
> ansi c (if they use MSVC they are more likely to use c++).

I concur. MinGW support should be all any Windows people will need.

> 4) Licensing: so far I'm restricting it to with BSD no advertising, or
> GPLv2+.
> >    More than that makes life too tricky for people wanting to use the
> code.
> We need a consistant license for everything - unfortunately for
> testing etc we our selves will be linking to our own code so if we
> have a mix of licenses we might be breaking them ourselves.

This is going to be very tricky... it does depend to some degree on what a
CCAN package provides. If you are building libraries from CCAN packages
instead of inlined stuff, different issues apply.

My gut instinct is to make sure that each uploaded package HAS a license
defined in metadata, and that the license falls within a set we accept, and
from there implement programmatic license tracking into the installer
client.

I'm unsure how difficult programmatically modelling license compatibility
would be though, but it would seem to be within the realm of the possible.

> 5) Uploading, automated testing, etc.  No idea here, plan to steal as much
> of
> >    CPAN as possible for this.
> I think we need to use something bzr or darcs where people can send
> patches to the maintainer. These need to be tested before added for
> inclusion into the repo. There need to be some requirements before a
> module goes in (like tests, license etc).

This is one area that I would really like to be quite insistent on.

The official canonical and authorative packages should be tarballs and only
tarballs.

This achieves any number of benefits, for example.

1. It makes the mirroring and distribution of code easy.

Both CPAN and JSAN at their core can be expressed as a single FTP server
with files in it.

CPAN has even had to largely abandon mirroring via rsync, as the load is too
intensive on the master mirror servers.

It also makes the packages stable over time.

2. It allows choice.

Developers care a lot about their development tools. It is a REALLY bad idea
to try to force the user to use someone else's tools.

You may want to know ABOUT a repo, that's totally fine.

But you describe the repository type and location in package metadata, you
don't actually release anything this way.

3. It simplifies the server indexing and management enormously.

By just opening up the tarball and looking inside it, you keep the indexer
much more sane.

It also lets third-party systems much more able to do analysis and support
tasks on the code.

> 6) I have documentation-extraction code, but it doesn't do any xml-style
> guff
> >    (the Linux kernel's system on which this is loosely based is perl
> based, so
> >    maybe we should just steal that for the documentation, as it does
> various
> >    different output forms).
> I personally dont like automated documentation generation - most of
> the time it tells you exactly what you would be able to read by just
> looking at the code and it allows people to be sloppy about writing
> documentation (hmm i can just run doxygen on this later - no need to
> document). I think reams of doxygen generated code from a thinly
> documented source tree is worse than do documentation at all.

If you don't like automated documentation generation then you may just not
have seen it done well.

For example, here's the web version of the generated documentation for one
of my CPAN modules.

http://search.cpan.org/perldoc?PPI

That is the generated from the inline docs of the top namespace... here is
the full list of namespaces for the package.

http://search.cpan.org/~adamk/PPI-1.201/

Here is the typical documentation for a few sample more-thinly documented
namespaces.

http://search.cpan.org/~adamk/PPI-1.201/lib/PPI/Statement/Include.pm

http://search.cpan.org/~adamk/PPI-1.201/lib/PPI/Element.pm

The same source documentation is also used to compile man pages for each
CPAN namespace.

The problem of bad standards of documentation is one I think that needs to
be solved culturally and by convention.

If the kernel standards provide a reasonably well-known standard for
per-function documentation, then perhaps we need to look at extending it to
allow all the additional man-like data to be added in there as well.

Adam K
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/ccan/attachments/20071220/6b3c6b51/attachment.htm>