[ccan] embedding ccan in other projects redux

[Rusty Russell]

Naveen Nathan <naveen at lastninja.net> writes:
> Earlier I had started a discussion about how to approach embedding
> specific ccan modules into a Python C extension project. I discussed
> the options available with David Gibson over IRC. This email
> serves to bring the conversation back to this mailing list.
>
> Two distinct issues were discussed:
>
> (1) ccan modules shouldn't implicitly include a "config.h" for their config.
>
> Another way to say this is each ccan module should stand alone with no shared
> dependencies. Instead a top-level build/configurator system will generate the
> needed config.h files and glue. In the case of ccan this is done by
> specifying the -include compiler directive in the Makefiles.

I really hate magic includes; I'd rather see ccan_config.h than
config.h.  We could do:

        #ifdef CCAN_CONFIG_HEADER
        #include CCAN_CONFIG_HEADER
        #endif

Though it seems pretty ugly...

> (2) creating a pathway to make ccan modules embeddable
>
> Instead of an overhaul of the existing build system we can make incremental
> progress to allow flexible embedding of ccan into other projects. A few
> steps were discussed:
>
>  * Adapt the ccan makefile for easier embedding, so it will just attempt
>    to build those modules that are present

My current Makefile method is to build ccan like so:

CCAN_OBJS :=					\
	ccan-crypto-ripemd160.o			\
	ccan-crypto-sha256.o			\
	ccan-crypto-shachain.o			\
	ccan-err.o                              \
	ccan-opt.o                              \
	ccan-opt-helpers.o                      \
	ccan-opt-parse.o                        \
	ccan-opt-usage.o

ccan-crypto-shachain.o: $(CCANDIR)/ccan/crypto/shachain/shachain.c
	$(CC) $(CFLAGS) -c -o $@ $<
ccan-crypto-sha256.o: $(CCANDIR)/ccan/crypto/sha256/sha256.c
	$(CC) $(CFLAGS) -c -o $@ $<
ccan-crypto-ripemd160.o: $(CCANDIR)/ccan/crypto/ripemd160/ripemd160.c
	$(CC) $(CFLAGS) -c -o $@ $<
ccan-err.o: $(CCANDIR)/ccan/err/err.c
	$(CC) $(CFLAGS) -c -o $@ $<
ccan-opt.o: $(CCANDIR)/ccan/opt/opt.c
	$(CC) $(CFLAGS) -c -o $@ $<
ccan-opt-helpers.o: $(CCANDIR)/ccan/opt/helpers.c
	$(CC) $(CFLAGS) -c -o $@ $<
ccan-opt-parse.o: $(CCANDIR)/ccan/opt/parse.c
	$(CC) $(CFLAGS) -c -o $@ $<
ccan-opt-usage.o: $(CCANDIR)/ccan/opt/usage.c
	$(CC) $(CFLAGS) -c -o $@ $<

A tool which automated this would be useful.

>  * Implement (1) for ccan using -include directives
>  * Make a ccan tool to spit out all config variables used by a module
>    (and its dependencies?)

Yes, this would be a win.  

>  * Document the existing ccan config options
>  * [possible] generate files to help autoconf systems (such as aclocal defs)

Yep.

> I can potentially see create-ccan-tree used as a way to create a
> separate ccan tree with a skeleton build environment and specified modules
> and dependencies.

I currently (ab)use create-ccan-tree like so in my Makefile:

# Set CCAN_NEW if you want to add a new module.
update-ccan:
	mv ccan ccan.old
	DIR=$$(pwd)/ccan; cd ../ccan && ./tools/create-ccan-tree -a $$DIR `cd $$DIR.old/ccan && find * -name _info | sed s,/_info,, | sort` $(CCAN_NEW)
	mkdir -p ccan/tools/configurator
	cp ../ccan/tools/configurator/configurator.c ccan/tools/configurator/
	$(MAKE) ccan/config.h
	grep -v '^CCAN version:' ccan.old/README > ccan/README
	echo CCAN version: `git -C ../ccan describe` >> ccan/README
	$(RM) -r ccan.old

It's pretty horrible, so any improvements welcome!

Cheers,
Rusty.