[ccan] Using ccan with a C python extension

[David Gibson]

On Mon, Jun 15, 2015 at 08:17:45PM -0400, Cody P Schafer wrote:
> On Mon, Jun 15, 2015 at 6:01 PM, Rusty Russell <rusty at rustcorp.com.au> wrote:
> > Stuart Longland <stuartl at longlandclan.yi.org> writes:
> >> On 15/06/15 12:01, Rusty Russell wrote:
> >>> Naveen Nathan <naveen at lastninja.net> writes:
> >>>> It is setup to be installed as a python package via pip/easy_install
> >>>> which will:
> >>>>     (1) fetch the python module
> >>>>     (2) build the C extension, and finally
> >>>>     (3) install it on the system (or virtual environment).
> >>>>
> >>>> I specifically want to use ccan/{bitmap,endian} but unsure how
> >>>> to integrate the code into our project's build process.
> >>>>
> >>>> I envision at step (2) above, we will build libccan and statically
> >>>> link it to the C extension. However, I will need a ccan config.h
> >>>> which will vary per system. So this will involve generating the
> >>>> file. Does that mean I should include the tools / configurator
> >>>> as part of our python package?
> >>>
> >>>         I think you should use autoconf, as it's the standard.  I keep
> >>> hoping someone will write the autotools stuff for CCAN for me :)
> >>
> >> Yeah, the trouble with that approach is that autoconf does a pretty poor
> >> job of integrating into Python module builds.
> >>
> >> The usual build system in Python is the `distutils` module, which
> >> basically gets included into a Python script called 'setup.py' and called.
> >>
> >> Then it's a *magic happens here* affair, where out the other end is your
> >> Python module.
> >>
> >> You'd have to hook something in that could call ./configure at the
> >> appropriate time, and somehow convince the automated packaging tools
> >> (stdeb, gsourcery-pypi, etc) to ignore ./configure and use setup.py.
> >>
> >> I'm not sure what's needed in config.h; maybe there's some way to
> >> generate this using some basic Python calls before building?
> >
> > It depends very much which modules.  For bitmap and endian you need:
> >
> > HAVE_BUILTIN_CLZL
> > HAVE_BYTESWAP_H
> > HAVE_BSWAP_64
> > HAVE_LITTLE_ENDIAN / HAVE_BIG_ENDIAN
> >
> > So you could get away with something like:
> >
> > if sys.byteorder == 'little':
> >    print "#define HAVE_LITTLE_ENDIAN 1"
> > else:
> >    print "#define HAVE_BIG_ENDIAN 1"
> 
> Do note that doing this will make any people trying to cross-compile
> across endian (not common due to the current decline in BE usage, but
> possible).
> Of course, the current configurator has the same issue :)
> (HAVE_*ENDIAN is a executed test).
> 
> I'm not sure how autotools solves this for cross compilation, but I'd
> guess it's something they are guessing based on a target's triple.

Actually, autoconf doesn't deal very well with cross-compilation
either.  It, too, relies on executed tests.

It has some ways around this - basically importing the configuration
information from elsewhere for the cross compile case, but making this
work for any custom tests you've written is extra effort and
frequently not done.

Like many configuration systems, autoconf is also a pain for packagers
because you need to be very careful when making scripts which are
supposed to produce a build with a known, rather than detected set of
features.

For these reasons, I wouldn't wholeheartedly recommend autoconf these
days.  Unfortunately, I also don't know of something out there that's
clearly better.

The rise of Linux means there are usually less platforms nowadays that
you're realistically likely to care about, which makes hand-rolling a
config system a lot less insane than it once was.

If the only reason you need a config is for ccan modules, I think
importing the ccan configurator could actually be a pretty good
option.  Unfortunately ccan doesn't presently provide tools to make
that straightforward (ccan in general could do with more tools for
making it a smoother process to use the modules in other projects).

One clunky, but workable, option is to incorporate the whole ccan tree
into your project as a git submodule.  You can then call into the ccan
makefiles to configure and build libccan.a which you can then link
from your own part of the project.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.ozlabs.org/pipermail/ccan/attachments/20150616/9b0b5408/attachment.sig>