[c-lightning] Plugins/passthrough for next version?
decker.christian at gmail.com
Sun Aug 26 22:23:20 AEST 2018
I think we have quite a few different options to foster extensibility on
top of a focused and lean lightning implementation, each with its own
pros and cons:
- Driver: a program that controls the lightning daemon, tails its
output, or has another way to stream events from the daemon, and
interacts with the JSON-RPC we already have. That's basically how we
drive the tests, both for c-lightning  as well as the
integration tests across all implementations . It's a bit cludgy
since we tail the logs which were not intended for machine
consumption, but it works right now and is how I've been building a
number of applications. An event stream, e.g., streaming JSON events,
would make this a lot easier.
- Sidecar: this is just a small variation of the above, in that it
doesn't control the daemon, and has to rely on polling it. It's
higher overhead, slower and would also benefit from an event stream.
- Callbacks: c-lightning hooks into certain events and calls out to a
registered external program if something of interest has
happened. This is basically the model `-blocknotify` option of
bitcoind. It's simple and allows the external tool to be written in
any conceivable language, but it forces the external tool to manage
state across multiple runs, and may come with some
overhead. Ultimately it'd result in people just using it as an event
stream that pipes into a long running program anyway.
- Embedded plugins: this is probably the most ambitious in that we
build plugins that are executed directly in the context of
c-lightning, either by compiling them in, dynamically loading them,
or embedding an interpreter. The first two options would again limit
us to languages that work well in C, while the last would give us
free choice of languages, and why not, we could even implement a
small wasm interpreter, e.g., wac . It'd be the least overhead
option, but also the hardest to pull off.
We already have number of tools that use the Driver or the Sidecar
model, and we have at least one PR that would give us a Callback
extension point (PR #1715). As for the last option, it'd still limit us
to Rust, C and C++, until the wasm community has outlined how to
interface with a garbage collector. The embedded plugins would also be
completely isolated, which is good for autopilots and similar
self-contained things, but not so good for wallets and things that
actually need to talk to the outside world.
Out of these options we probably should prioritize one, and try to make
that as appealing as possible. As simple as it sounds, just providing a
streaming API to get notified of internal events would get us quite far
already, as long as we don't need to change the automated behavior like
#1715 tries to do.
More information about the c-lightning