[c-lightning] Feature: run scripts on incoming transactions

[Corné Plooy]

>> * If an incoming transaction contains a realm number that would
>> otherwise not be supported (currently, realm 0 is the only one defined
>> by the RFC and supported by C-Lightning), lightningd tries to execute a
>> script "handle_realm_<realmnumber>" in the Lightning directory. It
>> receives information about the transaction on STDIN (format: TBD), and
>> sends a single number back on STDOUT indicating whether the transaction
>> should be accepted or rejected.
> I would probably not use a naming convention to route to different
> handlers here, but rather have a script that acts as a router,
> distinguishing which script to call next based on the information passed
> in. This can be either a helper listening to a socket, as Rusty
> suggested, or be a script that is called by `lightningd` and specified
> on the command line. Singling out the realm byte and not, for example, the
> next hop node ID seems strange.
That sounds reasonable - have a single script or socket as interface,
and send all data (including realm number) to it. Then it's
installation-defined on the basis of what data (realm number or
something else) certain actions are performed.
> I can immagine that even realm 0 calling out to a script might be
> interesting since it allows for entirely different transport protocols
> (I always wanted to use lightning over e-Mail ^^).
Sure; it might even simplify lightningd, towards a more UNIX-like "do
one simple thing, and do it well". This is of later concern however; for
now, I'd like to keep my changes small and high-quality (high-quality is
on the TODO list). Once this gets merged, we can always look further.
>> * Currently, if such a script hangs, lightningd hangs. I'm not
>> completely happy with this, but what would be the alternatives? Not have
>> the script send any data back? Asynchronous communication with the
>> script? You potentially have a "process leak" problem: an attacker may
>> crash a machine by triggering huge numbers of parallel script
>> executions. For now, this problem does not exist, and "don't make buggy
>> scripts" is an obvious solution to stop lightningd from hanging.
> Is this is one-shot, as in yay or nay, or is the script called upon HTLC
> add and we can report HTLC resolution back? If we have a long time until
> we decide how to resolve the HTLC then we might want to make this
> asynchronous, since otherwise we might hang the entire node indefinitely.
>
>> * Extra RPC commands will be added to allow applications to give a
>> transaction preimage to lightningd (inherently safe) and to indicate
>> that a certain payment will be canceled (unsafe - the application needs
>> to ensure that any outgoing goods/services/assets are stopped, before
>> canceling the incoming funds).
> This seems to suggest that indeed the above is already async, so that's
> definitely good :-)
Right now, lightningd waits for completion of the script, and it waits
for status information returned by the script. Based on the status
information, the incoming HTLC might be rejected immediately. This is
the synchronous part of the interface. If, based on the status
information, the incoming HTLC is accepted, lightningd continues its
normal operation. It is then up to the application to use the RPC to let
the transaction succeed (with the preimage) or do an early cancel. This
is the asynchronous part of the interface. (Note: if the application
does neither, lightningd will have to handle time-out of the incoming HTLC.)

The status information in the synchronous part is intended for immediate
rejection of the HTLC, based on factors that can be determined
immediately, for instance, because its data is bogus. You might call
this a premature optimization; the design could be simplified by leaving
it away, always accept the incoming HTLC, and require the application to
use the asynchronous part of the interface for any reject of incoming HTLCs.

The reason for including the status information was a practical issue:
in case a script did not exist / could not be run, I wanted to keep the
old behavior of failing a HTLC with the WIRE_INVALID_REALM code. This is
detected in a forked process that fails to do execv(); I wanted some way
for the forked process to report this failure back to lightningd. More
generally: if we don't have the synchronous way of reporting failure,
then, in the absence of an application script / process (the default
install situation), we don't have a way to reject incoming HTLCs.

Now that I think of it, there might be other solutions. The interface
might be opt-in on the lightningd commandline / config file, so that, if
nothing is configured, lightningd rejects HTLCs with unrecognized
routing data, and if it is configured, the script becomes responsible
for this, and does this through the RPC (asynchronously). Another option
is to *always* have this interface enabled (so, no commandline option),
but have, by default, a script installed that just rejects everything
immediately. Then, configuration consists of replacing that script.

I'll also look into the socket option. I need to learn more about
sockets: if lightningd can determine whether someone is listening on the
other end, then it has a good indication of whether it needs to reject
weird routing requests immediately, or whether it should forward them
through the socket.

CJP