[c-lightning] Towards a bus architecture for C-Lightning plugins (or: C-lightning plugins empowerment)
ZmnSCPxj
ZmnSCPxj at protonmail.com
Fri Mar 8 10:37:13 AEDT 2019
Good morning list,
These ideas have been percolating for a while now (since I heard of plans to implement plugins last year) but unfortunately have not been able to implement.
This email was triggered by email of Rene Pickhardt on lightning-dev mailinglist, regarding JIT Routing: https://lists.linuxfoundation.org/pipermail/lightning-dev/2019-March/001891.html
It may simplify implementation of such ideas, to have this re-architecturing of C-lightning.
Introduction
============
We may restructure C-lightning as a JSON-RPC bus, with the component parts of C-lightning (bitcoind interface, HTLC forwarding component, invoices component, etc.) interacting with one another by issuing commands over the same JSON-RPC interface that external users of C-lightning use.
If deemed necessary we may add a wrapper around this JSON-RPC bus interface to whitelist only specific commands, if we think that a separation of plugin from application would be desirable.
Alternately we can consider that an application built on top of C-lightning might itself conform to the C-lightning architecture by implementing itself as a set of plugins that interact with each other over the JSON-RPC bus.
With this architecture, plugins need not have any special relationship with the lightningd process.
They can be independent programs that are launched, stopped, and restarted independently of lightningd.
Existing plugins that use the current plugin protocol (interacting with lightningd via stdin/stdout which provides commands using a variant of JSON-RPC) can be supported by a new plugin program that wraps the proposed interface.
Motivating Examples
===================
We might want to allow extreme flexibility in adding new code, to allow for greater freedom to experiment with the protocol.
These are basically our justifications for why we would willingly inflict the pain of rearchitecturing C-lightning on ourselves.
Below are a few motivating examples.
Blockchain Interaction
----------------------
Currently we can only work with a specific backend, a bitcoind fullnode.
It would be desirable to be able to have blockchain interaction be replaceable.
For instance, we might want to support some other backend, or interact with an ElectrumX server, or use an implementation of neutrino.
Under the proposed architecture, we would define a set of commands that interact with the blockchain.
All components of lightningd that need interaction with the blockchain (monitoring transactions, spendedness of UTXOs, broadcasting transactions) would do so by issuing these commands over the JSON-RPC bus.
We would have a preinstalled plugin which provides these commands.
This preinstalled plugin, like the current system, would assume an installed `bitcoin-cli`.
However, new plugins may override these commands, and provide interaction with another backend.
Experimental TLV Types In Onion
-------------------------------
The TLV systems allows onion packets to contain new TLV types that are not yet defined in the standard, but which may be used for experimental purposes before proposing to be standardized.
It would be desirable if we could also move experimentation for new TLV types from lightningd to a plugin.
Under the proposed architecture, we would define a command `processoniontlv`.
By default this command simply fails.
When an incoming HTLC onion has a TLV we do not recognize, we call this command.
If it fails and the TLV is even, we fail the incoming HTLC.
Otherwise we continue with forwarding/receiving as normal.
New plugins may override this command using Chain-of-Responsibility Pattern, and from this can gather information from the onion TLV.
With sufficient further internal APIs also over the bus, such plugins may do anything with C-lightning while processing the TLV.
Command Usage Statistics
------------------------
In principle an autopilot is already implementable even without plugins.
However, an autopilot program might want to learn more about how the lightningd is used.
In particular it probably wants to know how often the user pays to a particular destination, and how much, and so on.
At the same time, it would be difficult a priori for us to determine what exact statistics would be useful for such autopilot programs.
It would be best if the decision of what statistics to gather, which would best guide the autopilot algorithm, is left to the autopilot implementer.
For instance, forwarding statistics may also be of use to an autopilot to determine which channels it could close (so it has enough onchain funds to open a channel to a commonly-paid destination, for example).
Under this proposed architecture, the autopilot can override the `pay` command using Decorator Pattern.
It would gather whatever statistics it needs from invocations of `pay`, then invoke whichever `pay` command was implemented at the time.
By using Decorator Pattern, various other components may gather statistics from the same command, if ever some other future component would want to know about `pay` command usage.
Forwarding Enhancements
-----------------------
Rene Pickhardt proposal of "JIT Routing" may be useful.
However, it requires some hook into the HTLC forwarding component.
Further, it should first be experimented in order to evaluate its usefulness before we consider actually enabling it by default.
It would be best if a plugin could perform this while evaluating the usefulness.
Under this proposed architecture, the HTLC forwarding component would invoke a `gothtlcwillforward` command.
It would wait for this command to finish before actually proposing an HTLC to the next hop in the onion route, and fail the incoming HTLC if the command fails.
The default implementation of this command would simply succeed without doing anything.
A plugin may override this command.
The plugin will, if this command is invoked, check if the next channel has enough capacity, and if not, will attempt rebalancing.
I believe Rene is already working on a rebalancing plugin, so the "JIT Routing" plugin would simply invoke a `rebalance` command, without having to implement rebalancing itself.
Bus Architecture
================
In essence, we would transform C-lightning so that it has a centralized core.
This is because we are secretly Blockstream lizards who want to centralize everything Bitcoin-related.
This central core would be a JSON-RPC interface.
The base JSON-RPC interface would simply be composed of `bus:` commands.
These are used to create command queues, and bind commands to specific command queues.
Existing `AUTODATA` command definitions can be gathered by an internal plugin which registers commands with the `bus:` interface and forwards the interactions appropriately.
Command Queues
--------------
A command queue represents a point by which a plugin can receive commands over the JSON-RPC interface.
The plugin creates one or more queues, one for each command it wishes to implement/override.
Then, at a later stage it binds a command queue to a command.
### `bus:makequeue`
Creates a command queue.
Arguments:
* `pid` - the process ID of the process that registers this queue.
The process must be running at the time this command is received by `lightningd`.
If the process dies, the queue is automatically destroyed.
When the queue is destroyed, it will be removed from the processing chain of a command it is bound to.
Any commands that have been dequeued but not replied will be failed.
Any commands that are still in the command queue will be forwarded to the next in the chain, or failed if this queue was the last in a chain.
Return:
* `qid` - the ID of the new queue, a string.
### `bus:command`
Issues a command to a command queue ID.
This command blocks until the command issued is replied to.
Arguments:
* `qid` - the command queue ID to issue commands to.
* `params` - the parameters to pass to the command.
Return:
* `result` - if existing, the command succeeded, and this is the result of the command.
* `error` - if existing, the command failed, and this is the error of the command.
### `bus:dequeue`
Gets a command from the specified command queue.
Blocks if the command queue has no commands to process.
Arguments:
* `qid` - the command queue ID to get a command from.
Return:
* `params` - the parameters of a command to be processed.
* `cid` - a string ID for replies to the command.
### `bus:reply`
Responds to a command.
Arguments:
* `cid` - the command ID to reply to.
* `result` - if present, indicates the command succeeded, and this is the result.
* `error` - if present, indicates the command failed, and this is the error.
Must be an error result object as per JSON-RPC 2.0.
Only one of `result` or `error` must be used.
Returns an empty result object on success.
### `bus:forward`
Issues a command to a command queue ID.
The command will then reply to a command that is pending.
This basically implements a "tail call", and leaves the responsibility for replying to the dequeued command to the queue ID.
This should generally be preferred over `bus:command`, unless you are using the Proxy pattern and need to augment the result of an existing command with additional or changed information.
If you find yourself using `bus:command` and then immediately issuing a `bus:reply` without augmenting the result of the `bus:command`, use `bus:forward` instead.
Arguments:
* `qid` - the command queue ID to forward the command to.
* `params` - the parameters to pass to the command.
* `cid` - the dequeued command ID that will be responded to by the command queue ID that is forwarded to.
Returns an empty result object on success.
Commands
--------
A command represents a linked list of command queues.
+---------------+ +---------------+
command ---> |-Q-+ +-> | ---> |-Q-+ +-> | --> "unrecognized command" error
| | | | | | | |
| v | | | v | |
+---------------+ +---------------+
| ^ | ^
| | | |
qid nextqid qid nextqid
| | | |
v | v |
+---------------+ +---------------+
| plugin | | plugin |
+---------------+ +---------------+
In actuality, a command is enqueued in a linked list node above.
Each linked list node has a "Q", which is the actual FIFO structure that holds commands for processing.
It is not actually possible to issue commands to a plugin-created `qid` directly.
Instead, the `bus:command` only accepts a `nextqid`, which represents "whatever node is the next node of a particular linked list node".
Such `nextqid`s are created by the `bus:bind` command, which binds a plugin `qid` to the front of the linked list.
The linked list structure above allows easy implementation of Decorator, Chain of Responsibility, and Proxy patterns.
When a plugin dies and destroys all queues it has created, any linked list nodes that have been bound to those queues will be removed.
When a linked list node is destroyed:
* Any commands in its queue that have not been dequeued yet will be forwarded to the next node.
* Any commands in its queue that have been dequeued but not replied by the plugin will be failed and the corresponding `cid` destroyed.
### `bus:bind`
Binds a queue ID to a command.
The queue ID must have been created via `bus:makequeue`, and must not have already been bound to a command by a previous `bus:bind`.
The queue ID is put to the front of the list (unless `fallback` is `true`).
Arguments:
* `method` - the name of the command to bind.
Methods starting with `bus:` prefix are privileged and cannot be overridden or bound.
* `helpdesc` - the help string to provide for this command.
The help string used is always the help string for the first node.
* `new` - fail if `true` and the command already has at least one binding.
Explicitly set this to false if you are expecting that your plugin is the only one that is implementing this command.
Optional, defaults `true`.
* `fallback` - if `true`, the binding should be added to the end of the linked list.
Optional, defaults `false`.
Returns:
* `nextqid` - the string queue ID for the next handler in the chain.
More information about the c-lightning
mailing list