[c-lightning] Towards an autopilot

[ZmnSCPxj]

Good morning Rusty,

Thank you for your input!

> ZmnSCPxj at protonmail.com writes:
> 
> > Good morning c-lightningers,
> > 
> > Ideally, a nontechnical human user will simply cause funds to be sent to their c-lightning node, and then some time after be able to send funds on Lightning.
> > 
> > A step towards that ideal, is an autopilot (similar to that of lnd) which manages onchain funds and opens channels to the network as it detects funds.
> > 
> > So I propose an autopilotd, of the below architecture
> 
> I think we want something much simpler.
> 
> 1.  Initial setup: get me a channel or two.
> 2.  I want to make a payment. Figure it out.

Unfortunately I do not think it is a good idea to make channels only when the human user makes a payment.  Making a channel can take several hours to confirm for a bad feerate, and we do not have RBF support yet to fix bad feerates after-the-fact.  Better I think to make channels as soon as we get onchain funds to make them.  In particular, if somebody else closes a channel where we have quite a bit of money, we would want to put them back onLightning ASAP I think.

>     
>     The setup should use a heuristic like:
>     
>     a. If we would burn more than 1% of funds to open a channel, only open 1.
>     
>     b. Otherwise, open two (or three?).

My planned heuristic is to have a minimum and maximum channel satoshi.  If the onchain funds exceeds the minimum, then we open 1.  If the onchain funds exceeds the maximum, then we open at least one (at most of the maximum size, but somewhere between minimum and maximum), then we re-check again later when the fundchannel succeeds (or if we fail to fund a channel).


>     
>     We should agree on a proximity algorithm which is likely to create an
>     
>     efficient robust network (Cyclic Superhubs?). This needs more research!
>     
>     In addition, we should avoid nodes which are too large, too new or too
>     
>     unreliable (how do we measure this?).

More statistics for the c-lightning to keep, of course!

Too large - can already judge from routemap from `listchannels`.  We already record channel capacity, we just need to report this in `listchannels` too.  We can judge a node as too large by summing up the channel capacities of all its channels listed in `listchannels` command.

Too new - we could store on DB the earliest time we saw a `node_announcement`.  However when a `node_announcment` is pruned we should take care to also prune it in the database.  We could add a new `listnodes` command that reports nodes, address hints for that node if any, and the earliest we heard about the node.

Too unreliable - when a payment fails, we know the erring node and we know the node after the erring node.  The node after the erring node is the one that is unreliable (the erring node is the one that reported the error, so we know it is reliably alive and responsive).  Keep a counter for each node that is just the number of times it was pointed at by the erring node as failing.  We get some noise due to channel balance issues, but this is better than nothing.  We could also increase the unreliability counter (or have a separate counter) if we connect to the node and fail.



>This bias against new nodes
>     
>     should weaken for successive channels,

Under my proposed architecture, you can simply apply another algorithm to be executed after the other algorithms.  That algorithm would check `listpeers` to know how many channels it has.  If it has none, it filters out too new/too large/too unreliable from the array of proposed fundees.  If it has a small number (one or two, or some tweakable setting) it can move them to the end of the array (so that they get funded last if ever), or flip a weighted coin (weighed for 100% removed if channels is 0, reducing greatly as the number of channels increases) to remove them from the array of proposed fundees.

Indeed, this is why I proposed my architecture: it allows addition of various heuristics in ways that humans can reason about.

>but after the first channel we
>     
>     might want a single channel to a "nearby" node in network latency on
>     
>     the theory that most payments (eventually) will be meatspace-local.

Yet another algorithm that can be plugged into the wrangler.  Ping nodes periodically, then rank proposals according to latency, or add very low-latency peers.  Pinging TOR nodes is somewhat more difficult but we could try a open-connection request and time it, to be used as ping.

>     
>     The payment should use a heuristic like:
>     
>     a. Can I send like normal?
>     
>     b. Can I rebalance channels to send?
>     
>     c. Do I need to set up a new channel?
>     

Seems more appropriate for `lightningd/payalgo`.

>     I agree with your instincts: this effort is really separate from the
>     
>     core c-lightning. It's really a user, and I'm not even sure if it
>     
>     should be in the same git repo.

I already have a pilotd in my git repo, it could be moved off into a separate project but for now is strongly dependent on many c-lightning `common/` code....

Regards,
ZmnSCPxj