[c-lightning] Towards atomic multipath payments

Wed Nov 21 02:47:10 AEDT 2018

Hi ZmnSCPxj,

the invoices counterpart made me think about how much data we already
have that we can use without adding any additional persistence at
all. In the case of the invoices we already have the incoming HTLCs with
the matching payment_hashes, so maybe we don't need any extra persisted
data at all. That would then boil down to summing up all HTLCs that have
this payment_hash, and that's how much we have guaranteed. Taking the
mirrored case on the sender side and counting all committed to HTLCs
with the same payment_hash might work as well, given that we write a
`payments` entry when starting to do the payment, and we can subtract
the outgoing HTLC values for that payment_hash, and the info for
resuming the rest of the partial payments is also in the `payments`
table.

Now that works nicely as long as the payment_hashes match across all
HTLCs, this might no longer be the case for the OG-AMP, which varies the
payment_hashes, but with enough added information in the `payments`
table we should be able to map incoming and outgoing HTLCs to a payment
or invoice respectively and account them properly, without needing to
persist them separately.

All of this is merely for the data persistence, and I quite like the
idea of adding some details to the RPC that explains the details, by
adding partial payments, but we can source them from the in-memory view
of the HTLCs.

Cheers,
Christian

ZmnSCPxj <ZmnSCPxj at protonmail.com> writes:
> Good morning Christian and other c-lightningers,
>
> On the opposite end is the reception and claiming of payments that were given as AMP.
> Now, the primary interface here is the invoices system, primarily `listinvoices`.
> Relevant to AMP, is the "is it paid already", "how much did they pay exactly", and "when did they pay?" fields.
>
> Like `listpayments`, I want to preserve the property, that `listinvoices` is a user-facing "high-level" interface.
> I wonder, however, if some low-level interface to the HTLC-claiming would be good, at least for extra reporting and curiosity ("is the payer even *trying* to pay me???").
>
> I would like to propose a command `listpartialinvoices`.
> `listinvoices` -> `listpartialinvoices` :: `listpayments` -> `listpartialpayments`.
> (But the command name feels strange to me...)
>
> The purpose of this command is simply to list incoming payment attempts that we have not claimed yet because they are partial (and we are not economically incentivized to release the payment preimage for an incomplete payment).
> So for example it might return something like:
>
> [ { payment_hash: "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"
>   , total_amount_msat: 42
>   , incoming_scid: "99999x99x9"
>   , partial_amount_msat: 6
>   , incoming_time_unix: 5555555
>   }
> , { payment_hash: "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"
>   , total_amount_msat: 42
>   , incoming_scid: "99999x99x9"
>   , partial_amount_msat: 9
>   , incoming_time_unix: 5555556
>   }
> ]
>
> Then, when all the incoming HTLCs have totalled to equal or greater than the `amt_to_forward` (in BOLT 4 "Base AMP" pull request; or `total_amount_msat` above), we claim all of the money, update the invoice to "paid", and remove the partial invoice entries.
>
> Regards,
> ZmnSCPxj
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Tuesday, November 20, 2018 12:18 AM, Christian Decker <decker.christian at gmail.com> wrote:
>
>> Hi ZmnSCPxj,
>>
>> thanks for digging in and getting the ball rolling on this. Indeed I
>> think your analysis that we are using the `payments` table for multiple
>> different purposes that now with AMP diverge is spot on. I think the
>> introduction of partial payments is good and we can build on top of
>> that, and maybe we should make the `payments` table a purely user-facing
>> table with entries about their transaction history (that was my initial
>> goal when it included incoming payments matching invoices as well).
>>
>> Cheers,
>> Christian
>>
>> ZmnSCPxj ZmnSCPxj at protonmail.com writes:
>>
>> > Good morning c-lightningers,
>> > I would like to propose a JSON-RPC interface for low-level atomic
>> > multipath payments.
>> > Currently exists, the `sendpay`/`waitsendpay`/`listpayments`.
>> > `sendpay` and `waitsendpay` are low-level interfaces for making a
>> > single payment attempt. `listpayments` indicates status of payments.
>> > These interfaces manage a table of payment attempts. `sendpay` will
>> > create one, or reuse a payment attempt if it is marked as failing.
>> > `waitsendpay` waits for a specific payment attempt to change its
>> > status to either completed or failing.
>> > This table of payment attempts is used also to provide the high-level `pay` interface.
>> > The `pay` interface leaves the payment attempt on the table if it succeeds, and just reuses the same entry (from the lower-level `sendpay` behavior) to keep retrying.
>> > Thus `listpayments` serves as both a high-level and low-level interface.
>> > Currently those interfaces are keyed to the `payment_hash`.
>> > Each payment attempt has a unique `payment_hash`.
>> > For simplicity to JSON-RPC users, I want to keep the existing `sendpay`/`waitsendpay` interfaces as-is without adding special multipath options to them.
>> > However, atomic multipath means making separate attempts simultaneously with the same `payment_hash`, and the `sendpay`/`waitsendpay` are keyed according to `payment_hash`.
>> > Further, the `listpayments` table also serves as a high-level interface, and I think atomic multipath should be low-level and not be exposed on `listpayments`.
>> > So I want to propose a separate interface:
>> > `sendpartialpay` `route` `payment_hash` `intended_msatoshi` [`description`]
>> > =>
>> > { "id" : 42, ... }
>> > `waitpartialpay` `id`
>> > `listpartialpayments` [`id`]
>> > This handles a separate table for partial payments.
>> > There is a many-to-one mapping from partial payments in `listpartialpayments` to payments in `listpayments`.
>> > Then, `sendpartialpay` does:
>> >
>> > 1.  Check if there already exists a payment attempt with same `payment_hash` in `listpayments` table.
>> >     1.1. If there is not, create it, with the `msatoshi` equal to the `intended_msatoshi` for this `sendpartialpay` call.
>> >     1.2. Otherwise:
>> >     1.2.1. If the `intended_msatoshi` does not match the `msatoshi` in the `listpayments` table, fail immediately and do nothing.
>> >     1.2.2. If the existing `listpayments` entry is completed, return immediately with success and do nothing.
>> >     1.2.3. If the existing `listpayments` entry is failed, change it to pending and continue.
>> >
>> > 2.  Create a payment attempt in `listpartialpayments` table, trigger the payment, and return the ID for the payment attempt.
>> >
>> > When a partial payment returns either `update_fulfill_htlc` or `update_fail_htlc`:
>> >
>> > 1.  Change the state of the `listpartialpayments` entry to completed/failed.
>> > 2.  If completed, also set the `listpayments` entry to completed and trigger any `waitsendpay` waiters.
>> > 3.  If failed, and all other partial payments to the same `payment_hash` failed, set the `listpayments` entry to failed and trigger any `waitsendpay` waiters.
>> > 4.  Trigger any `waitpartialpay` waiters.
>> >
>> > Regards,
>> > ZmnSCPxj
>> >
>> > ------------------
>> >
>> > c-lightning mailing list
>> > c-lightning at lists.ozlabs.org
>> > https://lists.ozlabs.org/listinfo/c-lightning