[c-lightning] plugin development

Wed Dec 19 05:07:46 AEDT 2018

OK, I don't think I am quite there yet

If I run from the cli `check hello` it kills the daemon

issue: `tok->type` is `JSMN_ARRAY` and not `JSMN_OBJECT` ref:
https://github.com/ElementsProject/lightning/blob/692d8e9685ce0c4e9293b31fca69115add7a479b/common/json.c#L147

What do I need to do to make it an object, or is this just an
oversight in the code?  Does the `check` command work against the
python code(my apologies, I know very little python and could not get
it to run)?
On Fri, Dec 14, 2018 at 12:05 PM Christian Decker
<decker.christian at gmail.com> wrote:
>
> Notice that this is already partially documented in the plugins.md file [1]. I say partially because it's missing the "configuration" key. I have a commit in my next PR that addresses this and adds documentation for the JSON-RPC passthrough.
>
>
> [1] https://github.com/ElementsProject/lightning/blob/master/doc/plugins.md#the-init-method
>
> On Fri, Dec 14, 2018 at 7:00 PM Richard Bondi <socketexception at gmail.com> wrote:
>>
>> Logs would help, so I created my own log to file and here is what the issue was
>>
>> the "init" message looks like this
>>
>> {  "jsonrpc": "2.0",   "method": "init",   "id": 1,   "params": {
>> "options": {      "greeting": "greeting"    },     "configuration": {
>>     "lightning-dir": "/home/richard/.lightning",       "rpc-file":
>> "lightning-rpc"    }  }}
>>
>> I was not expecting the "params" to be an object and the plugin was
>> crashing because of this, I was always trying to read as array of
>> strings, for the hello response and never imagined.  I am sure this
>> will lead me to a fix for the go version also.
>>
>> I think it would be good to document the the expected requests, I will
>> make a PR.
>>
>> Thank you both for your time and responses.
>> On Sun, Dec 9, 2018 at 10:48 PM Rusty Russell <rusty at rustcorp.com.au> wrote:
>> >
>> > Pretty sure Christian is right, you need to flush in your plugin, *or*
>> > you're not closing your JSON reply.  Both will give you hangs like this
>> > as c-lightning waits for the complete response.
>> >
>> > We should do I/O logging on plugins so that this kind of thing can be
>> > diagnosed more easily.
>> >
>> > Cheers,
>> > Rusty.
>> >
>> > Richard Bondi <socketexception at gmail.com> writes:
>> > > Additional info, it hangs here
>> > > https://github.com/ElementsProject/lightning/blob/8238fe6acfc7db9e397ecac21e9e82cdc25c40be/cli/lightning-cli.c#L322
>> > >
>> > > the `cmd` variable = 0x55555577d868 "{ \\"method\\" : \\"hello\\",
>> > > \\"id\\" : \\"lightning-cli-10719\\", \\"params\\" : [ \\"fred\\"] }"
>> > >
>> > > also, when I follow the `read` function in my ide, it takes me to
>> > > cli/test/run-large-input.c which does not look right, and I never seem
>> > > to hit a breakpoint there, what other info could I provide that would
>> > > be useful?
>> > > On Sat, Dec 8, 2018 at 11:53 AM Richard Bondi <socketexception at gmail.com> wrote:
>> > >>
>> > >> Thanks Christian,
>> > >>
>> > >> This is exactly the approach I took, except with formatted json
>> > >> response, but I updated to unformatted and the results I get are as
>> > >> expected from the shell, so it seems my assumptions were correct.
>> > >>
>> > >> However, I still have the issue as described.
>> > >>
>> > >> I created a repository
>> > >> https://github.com/rsbondi/clightning-dotnet-plugin/tree/9b0e73fa77691a40e3f2a010c8b8bff0deeb861a
>> > >>
>> > >> This shows in the gifs that in shell I get the responses expected, but
>> > >> when running clightning and calling commands from the cli, the result
>> > >> is not as expected
>> > >>
>> > >> > The JSON you posted got mangled somewhere, i.e., it's missing the
>> > >> > quotes. I think this might actually be the shell doing random things
>> > >> > with quotes. And testing with escaping or single-quoting the parameter
>> > >> > seems to work:
>> > >>
>> > >> My intention here was only to show that the plugin was registered and
>> > >> responding to the command, but only on errors(mangled json), not on
>> > >> valid input `lightning-cli hello fred`.
>> > >> On Sat, Dec 8, 2018 at 2:38 AM Christian Decker
>> > >> <decker.christian at gmail.com> wrote:
>> > >> >
>> > >> > Hi Richard,
>> > >> >
>> > >> > the plugins being simple executables that communicate over stdin and
>> > >> > stdout should be able to be started directly from the shell and you
>> > >> > should also be able to interact with the from there. So you can try
>> > >> > starting your plugin and issuing some JSON-RPC requests and see how it
>> > >> > reacts. For the initialization you should be able to do the following
>> > >> > (taking the helloworld.py plugin as an example:
>> > >> >
>> > >> > ```
>> > >> > {"jsonrpc":"2.0", "id":1, "method":"getmanifest", "params":[]}
>> > >> > {"jsonrpc": "2.0", "result": {"options": [{"name": "greeting", "type": "string", "default": "World", "description": "What name should I call you?"}], "rpcmethods": [{"name": "hello", "description": "Returns a personalized greeting for {name}"}, {"name": "fail", "description": "Always returns a failure for testing"}]}, "id": 1}
>> > >> > ```
>> > >> >
>> > >> > The first line is what you send, the second is the reply from the
>> > >> > plugin. After that you can issue the `init` call:
>> > >> >
>> > >> > ```
>> > >> > {"jsonrpc":"2.0", "id":2, "method":"init", "params": {"options": {"greeting": "Richard"}}}
>> > >> > {"jsonrpc": "2.0", "result": "ok", "id": 2}
>> > >> > ```
>> > >> >
>> > >> > And from hereon the plugin is registered and can start doing its
>> > >> > magic.
>> > >> >
>> > >> > > When I call the plugin it just hangs.  I have tried just ignoring all
>> > >> > > input and returning the same response as the init and getmanifest,
>> > >> > > still hangs.  It seems the passthrough is taking place, if I put in a
>> > >> > > command like
>> > >> >
>> > >> > This could have something to do with buffering, since some languages
>> > >> > will buffer output. Notice that the python3 plugin always adds newlines
>> > >> > and flushes to make sure it clears buffers. Not sure how the situation
>> > >> > is with C#, but python2 was acting up as well.
>> > >> >
>> > >> > > cli/lightning-cli hello {"greeting":"fred"}
>> > >> > >
>> > >> > > I get a response
>> > >> > >
>> > >> > > Invalid token in json input: '{ "method" : "hello", "id" :
>> > >> > > "lightning-cli-25109", "params" : [ {greeting:fred}] }'
>> > >> > >
>> > >> > > which make sense, if I do an echo passing the same json I get the
>> > >> > > broken json
>> > >> >
>> > >> >
>> > >> > The JSON you posted got mangled somewhere, i.e., it's missing the
>> > >> > quotes. I think this might actually be the shell doing random things
>> > >> > with quotes. And testing with escaping or single-quoting the parameter
>> > >> > seems to work:
>> > >> >
>> > >> > ```
>> > >> > lightning-cli hello '{"greeting":"fred"}'
>> > >> > "Hello {'greeting': 'fred'}"
>> > >> > ```
>> > >> >
>> > >> > > what I expect to happen is, when I type in
>> > >> > >
>> > >> > > cli/lightning-cli hello fred
>> > >> > >
>> > >> > > I expect my message with "fred" substituted in or I expect the command
>> > >> > > that I look for in code to be
>> > >> > >
>> > >> > > { "method" : "hello", "id" : "lightning-cli-25109", "params" : ["fred"] }
>> > >> >
>> > >> > The reason you get the full quoted text back is because `lightning-cli`
>> > >> > takes that first argument (the JSON serialized object) and passes it in
>> > >> > as the first positional parameter, and doesn't use the entire object as
>> > >> > the params (i.e., it doesn't unpack). So to get `Hello fred` you'd have
>> > >> > to run `lightning-cli hello fred` instead.
>> > >> >
>> > >> > > The other issue is when I try to do in go, it seems to call
>> > >> > > `plugin_read_json_one` more than once for the same plugin's
>> > >> > > getmanifest and I get the "Received a JSON-RPC response for
>> > >> > > non-existent request" error.  It seems to have the same buffer value,
>> > >> > > but no request.  I have gone as far in my code to remove getmanifest
>> > >> > > from the map so there is no way it could erroneously send the request
>> > >> > > a second time.
>> > >> >
>> > >> > This could happen in a couple of different ways: the plugin code issues
>> > >> > the request twice (this would be a lightningd bug), the plugin responds
>> > >> > multiple times (bug in the plugin), the plugin code doesn't clear the
>> > >> > buffer inbetween responses (lightningd issue).
>> > >> >
>> > >> > We can easily eliminate the second case by running the plugin from the
>> > >> > command line like explained above, and go from there.
>> > >> >
>> > >> > HTH,
>> > >> > Christian
>> > > --
>> > > c-lightning mailing list
>> > > c-lightning at lists.ozlabs.org
>> > > https://lists.ozlabs.org/listinfo/c-lightning