Request for comments: C++ embedded webserver

[Michael.E.Brown at dell.com]

Yes, my server is posted on my personal github.... however, usual caveats apply: this was done on my personal time in order to teach myself 'go' to see if it is worthwhile to pursue. It is nowhere near "production" quality and I am actively working on getting some actual developer time to get it all fleshed out. https://github.com/superchalupa/go-redfish/

I used a couple of frameworks to try them out, most of it could be cut out and trimmed down a bit, but it has middleware that provides logging and metrics that have proved very handy.

Design philosophy (the succinct, non-book version):

It is based on a standard DDD/CQRS/Event Source design, though it is going to need some modifications because standard Event Sourcing won't really work well in embedded: we'll need a way to coalesce events as we cannot really keep the entire event history forever. If you are familiar with DDD/CQRS, the quick way to explain it is that each Redfish Resource (ie. Redfish URI) is an "aggregate". I've defined the  following "commands", and each command has a corresponding event:
	CreateRedfishResource
	UpdateRedfishResourceProperties
	RemoteRedfishResourceProperty
	RemoveRedfishResource

The sort of high level philosophy is that we keep the JSON data in memory (as a golang map[string]interface{}) so that most reads are satisfied without ever going out of process. We have an internal event bus where we take external events and publish them on the internal bus for various modules to decide what to do.

Here is what I think is the important part: since it is an event bus and we have the JSON tree structure in memory, you can easily add OEM extensions. The OEM extension simply listens for the creation of objects that it is interested in and can then add new data to the object without conflicting with existing code. This is all very similar in spirit to what you are currently doing with DBUS, and I think these two could very well marry up nicely.

The startup process: to create the initial tree of JSON output, we generically read in a redfish dump and do the initial tree creation. This makes for easy customizability because you can simply edit the text JSON files on disk to change them. I'd like to see if there is a way to do the data mappings inside the initial JSON files as well.

On the other side of this thread, you have made what I think is a really excellent rundown of the types of data we will need to pull for redfish, so let me move over to that side to address some of those issues.

And finally: the existing REST interface that generically exposes DBUS: I think this could be implemented in go as well, though that's a completely separate issue. I have security concerns with how we expose DBUS externally and my overall impression is that it would be much nicer to have our UI sit on top of redfish, though I understand that is a fairly big lift.
--
Michael


-----Original Message-----
From: Tanous, Ed [mailto:ed.tanous at intel.com] 
Sent: Thursday, December 21, 2017 12:47 PM
To: Brown, Michael E <Michael_E_Brown at Dell.com>; benh at kernel.crashing.org; openbmc at lists.ozlabs.org
Subject: RE: Request for comments: C++ embedded webserver

Can you post your prototype server somewhere that's accessible externally?   I'm genuinely curious how it compares.

I had looked at golang, but there was no precedence for it in the project, nor did I have any experience with it, so I went another direction.  I would love if a language change made all the async handling easier to write than what's there.  My gut tells me to be concerned about the performance of the go garbage collection model on the AST using too much memory for some of the lesser capable systems to handle, but I have zero data on it so I'll withhold judgement until I have some.

-Ed

> -----Original Message-----
> From: Michael.E.Brown at dell.com [mailto:Michael.E.Brown at dell.com]
> Sent: Thursday, December 21, 2017 9:25 AM
> To: benh at kernel.crashing.org; Tanous, Ed <ed.tanous at intel.com>; 
> openbmc at lists.ozlabs.org
> Subject: RE: Request for comments: C++ embedded webserver
> 
> Benjamin,
> 
> I'm new around here, so I didn't want to be the first to bring this up 
> without doing my research on any older threads, so thank you for bringing this up.
> 
> My strong opinion is that golang is likely the best candidate for this 
> specific type of programming environment (ie. OpenBMC).
> 	- Low runtime memory usage
> 	- Automatic Memory Management
> 	- Clean cross compile for ARMv5 and above
> 	- Compiled
> 	- strongly typed
> 	- Fast
> 	- Cleanly handles threading/async
> 	- Nice standard library
> 
> I had prototyped a redfish server in golang enough to get to the point 
> where I believe that it's the best solution. I have great things to 
> say internally about the existing openbmc design: I love the use of 
> DBUS, the more I look at it the more clearly the advantages seem to 
> me. The nice thing about this is that golang has a DBUS library and we 
> ought to be able to write different components in different languages 
> freely, especially if we have the underlying dbus apis correct.
> 
> --
> Michael
> 
> -----Original Message-----
> From: openbmc [mailto:openbmc-
> bounces+michael.e.brown=dell.com at lists.ozlabs.org] On Behalf Of
> Benjamin Herrenschmidt
> Sent: Wednesday, December 20, 2017 6:34 PM
> To: Tanous, Ed <ed.tanous at intel.com>; OpenBMC Maillist 
> <openbmc at lists.ozlabs.org>
> Subject: Re: Request for comments: C++ embedded webserver
> 
> On Tue, 2017-12-19 at 04:07 +0000, Tanous, Ed wrote:
> > I’m looking for comments on a code review that’s been outstanding.
> > One of the large pushes we’ve made is to attempt to make the web 
> > server more efficient, and add capabilities that comprehend long 
> > term needs of OpenBmc.  One key that wasn’t made clear in the commit 
> > message is that it includes the basic redfish implementation that 
> > (we
> > hope) should be extensible to the full redfish specification in the 
> > short term future.
> 
> There are plenty of existing HTTP server implementations out there 
> that are well maintained, any reason why we should invent another one here ?
> 
> Also, do we really think perpetuating the use of that monstruosity 
> masquerading as a programming language that is C++ is a good idea ?
> 
> > https://gerrit.openbmc-project.xyz/#/c/7786/
> >
> > We would very much appreciate comments to see if we can move this
> forward.
> >
> > The following description is pulled from the readme.
> >
> > # OpenBMC webserver #
> >
> > This component attempts to be a "do everything" embedded webserver
> for openbmc.
> >
> > ## Capabilities ##
> > At this time, the webserver implements a few interfaces:
> > + Authentication middleware that supports cookie and token based
> authentication, as well as CSRF prevention backed by linux PAM 
> authentication credentials.
> > + An (incomplete) attempt at replicating phosphor-dbus-rest 
> > + interfaces in
> C++.  Right now, a few of the endpoint definitions work as expected, 
> C++but
> there is still a lot of work to be done.  The portions of the 
> interface that are functional are designed to work correctly for 
> phosphor-webui, but may not yet be complete.
> > + Replication of the rest-dbus backend interfaces to allow bmc debug 
> > + to
> logged in users.
> > + An initial attempt at a read-only redfish interface.  Currently 
> > + the redfish
> interface targets ServiceRoot, SessionService, AccountService, Roles, 
> and ManagersService.  Some functionality here has been shimmed to make 
> development possible.  For example, there exists only a single user role.
> > + SSL key generation at runtime.  If an RSA key and cert pair are 
> > + not
> available to the server at runtime, keys are generated using the 
> openssl routines, and written to disk.
> > + Static file hosting.  Currently, static files are hosted from the 
> > + fixed location
> at /usr/share/www.  This is intended to allow loose coupling with 
> yocto projects, and allow overriding static files at build time.
> > + Dbus-monitor over websocket.  A generic endpoint that allows UIs 
> > + to open a websocket and register for notification of events to 
> > + avoid polling in single page applications.  (this interface may be 
> > + modified in the future due to security concerns.)
> >
> >
> > Thanks,
> >
> > -Ed
> >