<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Nov 16, 2017 at 9:31 PM, Nathan Fontenot <span dir="ltr"><<a href="mailto:nfont@linux.vnet.ibm.com" target="_blank">nfont@linux.vnet.ibm.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
<br>
On 11/15/2017 11:37 PM, Bharata B Rao wrote:<br>
<div><div class="h5">> On Fri, Oct 20, 2017 at 6:51 PM, Nathan Fontenot <<a href="mailto:nfont@linux.vnet.ibm.com">nfont@linux.vnet.ibm.com</a> <mailto:<a href="mailto:nfont@linux.vnet.ibm.com">nfont@linux.vnet.ibm.<wbr>com</a>>> wrote:<br>
><br>
> This patch set provides a set of updates to de-couple the LMB information<br>
> provided in the ibm,dynamic-memory device tree property from the device<br>
> tree property format. A part of this patch series introduces a new<br>
> device tree property format for dynamic memory, ibm-dynamic-meory-v2.<br>
> By separating the device tree format from the information provided by<br>
> the device tree property consumers of this information need not know<br>
> what format is currently being used and provide multiple parsing routines<br>
> for the information.<br>
><br>
> The first two patches update the of_get_assoc_arrays() and<br>
> of_get_usable_memory() routines to look up the device node for the<br>
> properties they parse. This is needed because the calling routines for<br>
> these two functions will not have the device node to pass in in<br>
> subsequent patches.<br>
><br>
> The third patch adds a new kernel structure, struct drmem_lmb, that<br>
> is used to represent each of the possible LMBs specified in the<br>
> ibm,dynamic-memory* device tree properties. The patch adds code<br>
> to parse the property and build the LMB array data, and updates prom.c<br>
> to use this new data structure instead of parsing the device tree directly.<br>
><br>
> The fourth and fifth patches update the numa and pseries hotplug code<br>
> respectively to use the new LMB array data instead of parsing the<br>
> device tree directly.<br>
><br>
> The sixth patch moves the of_drconf_cell struct to drmem.h where it<br>
> fits better than prom.h<br>
><br>
> The seventh patch introduces support for the ibm,dynamic-memory-v2<br>
> property format by updating the new drmem.c code to be able to parse<br>
> and create this new device tree format.<br>
><br>
> The last patch in the series updates the architecture vector to indicate<br>
> support for ibm,dynamic-memory-v2.<br>
><br>
><br>
> Here we are consolidating LMBs into LMB sets but still end up working with individual LMBs during hotplug. Can we instead start working with LMB sets together during hotplug ? In other words<br>
<br>
</div></div>In a sense we do do this when handling memory DLPAR indexed-count requests. This takes a starting<br>
drc index for a LMB and adds/removes the following <count> contiguous LMBs. This operation is<br>
all-or-nothing, if any LMB fails to add/remove we revert back to the original state.<br></blockquote><div><br></div><div>I am aware of count-indexed and we do use it for memory hotplug/unplug for KVM on Power. However the RTAS and configure-connector calls there are still per-LMB.<br><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Thi isn't exactly what you're asking for but...<br>
<span class="">><br>
> - The RTAS calls involved during DRC acquire stage can be done only once per LMB set.<br>
> - One configure-connector call for the entire LMB set.<br>
<br>
</span>these two interfaces work on a single drc index, not a set of drc indexes. Working on a set<br>
of LMBs would require extending the current rtas calls or creating new ones.<br></blockquote><div><br></div><div>Yes.<br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
One thing we can look into doing for indexed-count requests is to perform each of the<br>
steps for all LMBs in the set at once, i.e. make the acquire call for LMBs, then make the<br>
configure-connector calls for all the LMBs...<br></blockquote><div><br></div><div>That is what I am hinting at to check the feasibility of such a mechanism. Given that all the LMBs of the set are supposed to have similar attributes (like node associativity etc), it makes sense to have a single DRC acquire call and single configure-connector call for the entire set.<br><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
The only drawback is this approach would make handling failures and backing out of the<br>
updates a bit messier, but I've never really thought that optimizing for the failure<br>
case to be as important.<br></blockquote><div><br></div><div>Yes, error recovery can be messy given that we have multiple calls under DRC acquire call (get-sensor-state and set-indicator).<br><br></div><div>BTW I thought this reorganization involving ibm,drc-info and ibm,dynamic-memory-v2 was for representing and hotplugging huge amounts of memory efficiently and quickly. So you have not yet found the per-LMB calls to be hurting when huge amount of memory is involved ?<br><br></div><div>Regards,<br></div><div>Bharata.</div></div><br></div></div>