[SLOF] [PATCH slof v3 5/5] fdt: Pass the resulting device tree to QEMU

[Alexey Kardashevskiy]

On 04/10/17 12:02, David Gibson wrote:
> On Tue, Oct 03, 2017 at 11:13:51PM +1100, Alexey Kardashevskiy wrote:
>> On 03/10/17 16:15, Alexey Kardashevskiy wrote:
>>> This creates flatten device tree and passes it to QEMU via a custom
>>> hypercall right before jumping to RTAS.
>>>
>>> On a machine with 256 CPUs and 256 virtual Intel E1000 devices the blob
>>> is 360KB (356KB structs and 20KB of strings), building such a tree takes
>>> ~2s on a POWER8 box. A simple tree with 1 CPU and a couple of devices
>>> takes 38ms and creates 16KB blob.
>>>
>>> This preloads strings with 40 property names from CPU and PCI device nodes
>>> and the strings lookup only searches within these. Without string reusing
>>> at all, the strings blob is 200KB and rendering time is 1.7sec; with
>>> unlimited reusing, the strings blob is 4KB and rendering time is 2.8sec.
>>>
>>> While we are here, fix the version handling in fdt-init. It only matters
>>> a little for the fdt-debug==1 case though.
>>>
>>> Signed-off-by: Alexey Kardashevskiy <aik at ozlabs.ru>
>>> ---

[...]

Adding Ben to cc:.

>>
>> Doing as below still works but brings the time for the huge guest (256
>> CPUs, 256 E1000) down to 366ms as it does not search for properties every
> 
> Ah, much better.  That's a cost I can live with.


I did some more debugging. There is a noticeable delay when booting such a
big guest, I cannot really measure prom_init() as the "jiffies" symbol is
not available there but a stopwatch shows 8.5sec in the guest's
flatten_device_tree() which fetches the entire device tree via the client
interface, node-by-node, property-by-property. With two small patches
(below, for the guest kernel and for SLOF) the delay is not noticeable any
more.

The questions are:

1. are we interested in such optimization?

2. do I have to add lunux,phandle as well, or just "phandle" is enough (it
is now for my tests)

3. what should allocate the fdt - the guest or slof? In my draft - it is
the guest as it simply reserves 1MB for this and that's it

4. reserved map - SLOF puts a zero and my draft adds the real map to the
end - is that hacky or ok?







diff --git a/arch/powerpc/kernel/prom_init.c b/arch/powerpc/kernel/prom_init.c
index 613f79f03877..6102eb682c71 100644
--- a/arch/powerpc/kernel/prom_init.c
+++ b/arch/powerpc/kernel/prom_init.c
@@ -2466,6 +2466,19 @@ static void __init flatten_device_tree(void)
                prom_panic("Can't allocate initial device-tree chunk\n");
        mem_end = mem_start + room;

+       if (!call_prom_ret("fdt-fetch", 2, 1, NULL, mem_start, room)) {
+               hdr = (void *) mem_start;
+               dt_header_start = (unsigned long)hdr;
+               hdr->boot_cpuid_phys = cpu_to_be32(prom.cpu);
+               hdr->off_mem_rsvmap = hdr->totalsize;
+               rsvmap = (void*)hdr + be32_to_cpu(hdr->totalsize);
+               hdr->totalsize = be32_to_cpu(hdr->totalsize) +
sizeof(mem_reserve_map);
+               hdr->totalsize = cpu_to_be32(hdr->totalsize);
+               memcpy(rsvmap, mem_reserve_map, sizeof(mem_reserve_map));
+               mem_reserve_cnt = MEM_RESERVE_MAP_SIZE;
+               return;
+       }
+
        /* Get root of tree */
        root = call_prom("peer", 1, 1, (phandle)0);
        if (root == (phandle)0)




diff --git a/slof/fs/client.fs b/slof/fs/client.fs
index 7d537a6..67976fb 100644
--- a/slof/fs/client.fs
+++ b/slof/fs/client.fs
@@ -308,4 +308,18 @@ ALSO client-voc DEFINITIONS
 : set-callback ( newfunc -- oldfunc )
   client-callback @ swap client-callback ! ;

+\ Curtom method to get FDT blob
+: fdt-fetch ( buf len -- 0 )
+    " /" find-node fdt-flatten-tree ( buf len dtb )
+    dup >r
+    >fdth_tsize l@              ( buf len size r: dtb )
+    min                         ( buf minsize r: dtb )
+    r@ -rot
+    move
+
+    r> fdt-flatten-tree-free
+
+    0
+;
+




-- 
Alexey

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 839 bytes
Desc: OpenPGP digital signature
URL: <http://lists.ozlabs.org/pipermail/slof/attachments/20171013/9a3ce9d4/attachment.sig>