[Lguest] lguest: mapping switcher would thwack fixmap

[Rusty Russell]

Paul Bolle <pebolle at tiscali.nl> writes:
> On Mon, 2013-05-06 at 12:46 +0930, Rusty Russell wrote:
>> Paul Bolle <pebolle at tiscali.nl> writes:
>> > Please note that I've jumped to v3.9 in the meantime. It still triggers
>> > this triple fault. But note that CONFIG_MICROCODE_INTEL_EARLY=y causes
>> > the unhandled trap 13 error, so this .config will only work (ie, triple
>> > fault your machine), on AMD hardware. I hope to send a separate message
>> > for this last issue shortly. 
>> 
>> Hmm, neither for me.  When I remove CONFIG_MICROCODE, it just works
>> (well, it panics unable to mount root, but that's due to lack of
>> non-modular block device).
>> 
>> I'm running under KVM on Intel, using my latest kernel.
>
> Well, after doing desperate things, like studying the code, I've finally
> chosen a more structured approach: I've littered the lguest driver with
> printk's! And that helped me to pinpoint the problem here.
>
> See, basically the last thing I could see was a call to guest_set_pgd()
> with idx=1023. So, apparently the guest tells us the page table entry
> for the upper 4MB of the virtual address space has changed. But that is
> were the Switcher hangs out! And, somehow, the call to
> allocate_switcher_mapping() doesn't put the switcher's single page back
> into the guest's page tables.

That's bad...  Ah, I think I see it.  It will put back the switcher
text page, but doesn't clear the last_host_cpu field.  So the code
thinks it's still mapped, and doesn't remap it!

See below for one-liner fix?

> Anyhow, adding a line to make sure the Switcher is placed at a 4M
> boundary does the trick:
>     switcher_addr = (switcher_addr / 0x400000ul) * 0x400000ul;
>
> (That line was copied by hand.)

That's probably because the Guest then doesn't try to map anything
there, knowing it's reserved.  The bug is still latent though...

> And now I'm able to actually run a guest in qemu (that is, end up in a
> functional dracut emergency shell). That is way past the moment the
> guest would cause a Triple Fault beforehand.
>
> (The Intel early microcode stuff prevents me from running lguest on real
> hardware. I haven't yet recompiled my kernel. I'm glad to do so if you
> want additional testing.)
>
>
> Paul Bolle

lguest: clear cached last cpu when guest_set_pgd() called.

commit v3.9-rc1-53-g6d0cda9 "lguest: cache last cpu we ran on." missed
one case, which causes a triple fault.  The guest calls guest_set_pgd()
on the top page, and we carefully remap the Switcher text page.  But
we didn't reset last_host_cpu, so map_switcher_in_guest() thinks
the guest's regs and IDT/GDT etc are already mapped.

Reported-by: Paul Bolle <pebolle at tiscali.nl>
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>

diff --git a/drivers/lguest/page_tables.c b/drivers/lguest/page_tables.c
index 699187a..5b9ac32 100644
--- a/drivers/lguest/page_tables.c
+++ b/drivers/lguest/page_tables.c
@@ -1002,6 +1002,7 @@ void guest_set_pgd(struct lguest *lg, unsigned long gpgdir, u32 idx)
 			kill_guest(&lg->cpus[0],
 				   "Cannot populate switcher mapping");
 		}
+		lg->pgdirs[pgdir].last_host_cpu = -1;
 	}
 }