mapping memory in 0xb space

[David Gibson]

On Tue, Sep 28, 2004 at 01:52:16PM -0500, Igor Grobman wrote:
> On Tue, 28 Sep 2004, David Gibson wrote:
> >
> > Err... this seems likely to cause trouble.
> 
> So it has :-)
> 
> >  Recent kernels don't even
> > have VSIDs allocated for the 0xb... region.
> 
> Looking at both 2.6.8 and 2.4.21, I don't see a difference in
> get_kernel_vsid() code.

Ok, *very* recent kernels.  The new VSID algorithm has gone into the
BK tree since 2.6.8.

> >  2.4.21 will have some
> > kernel VSIDs there, but there will probably be a bunch of places that
> > assume addresses below 0xc... are user addresses and compute the vsids
> > accordingly.
> 
> Note that since I put in a bolted entry into the HPT, I assume I don't
> have to worry about page fault handling code.  

That sounds correct.

> This leaves segments.  Both
> DataAccess_common and DataAccessSLB_common call
> do_stab_bolted/do_slb_bolted when confronted with an address in 0xb
> region.

Oh, so it does.  That, I think is a 2.4 thing, long gone in 2.6 (even
before the SLB rewrite, I'm pretty sure do_slb_bolted was only called
for 0xc addresses).

> Presumably, this will fault in the segments I am interested in.

Yes, actually, it should.  Ok, I guess the problem is deeper than I
thought.

> > Differences in the code for segment table and SLB
> > machines are probably why it works (or seems to) on power3 but not
> > power4.
> 
> SLB machines are LPAR machines, correct?

Not necessarily.

> In that case, neither the power3
> nor the power4 I am using are SLB machines.  

No, power4 is definitely SLB.

> Also, I narrowed it down to
> working (or appearing to work) as long as the highest 5 bits of the page
> index (those that end up as partial index in the HPTE) are zero.  This may
> just be a weird coincidence.

Could be.

> > Also, unless you've created a new set, there are no Linux page tables
> > for the 0xb region - the only linux page tables for the kernel are for
> > the vmalloc (0xd) and ioremap (0xe) regions.
> 
> I am building page tables off the bolted_dir, which does exist in 2.4.21
> for the 0xb region.  But again, assuming I insert a bolted entry in HPT,
> do I even need to worry about linux page tables as such?

Oh, so they do.  But yes, you shouldn't need to worry about them.

> > Why on earth do you want to do this?
> 
> Good question ;-).  A long long time ago, I posted on this list and
> explained.  Since then, I found what appeared to be a solution, except
> that it appears power4 breaks it.  I am building a tool that allows
> dynamic splicing of code into a running kernel (see
> http://www.paradyn.org/html/kerninst.html).  In order for this to work, I
> need to be able to overwrite a single instruction with a jump to
> spliced-in code.  The target of the jump needs to be within the range (26
> bits).  Therefore, I have a choice of 0xbfff.. addresses with backward
> jumps from 0xc region, or the 0xff.. addresses for absolute jumps.  I
> chose 0xbff.., because I found already-working code, originally written
> for the performance counter interface.  Am I making more sense now?

Aha!  But this does actually explain the problem - there are only
VSIDs assigned for the first 2^41 bits of each region - so although
there are vsids for 0xb000000000000000-0xb00001ffffffffff, there
aren't any for 0xbff... addresses.  Likewise the Linux pagetables only
cover a 41-bit address range, but that won't matter if you're creating
HPTEs directly.

You may have seen the comment in do_slb_bolted which claims to permit
a full 32-bits of ESID - it's wrong.  The code doesn't mask the ESID
down to 13 bits as get_kernel_vsid() does, but it probably should - an
overlarge ESID will cause collisions with VSIDs from entirely
different address places, which would be a Bad Thing.

Actually, you should be able to allow ESIDs of up to 21 bits there (36
bit VSID - 15 bits of "context").  But you will need to make sure
get_kernel_vsid(), or whatever you're using to calculate the VAs for
the hash HPTEs is updated to match - at the moment I think it will
mask down to 13 bits.  I'm not sure if that will get you sufficiently
close to 0xc0... for your purposes.

> > On Mon, Sep 27, 2004 at 02:47:15PM -0500, Igor Grobman wrote:
> > > I would like to be able to remap memory into 0xb space with 2.4.21
> kernel.
> > > I have code that works fine on power3 box, but fails miserably on my
> dual
> > > power4+ box.  I am using btmalloc() from pmc.c with a modified range.
> > > Normal btmalloc() allocation works fine, but if I change the range to
> > > start with, e.g. 0xb00001FFFFF00000 (instead of 0xb00..), the kernel
> > > crashes when accessing the allocated page.
> > >
> > > For those of you unfamiliar with btmalloc() code, it finds a physical
> page
> > > using get_free_pages(), subtracts PAGE_OFFSET (i.e. 0xc00...) to form
> a
> > > physical address, then inserts the page into linux page tables, using
> the
> > > VSID calculated with get_kernel_vsid() and the physical address
> > > calculated above.  It also inserts an HPTE into the hardware page
> table.
> > > It doesn't do anything with regards to allocating segments.  I
> understand
> > > the segments should be faulted in.  I looked at the code in head.S,
> and it
> > > appears that do_stab_bolted should be doing this.  Yet, I am missing
> someth$
> > > because the btmalloc() code does not in fact work for pages in the
> range I
> > > specified above.
> > >
> > > I am running this on 7029-6E3 (p615?) machine with two power4+
> processors.
> > > I am using the kernel from Suse's 2.4.21-215 source package.
> > >
> > > Any ideas and attempts to un-confuse me are welcome.
> 

-- 
David Gibson			| For every complex problem there is a
david AT gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.
http://www.ozlabs.org/people/dgibson