using different format for hugetlbfs

[David Gibson]

On Tue, Dec 08, 2009 at 09:44:55AM -0600, Kumar Gala wrote:
> 
> On Dec 7, 2009, at 8:28 PM, David Gibson wrote:
> 
> >On Mon, Dec 07, 2009 at 12:04:37PM +1100, Benjamin Herrenschmidt
> >wrote:
> >>
> >>>
> >>>Even than, does that preclude the format I suggested?  I'm assuming
> >>>that pgd_t/pud_t/pmd_t are always a double word so the low order 4-
> >>>bits should be 0 (on 64-bit),
> >
> >Double word alignment only gives us 3 low bits.
> >
> >>so using the lsb as the flag between
> >>>hugetlb and normal pointer should still work.
> >>
> >>Might do, depends if David has enough bits ...  David ?
> >
> >Well, the flag can go at the bottom, but that will mean grabbing more
> >bits at the bottom.  At the moment to cover all the page table sizes
> >that are wanted on the various setups we have, I need 5 bits, this
> >would push it to 6.  At present, I just force up the minimum alignment
> >of any page directory (even if it's natural alignment is smaller) so
> >as to make sure I have those bits.  That's pretty easy to adjust, but
> >pushing it up too high will start wasting memory, of course.
> >
> >If we move to a variable sized encoding, as Ben and I have discussed
> >on a couple of occasions, I think we could do this though.
> 
> I don't understand.  It seems like only the flag bit of normal
> pointer vs hugetlb is the only thing that we need to distinguish.
> Once we've done that all the other bits are free to use as we see
> fit.  So the less significant bit can be used for that purpose and
> the size encoding, etc we are free to do what we want with.

Well, yes, but the huge page directory pointers are still pointers, so
this is one extra bit at the bottom which counts against our minimum
alignment for those pointers.  There's no natural lower bound on the
size of the hugepte directories, and with existing setups they already
go as low as 4 entries, which we already pad out to meet our minimum
alignment.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson