[Cbe-oss-dev] Only 86MB of RAM available running Ubuntu 8.04 powerpc64-smp kernel on PS3

[Geoff Levand]

Michael Ellerman wrote:
> On Mon, 2008-05-05 at 01:50 +0100, Dan Munckton wrote:
>> Hi all
>> 
>> Quick intro - I've recently taken it upon myself to do something about
>> getting Ubuntu Hardy running on the PS3. Unfortunately for most of the
>> Hardy development cycle there was no one working on the PS3 port
>> consequently the kernel is a bit broken.
> 
> Awesome, we'd all love it to work, we're just all a bit busy doing
> "real" work :/
> 
>> Although I have the motivation I am quite new to kernel work so please
>> be gentle with me.
>> 
>> Basically I think most of the issues we have can be solved by using the
>> correct configuration. Building and running a kernel using just the
>> options from ps3_defconfig makes all issues go away. Where I'm going to
>> need help is in identifying which options need to be changed and
>> hopefully understanding why.
>> 
>> The first issue on my list is this: only 86MB of RAM appear to be
>> available. We're tracking this issue at [0] where I've uploaded all
>> configs and logs.
>> 
>> There is a backtrace in the dmesg output from the failing kernel which I
>> think maybe related (attached). It starts with ...
>> 
>> swapper: page allocation failure. order:12, mode:0x80d0
> 
> I think that's caused by the VMEMMAP stuff. The PS3 boots with some
> memory (128MB?) and then the rest is hot-plugged later on. The VMEMMAP
> stuff has introduced an issue (bug), where the rest of memory can't be
> hotplugged because the VMEMMAP can't be allocated. (correct me if I'm
> wrong Ben or Geoff).


Yes, all firmware versions to date give 128 MiB of boot memory.

The VMEMMAP authors didn't consider systems that don't have much
memory when they did there design, so it doesn't really work well
on PS3.  With your config it needs a 16MiB page of boot memory to
add the extra hot plug memory.  The above message shows that this
alloc failed, so the hot plug mem could not be added.  The system
then continues to boot with just the 128 MiB of boot mem.

Looking at the .config here:

  http://launchpadlibrarian.net/13914707/config-2.6.24-16-powerpc64-smp

The trouble is that CONFIG_SPARSEMEM_VMEMMAP is set, and the kernel
image is very big do to all the other options set.

You have two choices for your Hardy support.  The easier way, and
better for PS3 users, is to make a ps3 specific kernel, the harder
way is to tune the multi-platform kernel so that it boots on the PS3.

Either way, change to these:

  CONFIG_SPARSEMEM_VMEMMAP=n
  CONFIG_FB_PS3_DEFAULT_SIZE_M=9
  CONFIG_PS3_USE_LPAR_ADDR=n

-Geoff