ppc32: Weird process scheduling behaviour with 2.6.24-rc

[Benjamin Herrenschmidt]

On Sat, 2008-01-26 at 09:37 +0530, Srivatsa Vaddagiri wrote:
> 
> Ben,
>         I presume you had CONFIG_FAIR_USER_SCHED turned on too?

Yes. It seems to be automatically turned on whenever FAIR_GROUP is
turned on. Considering how bad the behaviour is for a standard desktop
configuration, I'd be tempted to say to change it to default n.

> Also were the
> dd process and the niced processes running under different user ids? If
> so, that is expected behavior, that we divide CPU equally among
> users first and then among the processes within each user.

They were different users and that behaviour seems to be a very stupid
default behaviour for a desktop machine. Take this situation:

 - X running as root
 - User apps running as "user"
 - Background crap (indexing daemons etc...) running as their own user
or nobody

Unless you can get some kind of grouping based on user sessions
including suid binaries, X etc... I think this shouldn't default y in
Kconfig.

Not that it seems that Michel reported far worse behaviour than what I
saw, including pretty hickup'ish X behaviour even without the fair group
scheduler compared to 2.6.23. It might be because he's running X niced
to -1 (I leave X at 0 and let the scheduler deal with it in general)
though.

> When CONFIG_FAIR_GROUP_SCHED (and CONFIG_FAIR_USER_SCHED) is not
> enabled, X will be given higher priority for running on CPU when compared to 
> other niced tasks. When the above options are turned on, X (running
> under root uid) would be given lesser priority to run when compared to other
> niced tasks running user different uids. Hence I expect some drop in
> interactivity-experience with FAIR_GROUP_SCHED on.
> 
> Can you pls let me know if any of these makes a difference:
> 
> 1. Run niced tasks as root. This would bring X and niced tasks in the
> same "scheduler group" domain, which would give X much more CPU power
> when compared to niced tasks.

I'll dbl check. My tests where indeed done with different users.

> 2. Keep the niced tasks running under a non-root uid, but increase root users 
>    cpu share.
>         # echo 8192 > /sys/kernel/uids/0/cpu_share
> 
>    This should bump up root user's priority for running on CPU and also 
>    give a better desktop experience.

Allright, that's something that might need to be set by default by the
kernel ... as it will take some time to have knowledge of those knobs to
percolate to distros. Too bad you can't do the opposite by default for
"nobody" as there's no standard uid for it.

> The group scheduler's SMP-load balance in 2.6.24 is not the best it
> could be. sched-devel has a better load balancer, which I am presuming
> will go into 2.6.25 soon.
> 
> In this case, I suspect that's not the issue.  If X and the niced processes are 
> running under different uids, this (niced processes getting more cpu power) is 
> on expected lines. Will wait for Ben to confirm this. 

I would suggest turning the fair group scheduler to default n in stable
for now.

Cheers,
Ben.