ppc32: Weird process scheduling behaviour with 2.6.24-rc

[Srivatsa Vaddagiri]

On Fri, Jan 25, 2008 at 09:50:00AM +0100, Peter Zijlstra wrote:
> 
> On Fri, 2008-01-25 at 18:25 +1100, Benjamin Herrenschmidt wrote:
> > On Fri, 2008-01-25 at 18:03 +1100, Benjamin Herrenschmidt wrote:
> > > On Fri, 2008-01-25 at 17:54 +1100, Benjamin Herrenschmidt wrote:
> > > > 
> > > > Here, I do the test of running 4 times the repro-case provided by Michel
> > > > with nice 19 and a dd eating CPU with nice 0.
> > > > 
> > > > Without this option, I get the dd at 100% and the nice 19 shells down
> > > > below it with whatever is left of the CPUs.
> > > > 
> > > > With this option, dd gets about 50% of one CPU and the niced processes
> > > > still get most of the time.

Ben,
	I presume you had CONFIG_FAIR_USER_SCHED turned on too? Also were the
dd process and the niced processes running under different user ids? If
so, that is expected behavior, that we divide CPU equally among
users first and then among the processes within each user.

> > > FYI. This is a 4 way G5 (ppc64)
> > 
> > I also tested responsiveness of X running with or without that option
> > and with niced CPU eaters in the background (still 4 of them, one per
> > CPU), and I can confirm Michel observations, it gets very sluggish
> > (maybe not -as- bad as his but still pretty annoying) with the fair
> > group scheduler enabled.

When CONFIG_FAIR_GROUP_SCHED (and CONFIG_FAIR_USER_SCHED) is not
enabled, X will be given higher priority for running on CPU when compared to 
other niced tasks. When the above options are turned on, X (running
under root uid) would be given lesser priority to run when compared to other
niced tasks running user different uids. Hence I expect some drop in
interactivity-experience with FAIR_GROUP_SCHED on.

Can you pls let me know if any of these makes a difference:

1. Run niced tasks as root. This would bring X and niced tasks in the
same "scheduler group" domain, which would give X much more CPU power
when compared to niced tasks.

2. Keep the niced tasks running under a non-root uid, but increase root users 
   cpu share.
	# echo 8192 > /sys/kernel/uids/0/cpu_share

   This should bump up root user's priority for running on CPU and also 
   give a better desktop experience.

> > Here, X is running with nice=0
> 
> Curious, sounds like an issue with the group load balancer, vatsa, any
> ideas?

The group scheduler's SMP-load balance in 2.6.24 is not the best it
could be. sched-devel has a better load balancer, which I am presuming
will go into 2.6.25 soon.

In this case, I suspect that's not the issue.  If X and the niced processes are 
running under different uids, this (niced processes getting more cpu power) is 
on expected lines. Will wait for Ben to confirm this. 

-- 
Regards,
vatsa