Debian SID kernel doesn't boot on PowerBook 3400c

Fri Aug 6 16:09:25 AEST 2021

+nicholas piggin for the C interrupt stuff

Le 06/08/2021 à 03:06, Finn Thain a écrit :
> (Christophe, you've seen some of this before, however there are new
> results added at the end. I've Cc'd the mailing lists this time.)
> 
> On Wed, 4 Aug 2021, Stan Johnson wrote:
> 
>> On 8/4/21 8:41 PM, Finn Thain wrote:
>>
>>>
>>> $ curl https://lore.kernel.org/lkml/9b64dde3-6ebd-b446-41d9-61e8cb0d8c39@csgroup.eu/raw
>>> ../message.mbox
>> ok
>>
>> $ sha1 ../message.mbox
>> SHA1 (../message.mbox) = 436ce0adf893c46c84c54607f73c838897caeeea
>>
>>>
>>> On Wed, 4 Aug 2021, Christophe Leroy wrote:
>>>
>>>> Can you check if they happen at commit c16728835
>>>>
>>
>> $ git checkout c16728835eec
>> Checking out files: 100% (20728/20728), done.
>> Note: checking out 'c16728835eec'.
>>
>> You are in 'detached HEAD' state. You can look around, make experimental
>> changes and commit them, and you can discard any commits you make in this
>> state without impacting any branches by performing another checkout.
>>
>> If you want to create a new branch to retain commits you create, you may
>> do so (now or later) by using -b with the checkout command again. Example:
>>
>>    git checkout -b <new-branch-name>
>>
>> HEAD is now at c16728835eec powerpc/32: Manage KUAP in C
>> $ git am ../message.mbox
>> warning: Patch sent with format=flowed; space at the end of lines might be lost.
>> Applying: powerpc/32: Dismantle EXC_XFER_STD/LITE/TEMPLATE
>> $ cp ../dot-config-powermac-5.13 .config
>> $ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- -j4 clean olddefconfig vmlinux
>> $ strings vmlinux | fgrep 'Linux version'
>> Linux version 5.12.0-rc3-pmac-00078-geb51c431b81 (johnson at ThinkPad) (powerpc-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #1 SMP Wed Aug 4 21:50:47 MDT 2021
>>
>> 1) PB 3400c
>> Hangs at boot (Mac OS screen), no serial console output
>>
>> 2) Wallstreet
>> X fails, errors ("Kernel attempted to write user page", "BUG: Unable to
>> handle kernel instruction fetch"), see Wallstreet_console-1.txt.
>>
> 
> The log shows that the error "Kernel attempted to write user page
> (b3399774) - exploit attempt?" happens after commit c16728835eec
> ("powerpc/32: Manage KUAP in C").

I think I found a possible cause for this. After the above patch, locking KUAP on interrupt is done 
in interrupt_enter_prepare(). But in case of NMI interrupt, that function is not called. That means 
that when leaving interrupt through interrupt_exit_kernel_prepare(), the supposedly saved previous 
KUAP status is garbage.

An easy way to fix that is to add missing stuff in interrupt_nmi_enter_prepare(), I'll do that at 
least for testing, but at the end it is not so easy, because of booke32 and 40x.

The problem on booke32 and 40x is that the "critical interrupts" exit goes through interrupt_return 
when they happened in user mode and bypass interrupt_return when they happened in kernel mode. So it 
is not easy to manage.

> 
>>>>
>>>> Can you check if they DO NOT happen at preceding commit c16728835~
>>>>
>>
>> $ git checkout c16728835~
>> Previous HEAD position was c16728835eec powerpc/32: Manage KUAP in C
>> HEAD is now at 0b45359aa2df powerpc/8xx: Create C version of kuap save/restore/check helpers
>> $ git am ../message.mbox
>> warning: Patch sent with format=flowed; space at the end of lines might be lost.
>> Applying: powerpc/32: Dismantle EXC_XFER_STD/LITE/TEMPLATE
>> $ cp ../dot-config-powermac-5.13 .config
>> $ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- -j4 clean olddefconfig vmlinux
>>
>> Linux version 5.12.0-rc3-pmac-00077-gc9f6e8dd045
>>
>> 3) PB 3400c
>> Hangs at boot (Mac OS screen)
>>
>> 4) Wallstreet
>> X fails, errors in console log (different than test 2), see
>> Wallstreet_console-2.txt.
>>
> 
> This log shows that the errors "xfce4-session[1775]: bus error (7)" and
> "kernel BUG at arch/powerpc/kernel/interrupt.c:49!" happen prior to commit
> c16728835eec ("powerpc/32: Manage KUAP in C").

As mentionned by Nic, this is due to r11 being cloberred. For the time being the only r11 clobber 
identified is the one I have provided a fix for. I'm wondering whether it was applied for all 
further tests or not.

> 
>>
>> $ git checkout 0b45359aa2df
>> ...
>> HEAD is now at 0b45359aa2df powerpc/8xx: Create C version of kuap save/restore/check helpers
>> $ git am ../message.mbox
>> warning: Patch sent with format=flowed; space at the end of lines might be lost.
>> Applying: powerpc/32: Dismantle EXC_XFER_STD/LITE/TEMPLATE
>> $ cp ../dot-config-powermac-5.13 .config
>> $ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- -j4 clean olddefconfig vmlinux
>>
>> Linux version 5.12.0-rc3-pmac-00077-ge06b29ce146
>>
>> 5) PB 3400c
>> Hangs at boot (Mac OS screen)
>>
>> 6) Wallstreet
>> X failed (X login succeeded, but setting up desktop failed), errors in
>> console log, see Wallstreet_console-3.txt.
>>
> 
> (No need for those two tests: it's exactly the same code and almost the
> same failure modes: "kernel BUG at arch/powerpc/kernel/interrupt.c:50".)
> 
> On Thu, 5 Aug 2021, Stan Johnson wrote:
> 
>> On 8/5/21 12:47 AM, Finn Thain wrote:
>>
>>> On Wed, 4 Aug 2021, Christophe Leroy wrote:
>>>
>>>> Could you test without CONFIG_PPC_KUAP
>> ...
>>
>> $ git checkout c16728835eec
>> ...
>> HEAD is now at c16728835eec powerpc/32: Manage KUAP in C
>> $ git am ../message.mbox
>> warning: Patch sent with format=flowed; space at the end of lines might be lost.
>> Applying: powerpc/32: Dismantle EXC_XFER_STD/LITE/TEMPLATE
>> $ cp ../dot-config-powermac-5.13 .config
>> $ scripts/config -d CONFIG_PPC_KUAP
>> $ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- -j4 clean olddefconfig vmlinux
>> $ grep CONFIG_PPC_KUAP .config
>> # CONFIG_PPC_KUAP is not set
>>
>> Linux version 5.12.0-rc3-pmac-00078-g5cac2bc3752
>>
>> 7) PB 3400c
>> Hangs at boot (Mac OS screen)
>>
>> 8) Wallstreet
>> Everything works, no errors (see Wallstreet_console-4.txt).
>>
> 
> That would seem to implicate CONFIG_PPC_KUAP itself. (Note that all builds
> up until this one have CONFIG_PPC_KUAP=y.)

Yes I believe so, see at the begining of this mail.

> 
>>
>>>
>>>> Could you test with CONFIG_PPC_KUAP and CONFIG_PPC_KUAP_DEBUG
>> ...
>>
>> $scripts/config -e CONFIG_PPC_KUAP
>> $ scripts/config -e CONFIG_PPC_KUAP_DEBUG
>> $ make ARCH=powerpc CROSS_COMPILE=powerpc-linux-gnu- -j4 clean olddefconfig vmlinux
>> $ grep CONFIG_PPC_KUAP .config
>> CONFIG_PPC_KUAP=y
>> CONFIG_PPC_KUAP_DEBUG=y
>>
>> Linux version 5.12.0-rc3-pmac-00078-g5cac2bc3752
>>
>> 9) PB 3400c
>> Hangs at boot (Mac OS screen)
>>
>> 10) Wallstreet
>> X failed at first login, worked at second login, one error in console
>> log ("BUG: Unable to handle kernel instruction fetch"), see
>> Wallstreet_console-5.txt.
>>
> 
> One might expect to see "Kernel attempted to write user page (b3399774) -
> exploit attempt?" again here (see c16728835eec build above) but instead
> this log says "Oops: Kernel access of bad area, sig: 11".

Maybe the test should be done a second time. As r11 is garbage it may or may not be a user address. 
If it is a user address the we get "Kernel attempted to write user page". If it is a random kernel 
address, we likely get "Kernel access of bad area" instead.

> 
> BTW, this procedure could be made simpler and easier if I pushed git
> branches to a public repo for Stan to build, which included Christophe's
> fix plus hard-wired Kconfig changes. That way, the .config file could be
> held constant and the commit hash in the serial console log would be more
> meaningful.
> 

I like the idea, I think I'm going to provide testing fixes through a git repo, that will for sure 
make things easier.

Thanks
Christophe