[PATCH v4 4/4] powerpc: Book3S 64-bit "heavyweight" KASAN support
Christophe Leroy
christophe.leroy at c-s.fr
Thu Dec 19 21:07:57 AEDT 2019
On 12/19/2019 10:05 AM, Christophe Leroy wrote:
>
>
> Le 19/12/2019 à 10:50, Daniel Axtens a écrit :
>> Christophe Leroy <christophe.leroy at c-s.fr> writes:
>>
>>> On 12/19/2019 12:36 AM, Daniel Axtens wrote:
>>>> KASAN support on Book3S is a bit tricky to get right:
>>>>
>>>> - It would be good to support inline instrumentation so as to be
>>>> able to
>>>> catch stack issues that cannot be caught with outline mode.
>>>>
>>>> - Inline instrumentation requires a fixed offset.
>>>>
>>>> - Book3S runs code in real mode after booting. Most notably a lot
>>>> of KVM
>>>> runs in real mode, and it would be good to be able to
>>>> instrument it.
>>>>
>>>> - Because code runs in real mode after boot, the offset has to
>>>> point to
>>>> valid memory both in and out of real mode.
>>>>
>>>> [ppc64 mm note: The kernel installs a linear mapping at effective
>>>> address c000... onward. This is a one-to-one mapping with
>>>> physical
>>>> memory from 0000... onward. Because of how memory accesses
>>>> work on
>>>> powerpc 64-bit Book3S, a kernel pointer in the linear map
>>>> accesses the
>>>> same memory both with translations on (accessing as an 'effective
>>>> address'), and with translations off (accessing as a 'real
>>>> address'). This works in both guests and the hypervisor. For more
>>>> details, see s5.7 of Book III of version 3 of the ISA, in
>>>> particular
>>>> the Storage Control Overview, s5.7.3, and s5.7.5 - noting that
>>>> this
>>>> KASAN implementation currently only supports Radix.]
>>>>
>>>> One approach is just to give up on inline instrumentation. This way all
>>>> checks can be delayed until after everything set is up correctly,
>>>> and the
>>>> address-to-shadow calculations can be overridden. However, the
>>>> features and
>>>> speed boost provided by inline instrumentation are worth trying to do
>>>> better.
>>>>
>>>> If _at compile time_ it is known how much contiguous physical memory a
>>>> system has, the top 1/8th of the first block of physical memory can
>>>> be set
>>>> aside for the shadow. This is a big hammer and comes with 3 big
>>>> consequences:
>>>>
>>>> - there's no nice way to handle physically discontiguous memory,
>>>> so only
>>>> the first physical memory block can be used.
>>>>
>>>> - kernels will simply fail to boot on machines with less memory than
>>>> specified when compiling.
>>>>
>>>> - kernels running on machines with more memory than specified when
>>>> compiling will simply ignore the extra memory.
>>>>
>>>> Implement and document KASAN this way. The current implementation is
>>>> Radix
>>>> only.
>>>>
>>>> Despite the limitations, it can still find bugs,
>>>> e.g. http://patchwork.ozlabs.org/patch/1103775/
>>>>
>>>> At the moment, this physical memory limit must be set _even for outline
>>>> mode_. This may be changed in a later series - a different
>>>> implementation
>>>> could be added for outline mode that dynamically allocates shadow at a
>>>> fixed offset. For example, see
>>>> https://patchwork.ozlabs.org/patch/795211/
>>>>
>>>> Suggested-by: Michael Ellerman <mpe at ellerman.id.au>
>>>> Cc: Balbir Singh <bsingharora at gmail.com> # ppc64 out-of-line radix
>>>> version
>>>> Cc: Christophe Leroy <christophe.leroy at c-s.fr> # ppc32 version
>>>> Signed-off-by: Daniel Axtens <dja at axtens.net>
>>>>
>>>> ---
>>>> Changes since v3:
>>>> - Address further feedback from Christophe.
>>>> - Drop changes to stack walking, it looks like the issue I
>>>> observed is
>>>> related to that particular stack, not stack-walking generally.
>>>>
>>>> Changes since v2:
>>>>
>>>> - Address feedback from Christophe around cleanups and docs.
>>>> - Address feedback from Balbir: at this point I don't have a good
>>>> solution
>>>> for the issues you identify around the limitations of the
>>>> inline implementation
>>>> but I think that it's worth trying to get the stack
>>>> instrumentation support.
>>>> I'm happy to have an alternative and more flexible outline mode
>>>> - I had
>>>> envisoned this would be called 'lightweight' mode as it imposes
>>>> fewer restrictions.
>>>> I've linked to your implementation. I think it's best to add it
>>>> in a follow-up series.
>>>> - Made the default PHYS_MEM_SIZE_FOR_KASAN value 1024MB. I think
>>>> most people have
>>>> guests with at least that much memory in the Radix 64s case so
>>>> it's a much
>>>> saner default - it means that if you just turn on KASAN without
>>>> reading the
>>>> docs you're much more likely to have a bootable kernel, which
>>>> you will never
>>>> have if the value is set to zero! I'm happy to bikeshed the
>>>> value if we want.
>>>>
>>>> Changes since v1:
>>>> - Landed kasan vmalloc support upstream
>>>> - Lots of feedback from Christophe.
>>>>
>>>> Changes since the rfc:
>>>>
>>>> - Boots real and virtual hardware, kvm works.
>>>>
>>>> - disabled reporting when we're checking the stack for exception
>>>> frames. The behaviour isn't wrong, just incompatible with KASAN.
>>>>
>>>> - Documentation!
>>>>
>>>> - Dropped old module stuff in favour of KASAN_VMALLOC.
>>>>
>>>> The bugs with ftrace and kuap were due to kernel bloat pushing
>>>> prom_init calls to be done via the plt. Because we did not have
>>>> a relocatable kernel, and they are done very early, this caused
>>>> everything to explode. Compile with CONFIG_RELOCATABLE!
>>>> ---
>>>> Documentation/dev-tools/kasan.rst | 8 +-
>>>> Documentation/powerpc/kasan.txt | 112
>>>> ++++++++++++++++++-
>>>> arch/powerpc/Kconfig | 2 +
>>>> arch/powerpc/Kconfig.debug | 21 ++++
>>>> arch/powerpc/Makefile | 11 ++
>>>> arch/powerpc/include/asm/book3s/64/hash.h | 4 +
>>>> arch/powerpc/include/asm/book3s/64/pgtable.h | 7 ++
>>>> arch/powerpc/include/asm/book3s/64/radix.h | 5 +
>>>> arch/powerpc/include/asm/kasan.h | 21 +++-
>>>> arch/powerpc/kernel/prom.c | 61 +++++++++-
>>>> arch/powerpc/mm/kasan/Makefile | 1 +
>>>> arch/powerpc/mm/kasan/init_book3s_64.c | 70 ++++++++++++
>>>> arch/powerpc/platforms/Kconfig.cputype | 1 +
>>>> 13 files changed, 316 insertions(+), 8 deletions(-)
>>>> create mode 100644 arch/powerpc/mm/kasan/init_book3s_64.c
>>>>
>>>> diff --git a/arch/powerpc/include/asm/kasan.h
>>>> b/arch/powerpc/include/asm/kasan.h
>>>> index 296e51c2f066..f18268cbdc33 100644
>>>> --- a/arch/powerpc/include/asm/kasan.h
>>>> +++ b/arch/powerpc/include/asm/kasan.h
>>>> @@ -2,6 +2,9 @@
>>>> #ifndef __ASM_KASAN_H
>>>> #define __ASM_KASAN_H
>>>> +#include <asm/page.h>
>>>> +#include <asm/pgtable.h>
>>>
>>> What do you need asm/pgtable.h for ?
>>>
>>> Build failure due to circular inclusion of asm/pgtable.h:
>>
>> I see there's a lot of ppc32 stuff, I clearly need to bite the bullet
>> and get a ppc32 toolchain so I can squash these without chewing up any
>> more of your time. I'll sort that out and send a new spin.
>>
>
> I'm using a powerpc64 toolchain to build both ppc32 and ppc64 kernels
> (from https://mirrors.edge.kernel.org/pub/tools/crosstool/ )
>
>
> Another thing, did you test PTDUMP stuff with KASAN ? It looks like
> KASAN address markers don't depend on PPC32, but are only initialised by
> populate_markers() for PPC32.
>
> Regarding kasan.h, I think we should be able to end up with something
> where the definition of KASAN_SHADOW_OFFSET should only depend on the
> existence of CONFIG_KASAN_SHADOW_OFFSET, and where only
> KASAN_SHADOW_SIZE should depend on the target (ie PPC32 or BOOK3S64)
> Everything else should be common. KASAN_END should be START+SIZE.
>
> It looks like what you have called KASAN_SHADOW_SIZE is not similar to
> what is called KASAN_SHADOW_SIZE for PPC32, as yours only covers the
> SHADOW_SIZE for linear mem while PPC32 one covers the full space.
>
More or less something like that:
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __ASM_KASAN_H
#define __ASM_KASAN_H
#include <asm/page.h>
#ifdef CONFIG_KASAN
#define _GLOBAL_KASAN(fn) _GLOBAL(__##fn)
#define _GLOBAL_TOC_KASAN(fn) _GLOBAL_TOC(__##fn)
#define EXPORT_SYMBOL_KASAN(fn) EXPORT_SYMBOL(__##fn)
#else
#define _GLOBAL_KASAN(fn) _GLOBAL(fn)
#define _GLOBAL_TOC_KASAN(fn) _GLOBAL_TOC(fn)
#define EXPORT_SYMBOL_KASAN(fn)
#endif
#ifndef __ASSEMBLY__
#define KASAN_SHADOW_SCALE_SHIFT 3
#define KASAN_SHADOW_START (KASAN_SHADOW_OFFSET + \
(PAGE_OFFSET >> KASAN_SHADOW_SCALE_SHIFT))
#define KASAN_SHADOW_END (KASAN_SHADOW_START + KASAN_SHADOW_SIZE)
#ifdef CONFIG_KASAN_SHADOW_OFFSET
#define KASAN_SHADOW_OFFSET ASM_CONST(CONFIG_KASAN_SHADOW_OFFSET)
#endif
#ifdef CONFIG_PPC32
#define KASAN_SHADOW_SIZE ((-PAGE_OFFSET) >> KASAN_SHADOW_SCALE_SHIFT)
#endif
#ifdef CONFIG_PPC_BOOK3S_64
#define KASAN_SHADOW_SIZE (ASM_CONST(CONFIG_PHYS_MEM_SIZE_FOR_KASAN) *
SZ_1G) >> \
KASAN_SHADOW_SCALE_SHIFT)
#endif
#ifdef CONFIG_KASAN
void kasan_early_init(void);
void kasan_mmu_init(void);
void kasan_init(void);
#else
static inline void kasan_init(void) { }
static inline void kasan_mmu_init(void) { }
#endif
#endif /* __ASSEMBLY */
#endif
Christophe
More information about the Linuxppc-dev
mailing list