[RFC PATCH v2 0/2] Randomization of address chosen by mmap.

[Rich Felker]

On Fri, Mar 23, 2018 at 12:29:52PM -0700, Matthew Wilcox wrote:
> On Fri, Mar 23, 2018 at 03:16:21PM -0400, Rich Felker wrote:
> > > Huh, I thought libc was aware of this.  Also, I'd expect a libc-based
> > > implementation to restrict itself to, eg, only loading libraries in
> > > the bottom 1GB to avoid applications who want to map huge things from
> > > running out of unfragmented address space.
> > 
> > That seems like a rather arbitrary expectation and I'm not sure why
> > you'd expect it to result in less fragmentation rather than more. For
> > example if it started from 1GB and worked down, you'd immediately
> > reduce the contiguous free space from ~3GB to ~2GB, and if it started
> > from the bottom and worked up, brk would immediately become
> > unavailable, increasing mmap pressure elsewhere.
> 
> By *not* limiting yourself to the bottom 1GB, you'll almost immediately
> fragment the address space even worse.  Just looking at 'ls' as a
> hopefully-good example of a typical app, it maps:
> 
> 	linux-vdso.so.1 (0x00007ffef5eef000)
> 	libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007fb3657f5000)
> 	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb36543b000)
> 	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fb3651c9000)
> 	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fb364fc5000)
> 	/lib64/ld-linux-x86-64.so.2 (0x00007fb365c3f000)
> 	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fb364da7000)
> 
> The VDSO wouldn't move, but look at the distribution of mapping 6 things
> into a 3GB address space in random locations.  What are the odds you have
> a contiguous 1GB chunk of address space?  If you restrict yourself to the
> bottom 1GB before running out of room and falling back to a sequential
> allocation, you'll prevent a lot of fragmentation.

Oh, you're talking about "with random locations" case. Randomizing
each map just hopelessly fragments things no matter what you do on
32-bit. If you reduce the space over which you randomize to the point
where it's not fragmenting/killing your available vm space, there are
so few degrees of freedom left that it's trivial to brute-force. Maybe
"libs randomized in low 1GB, everything else near-sequential in high
addresses" works half decently, but I have a hard time believing you
can get any ASLR that's significantly better than snake oil in a
32-bit address space, and you certainly do pay a high price in total
available vm space.

Rich