Merging multiple erofs file systems on the same block device

[Daan De Meyer]

> On 2023/5/1 22:09, Daan De Meyer wrote:
> > Hi,
> >
> > I've been looking into erofs as an initramfs replacement by using
> > root=/dev/ram0 to tell the kernel to load the initramfs as a ramdisk.
>
> Sorry, I'm on vacation now.
>
> May I ask what's your detailed use cases?  Sure, you could use
> /dev/ram0 as a replacement, but currently it still takes double
> memory compared with initramfs since ramdisk doesn't support FSDAX
> for now (by enabling FSDAX, it won't take double memory at all.)

I'm experimenting with larger initramfses and running into memory
bottlenecks since the entire compressed cpio has to be decompressed
into memory. I was hoping to use erofs as a replacement that could stay
compressed, where only the files that are actually accessed are
decompressed at runtime.

> Actually I think ramdisk FSDAX is useful and I might sync up this on
> the following LSF/MM/BPF 2023.
>
> > However, by using a ramdisk instead of the usual compressed cpio, I
> > would lose the feature where the kernel merges multiple individual
> > cpios together into a single tmpfs filesystem. Looking at the
> > documentation for erofs, I noticed that erofs already seems to support
> > merging multiple erofs filesystems on separate block devices using the
> > device= cmdline option. Would it be possible to extend this so that
> Here `device=` is actually used to refer to seperate blobs with the
> merged metadata.  For example, you could have
>
>    device=/dev/ram1 original tar1
>    device=/dev/ram2 original tar2
>    /dev/ram0        merged metadata for tar1 + tar2.
>
> which means, if you'd like to merge multiple EROFS filesystems, you
> might need another step to build a merged metadata in advance in order
> to merges multiple individual tarballs together, which could be built
> when applying images or booting (by using a special bootloader with
> such functionality.)

Ahh, I misunderstood the device= option then.

> EROFS doesn't support stacking multiple fses runtimely since it seems
> a duplicated feature of overlayfs (you could consider using overlayfs
> honestly.)

I would love to use overlayfs, but there's no way to specify to the kernel that
the initrd should be set up as an overlayfs of a set of ram disks. It would be
interesting if I could put multiple filesystems in the initrd and the
kernel would
notice and automatically set up an overlayfs of them.

> > multiple erofs filesystems that follow each other on the same block
> > device can also be merged? This would allow me to pass multiple erofs
> > filesystems to the kernel via initrd=, which would get concatenated
> > together into a single buffer, which the kernel would write to a
> > ramdisk (using root=/dev/ram0) which the kernel would then have erofs
> > mount to /dev/root. erofs would notice that there's multiple erofs
> > filesystems on the ramdisk and overlay them together (perhaps only if
> > a cmdline option is enabled).
>
> Recently, A merged EROFS filesystem with a single block device can
> be passed but you might still need a step to build a merged fs tree
> before mounting.  Such feature is used to build a metadata and
> reuse original tar blobs.
>
> >
> > Does this make sense at all?
>
> Hopefully it helps.