Worse performance than SquashFS for small filesystems

[Paul Spooren]

Hi,

Thank you for the quick response!

> On 31. May 2022, at 02:00, Gao Xiang <xiang at kernel.org> wrote:
> 
> On Mon, May 30, 2022 at 08:41:34PM +0200, Paul Spooren wrote:
>> Hi,
>> 
>> I’m an OpenWrt developer and we work on an operation system for routers and overall network facing embedded devices. Since storage is often limited on our targeted devices we use SquashFS to compress everything, total image sizes are less than 10 MegaBytes.
>> 
>> Reading about erofs I’m very keen to adopt it for our case, however giving it a first try the storage performance seems to be significantly _worse_ than the default SquashFS implementation. Since you ran benchmarks in the past, could you give me advise if I’m missing anything?
> 
> What the meaning of `performance'? I don't think the size of images
> mean runtime performance anyway.

Performance as in looking at how well it performs on minimizing the space requirement.

> 
>> 
>> For the test I used erofs-utils as of a134ce7c39a5427343029e97a62665157bef9bc3 (2022-05-17) and compressed the x86/64 root filesystem of a standard OpenWrt image[1]. I’m seeing a difference of one MegaByte which is about 30% worse in this context.
>> 
>> My test:
>> 
>> $ ./staging_dir/host/bin/mkfs.erofs -zlzma erofs.root ./build_dir/target-x86_64_musl/root-x86
> 
> Have you try with
> 
> -zlzma -C 65536 -Eztailpacking
> 
> by default, EROFS uses 4k compression rather than Squashfs 128k.

That indeed improves the total size, I’m now down to 3.6MB (vs 3.3MB for squashfs).

> 
>> mkfs.erofs 1.4
>> <W> erofs: EXPERIMENTAL MicroLZMA feature in use. Use at your own risk!
>> <W> erofs: Note that it may take more time since the compressor is still single-threaded for now.
>> Build completed.
>> 
>> $ mksquashfs -comp xz ./build_dir/target-x86_64_musl/root-x86 squashfs.root
>> 
>> $ ls -lh *.root
>> -rw-r--r-- 1 ubuntu ubuntu 4.3M May 30 20:27 erofs.root
>> -rw-r--r-- 1 ubuntu ubuntu 3.3M May 30 20:28 squashfs.root
> 
> Even EROFS now supports big pcluster and ztailpacking features,
> Squashfs supports a feature called fragments which compresses
> several tails of files in a fragment. It's obviously beneficial
> to the final size of images but it can read unrelated data of
> other files even such files are very small.
> 
> You can try a big file with EROFS and Squashfs, and you can
> see the difference.
> 
> Btw, MicroLZMA is still an experimental feature for now, due to 
> three reasons:
> - XZ utils hasn't release a stable version for now, which I think
>   Lasse will release a stable version this year;
> 
> - EROFS has a finalized on-disk MircoLZMA support since Linux 5.16,
>   so users can mount MicroLZMA since 5.16.  Yet currently EROFS is
>   not quite optimized for such slow algorithm, since it needs a
>   completely different strategy from LZ4.  I'm working on that
>   together with folio work;
> 
> - We need a similiar `fragment' feature to catch squashfs under a
>   lot of files.

Is that feature likely to be implemented for EROFS or is that out of scope?

> 
>> 
>> Is erofs just not meant for such small storages?
> 
> If you care more about the size of images, I personally prefer to
> squashfs for now until we handle all the things above.

Sure, I didn’t plan to migrate things over to EROFS just like that but it looks like a good candidate in the future!

Thank you and everyone involved for working on EROFS!

Best,
Paul Spooren

> 
> Thanks,
> Gao Xiang
> 
>> 
>> Thanks for all further comments!
>> 
>> Best,
>> Paul
>> 
>> [1]: https://downloads.openwrt.org/snapshots/targets/x86/64/openwrt-x86-64-rootfs.tar.gz