Optimizing write_uncompressed_file_from_fd()

[Gao Xiang]

Hi Daan,

On 2023/8/31 17:48, Gao Xiang wrote:
> Hi Daan,
> 
> On 2023/8/31 17:33, Daan De Meyer wrote:
>> Hi,
>>
>> For hacking on systemd, we build disk images using mkosi, and use an erofs filesystem for the /usr directory. When hacking on systemd, we would like to be able to rebuild the disk image as fast as possible. One part of rebuilding the image that takes a while is generating the erofs filesystem. I had a look at the mkfs source code for erofs and noticed that in
>> write_uncompressed_file_from_fd(), there is no usage of FICLONERANGE or copy_file_range() to speed up copying data from the filesystem to the erofs filesystem. Would it be possible to use either of these to make copying data in mkfs.erofs faster when the data does not need to be compressed?
> 
> Thanks for the email!
> 
> Yes, that is indeed useful, actually mmap I/Os can be used
> to boost up write_uncompressed_file_from_fd () as well...
> Actually many enhancements are limited by the current
> development resource (multithreaded compression support might
> be the top-1 priority for userspace side from end users),
> but I have to resolve them one-by-one since EROFS is still a
> quite young project compared to other approaches and beyond
> the original targeted use cases (so most Android vendors won't
> pay any attention to improvements that are not critically
> important for their scenarios.)
> 
> I mostly focus on some kernel improvements since Google needs
> us to support 16k page size + 4k EROFS block size.  That is
> currently my top thing to resolve first...
> 
> (quite limited by real world time...)

I already checked this part, it optimizes about 100ms for
the linux source code tree on my own developping machine.

Also after some profiling, there are some other stuffs that
needs to be optmized as well (e.g. optimize metadata write
counts)

Anyway, I'm about releasing erofs-utils v1.7.  I tend to
resolve this in the next version and there are a lot
TODOs on my side.

Thanks,
Gao Xiang

> 
> Thanks,
> Gao Xiang
> 
>>
>> Cheers,
>>
>> Daan De Meyer