[PATCH] erofs-utils: avoid redundant memcpy and sha256() for dedupe
wangzijie
wangzijie1 at honor.com
Fri Aug 15 19:54:49 AEST 2025
> Hi Zijie,
>
> On 2025/8/15 16:44, wangzijie wrote:
> > We have already use xxh64() for filtering first for dedupe, when we
> > need to skip the same xxh64 hash, no need to do memcpy and sha256(),
> > relocate the code to avoid it.
> >
> > Signed-off-by: wangzijie <wangzijie1 at honor.com>
>
> Thanks for the patch, it makes sense to me since we only keep one
> record according to xxh64 (instead of sha256) for now:
>
> Reviewed-by: Gao Xiang <hsiangkao at linux.alibaba.com>
>
> Although I think multi-threaded deduplication is more useful, see:
> https://github.com/erofs/erofs-utils/issues/25
> but I'm not sure if you're interested in it... ;-)
Hi Xiang,
Thank you for providing this information, I want to optimize mkfs time with
dedupe option and send this patch. I will find time to research Yifan's demo
of multi-threaded deduplication and try to provide some help.
> Thanks,
> Gao Xiang
More information about the Linux-erofs
mailing list