Re: [GSoC 2026] Multi-threaded decompression for fsck.erofs — design question on z_erofs_decompress() parallelism

[Utkal Singh]

On Sun, Mar 29, 2026 at 6:47 PM, Deepak Pathik wrote:
> for LZMA-compressed images, are pclusters in fsck.erofs always
> fixed-size and independently decompressible at the userspace level,
> or are there cases where a pcluster depends on the state left by a
> previous one?

Hi Deepak,

To answer your LZMA question: yes, each pcluster is independently
decompressible by design. You can verify this directly in
lib/decompress.c — z_erofs_decompress_lzma() calls lzma_stream_decoder()
and lzma_end() within a single invocation, with no persistent lzma_stream
across calls. The same holds for ZSTD and deflate. The on-disk format
enforces this: no pcluster depends on decompressor state from a
previous one.

The parallelism boundary you identified is correct. The deeper issue
is one level up: erofs_check_inode() is called sequentially in the
dispatch loop in fsck/main.c, and each call may decompress many
pclusters per inode. Inode-level dispatch is simpler than
pcluster-level because it avoids output ordering constraints.

One thing worth thinking through before wiring erofs_workqueue into
the fsck path: the existing queue in lib/workqueue.c is an unbounded
producer queue built for mkfs compression workloads. On a 34,000+
inode image, it will accumulate all inode descriptors in memory before
workers can drain it. Backpressure — either a bounded queue or a
semaphore on the existing one — matters here.

Two paths in the surrounding infrastructure also need fixing before
concurrent dispatch is correct:

  - erofs_read_one_data() in lib/io.c: lseek()+read() on a shared fd
    is a TOCTOU race under concurrent calls. pread(2) fixes it cleanly.

  - erofs_iget()/erofs_iput() in lib/inode.c: ref-count mutations
    without synchronisation. Concurrent iput() can double-free.

I sent an RFC on March 22 covering this design if it is useful context:

  https://lore.kernel.org/linux-erofs/CAGSu4WNBdB30K61xoUCi3FB9QR081fNh-1hoX1z2TZMk0nGpHQ@mail.gmail.com/

Happy to discuss further on the list.

Regards,
Utkal Singh


On Sun, 29 Mar 2026 at 18:47, Deepak Pathik <deepakpathik2005 at gmail.com> wrote:
>
> Hi,
>
> I'm Deepak Pathik, a second-year B.Tech student applying for the GSoC 2026 project on multi-threaded decompression support in fsck.erofs.
>
> While reading through the source, I traced the decompression path in erofs_verify_inode_data() and noticed that z_erofs_decompress() operates on a locally scoped struct z_erofs_decompress_req with its own input and output buffers — no shared mutable state between calls. My plan is to wire the existing erofs_workqueue (already used in lib/compress.c for mkfs.erofs) into the fsck extraction path at the pcluster level, with pwrite() for position-based output writes to avoid ordering locks.
>
> One thing I wanted to confirm before finalizing my proposal: for LZMA-compressed images, are pclusters in fsck.erofs always fixed-size and independently decompressible at the userspace level, or are there cases where a pcluster depends on the state left by a previous one? I want to make sure I'm not understating the LZMA case in my design.
>
> I've drafted a proposal and would be happy to share it for early feedback if that's useful.
>
> Thanks,
> Deepak Pathik
> https://github.com/deepakpathik
> deepakpathik2005 at gmail.com