Trying to work with the tests

[Ian Kent]

On 22/4/24 17:12, Gao Xiang wrote:
> Hi Ian,
>
> (+Cc Jingbo here).
>
> On 2024/4/22 16:31, Ian Kent wrote:
>> I'm new to the list so Hi to all,
>>
>>
>> I'm working with a heavily patched 5.14 kernel and I've gathered 
>> together patches to bring erofs
>>
>> up to 5.19 and I'm trying to run the erofs and fscache tests from a 
>> checkout of the 1.7.1 repo.
>>
>> (branch experimental-tests-fscache) and I have a couple of fails I 
>> can't quite work out so I'm
>>
>> hoping for a little halp.
>
> Thanks for your interest and provide the detailed infos.
>
> I guess a modified 5.14 kernel may be originated from RHEL 9?

Yes, that's right.

I am working on improving erofs support in RHEL which of course goes via 
CentOS Stream 9.


>
> I have a plan to backport the latest EROFS to CentOS stream 9, but
> currently I'm busy in internal stuffs, so it's still a bit delayed...

Right, Eric mentioned you were keen to help out.


The full back port is a bit much to do in one step, I'd like to let it 
settle for a minor release before considering

further back port effort. Of course any assistance is also welcome if 
and when you have time.


I must admit posting to the list is as much to introduce myself as it is 
to get a little help and advice.


>
>>
>>
>> For those familiar with erofs development and history it may look 
>> like some patches are missing
>>
>> but they may have already been present in the source tree I'm working 
>> with ... so just ask if
>>
>> you spot anything ...
>>
>>
>> Current set of patches I have is (the patch name corresponds to 
>> commit title, more or less):
>>
>> + erofs-clear-compacted_2b-if-compacted_4b_initial-gt-totalidx.patch
>> + erofs-add-support-for-the-full-decompressed-length.patch
>> + erofs-add-fiemap-support-with-iomap.patch
>> + 
>> erofs-remove-the-mapping-parameter-from-erofs_try_to_free_cached_page.patch
>> + erofs-directly-use-wrapper-erofs_page_is_managed-when-shrinking.patch
>> + erofs-dax-support-for-non-tailpacking-regular-file.patch
>> + erofs-fix-deadlock-when-shrink-erofs-slab.patch
>> + erofs-remove-useless-cache-strategy-of-DELAYEDALLOC.patch
>> + erofs-fix-unsafe-pagevec-reuse-of-hooked-pclusters.patch
>> + erofs-remove-the-fast-path-of-per-CPU-buffer-decompression.patch
>> + erofs-decouple-basic-mount-options-from-fs_context.patch
>> + erofs-add-multiple-device-support.patch
>> + erofs-get-compression-algorithms-directly-on-mapping.patch
>> + erofs-introduce-the-secondary-compression-head.patch
>> + erofs-introduce-readmore-decompression-strategy.patch
>> + erofs-rename-some-generic-methods-in-decompressor.patch
>> + 
>> libxz-Avoid-overlapping-memcpy-with-invalid-input-with-in-place-decompression.patch
>> + libxz-Add-MicroLZMA-decoder.patch
>> + erofs-lzma-compression-support.patch
>> + erofs-get-rid-of-lru-usage.patch
>> + erofs-dont-trigger-WARN-when-decompression-fails.patch
>> + erofs-rename-lz4_0pading-to-zero_padding.patch
>> + erofs-add-sysfs-interface.patch
>> + erofs-add-sysfs-node-to-control-sync-decompression-strategy.patch
>> + erofs-Replace-zero-length-array-with-flexible-array-member.patch
>> + erofs-clean-up-erofs_map_blocks-tracepoints.patch
>> + erofs-tidy-up-z_erofs_lz4_decompress.patch
>> + erofs-introduce-z_erofs_fixup_insize.patch
>> + erofs-support-unaligned-data-decompression.patch
>> + erofs-support-inline-data-decompression.patch
>> + erofs-add-on-disk-compressed-tail-packing-inline-support.patch
>> + erofs-introduce-meta-buffer-operations.patch
>> + erofs-use-meta-buffers-for-inode-operations.patch
>> + erofs-use-meta-buffers-for-super-operations.patch
>> + erofs-use-meta-buffers-for-xattr-operations.patch
>> + erofs-use-meta-buffers-for-zmap-operations.patch
>> + erofs-fix-fsdax-partition-offset-handling.patch
>> + erofs-avoid-unnecessary-z_erofs-decompressqueue_work-declaration.patch
>> + erofs-fix-small-compressed-files-inlining.patch
>> + erofs-fix-ztailpacking-on-gt-4GiB-filesystems.patch
>> + erofs-use-meta-buffers-for-erofs_read_superblock.patch
>> + erofs-get-rid-of-struct-z_erofs_collector.patch
>> + erofs-clean-up-preload_compressed_pages.patch
>> + erofs-silence-warnings-related-to-impossible-m_plen.patch
>> + erofs-clean-up-z_erofs_extent_lookback.patch
>> + erofs-refine-managed-inode-stuffs.patch
>> + erofs-add-sanity-check-0for-kobject-in-erofs_unregister_sysfs.patch
>> + erofs-use-meta-buffers-for-reading-directories.patch
>> + erofs-use-meta-buffers-for-inode-lookup.patch
>> + erofs-rename-ctime-to-mtime.patch
>> + erofs-Convert-from-invalidatepage-to-invalidate_folio.patch
>> + erofs-fix-use-after-free-of-on-stack-io.patch
>> + erofs-Convert-erofs-zdata-to-read_folio.patch
>> + erofs-Convert-to-release_folio.patch
>> + erofs-do-not-prompt-for-risk-any-more-when-using-big-pcluster.patch
>> + erofs-remove-obsoleted-comments.patch
>> + erofs-refine-on-disk-definition-comments.patch
>> + erofs-fix-buffer-copy-overflow-of-ztailpacking-feature.patch
>> + erofs-make-filesystem-exportable.patch
>> + erofs-support-idmapped-mounts.patch
>> + cachefiles-document-on-demand-read-mode.patch
>> + erofs-make-erofs_map_blocks-generally-available.patch
>> + erofs-add-fscache-mode-check-helper.patch
>> + erofs-register-fscache-volume.patch
>> + erofs-add-fscache-context-helper-functions.patch
>> + erofs-add-anonymous-inode-caching-metadata-for-data-blobs.patch
>> + erofs-add-erofs_fscache_read_folios-helper.patch
>> + erofs-register-fscache-context-for-primary-data-blob.patch
>> + erofs-register-fscache-context-for-extra-data-blobs.patch
>> + erofs-implement-fscache-based-metadata-read.patch
>> + erofs-implement-fscache-based-data-read-for-non-inline-layout.patch
>> + erofs-implement-fscache-based-data-read-for-inline-layout.patch
>> + erofs-implement-fscache-based-data-readahead.patch
>> + erofs-add-fsid-mount-option.patch
>> + 
>> erofs-change-to-use-asynchronous-io-for-fscache-readpage_readahead.patch
>> + erofs-scan-devices-from-device-table.patch
>> + 
>> erofs-leave-compressed-inodes-unsupported-in-fscache-mode-for-now.patch
>> + erofs-fix-crash-when-enable-tracepoint-cachefiles_prep_read.patch
>> + erofs-get-rid-of-struct-z_erofs_collection.patch
>> + erofs-get-rid-of-label-restart_now.patch
>> + erofs-simplify-z_erofs_pcluster_readmore.patch
>> + erofs-fix-backmost-member-of-z_erofs_decompress_frontend.patch
>> + erofs-missing-hunks.patch
>>
>>
>> The last patch consists of what looks like a few hunks added by Linus 
>> to complete a folio pull
>>
>> request that came in at the same time as the 5.19 erofs merge 
>> request. I know the list of
>>
>> patches isn't very useful but it should give some idea of what I have 
>> and maybe someone can
>>
>> spot a missing patch or so.
>>
>>
>> Anyway, my failing tests are erofs/021, erofs/022, erofs/024 and 
>> fscache/005.
>
> I guess the following failure fails as expected:
> erofs/021  -- uncompressed sub-page block sizes (esp. 512-byte block 
> sizes, since v6.4)
> erofs/022  -- long xattr prefix (since v6.4)

Yes, I saw this was xattr related.


There have been quite a lot of back ports done in the current RHEL 
kernel and I'm planning on working on

some VFS back ports too. But the VFS is so very spread out over several 
areas I'm probably going to need

to restrict my scope somewhat. I'll see what I can find regarding xattrs 
on my journey.

But you mention 6.4 so that's quite a long way ahead so I might leave 
that to the next release too ...


> erofs/024 -- deflate algorithm support (since v6.6)

Ok, I hadn't spotted that yet.

Thanks, deflate support sounds a bit far off too, at least for now.


>
> So these failures can be skipped on your side, I think I need to modify
> these tests for gracefully skipping ... That is also why all testcases
> are marked as "experimental" :-)

Sure, but having any tests at all is really useful when back porting 
changes.

I really appreciate the fact they are present at all.


Ian

>
> I'm not quite sure why "fscache/005" fails, hopefully Jingbo could
> help you on this.
>
> Thanks!
> Gao Xiang
>
>>
>> erofs/018 does not run due to "lzma compression is disabled, 
>> skipped." message which I think
>>
>> is too old a version of xz.
>>
>>
>> Any insight into cases that could cause these tests to fail would be 
>> much appreciated.
>>
>>
>> Ian
>>
>>