Problem in EROFS: Not able to read the files after mount

Saumya Panda saumya.iisc at gmail.com
Wed Jan 29 15:13:37 AEDT 2020 

Hi Gao,
  How you got the read amplification? I ran FIO on enwik9 (both Erofs and
SquashFs) and got the below output. Is there anyway to calculate the read
amplification from the below logs.

     Here filename (/mnt/enwik9_erofs/enwik9, /mnt/enwiki_sqfs/enwik9)
points to the mounted readonly file system(squasfs, erofs). But if I give
directory as a parameter instead of filename I am getting error(see the
logs at the end).

*FIO on Erofs:*

localhost:~> fio --name=randread --ioengine=libaio --iodepth=16
--rw=randread --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240
--group_reporting --filename=/mnt/enwik9_erofs/enwik9

randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=libaio, iodepth=16

...

fio-3.17-90-gd9b7

Starting 4 processes

Jobs: 4 (f=4): [r(4)][100.0%][r=381MiB/s][r=97.6k IOPS][eta 00m:00s]

randread: (groupid=0, jobs=4): err= 0: pid=34282: Mon Jan 27 01:04:55 2020

  read: IOPS=36.7k, BW=144MiB/s (150MB/s)(2048MiB/14271msec)

    slat (nsec): min=1305, max=135688k, avg=106650.48, stdev=493480.73

    clat (nsec): min=1970, max=136593k, avg=1629459.90, stdev=2639786.83

     lat (usec): min=3, max=136625, avg=1736.29, stdev=2772.32

    clat percentiles (usec):

     |  1.00th=[    48],  5.00th=[    69], 10.00th=[   251], 20.00th=[
437],

     | 30.00th=[   570], 40.00th=[   701], 50.00th=[   848], 60.00th=[
1029],

     | 70.00th=[  1336], 80.00th=[  2147], 90.00th=[  4015], 95.00th=[
5932],

     | 99.00th=[ 11600], 99.50th=[ 13304], 99.90th=[ 17171], 99.95th=[
20579],

     | 99.99th=[135267]

   bw (  KiB/s): min=16510, max=295435, per=76.91%, avg=113025.79,
stdev=23830.42, samples=112

   iops        : min= 4126, max=73857, avg=28254.82, stdev=5957.62,
samples=112

  lat (usec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=1.37%

  lat (usec)   : 100=5.45%, 250=3.15%, 500=14.74%, 750=18.99%, 1000=14.99%

  lat (msec)   : 2=20.14%, 4=11.09%, 10=8.42%, 20=1.62%, 50=0.04%

  lat (msec)   : 250=0.01%

  cpu          : usr=1.87%, sys=8.28%, ctx=144023, majf=1, minf=114

  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%,
>=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%,
>=64=0.0%

     issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0

     latency   : target=0, window=0, percentile=100.00%, depth=16



Run status group 0 (all jobs):

   READ: bw=144MiB/s (150MB/s), 144MiB/s-144MiB/s (150MB/s-150MB/s),
io=2048MiB (2147MB), run=14271-14271msec



Disk stats (read/write):

  loop0: ios=137357/0, merge=0/0, ticks=23020/0, in_queue=460, util=97.70%


*FIO on SquashFs:*


localhost:~/Downloads/erofs-utils> fio --name=randread --ioengine=libaio
--iodepth=16 --rw=randread --bs=4k --direct=0 --size=512M --numjobs=4
--runtime=240 --group_reporting --filename=/mnt/enwik9_sqsh/enwik9

randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=libaio, iodepth=16

...

fio-3.17-90-gd9b7

Starting 4 processes

Jobs: 4 (f=4): [r(4)][66.7%][r=1175MiB/s][r=301k IOPS][eta 00m:05s]

randread: (groupid=0, jobs=4): err= 0: pid=34389: Mon Jan 27 01:07:56 2020

  read: IOPS=55.4k, BW=216MiB/s (227MB/s)(2048MiB/9467msec)

    slat (nsec): min=1194, max=61065k, avg=67581.76, stdev=754174.73

    clat (usec): min=2, max=222014, avg=1075.25, stdev=5969.94

     lat (usec): min=3, max=235437, avg=1143.13, stdev=6341.32

    clat percentiles (usec):

     |  1.00th=[    39],  5.00th=[    40], 10.00th=[    40], 20.00th=[
41],

     | 30.00th=[    42], 40.00th=[    43], 50.00th=[    43], 60.00th=[
44],

     | 70.00th=[    45], 80.00th=[    48], 90.00th=[    63], 95.00th=[
3163],

     | 99.00th=[ 28443], 99.50th=[ 41157], 99.90th=[ 78119], 99.95th=[
89654],

     | 99.99th=[125305]

   bw (  KiB/s): min= 1985, max=991826, per=63.49%, avg=140649.83,
stdev=78204.76, samples=72

   iops        : min=  495, max=247955, avg=35161.00, stdev=19551.19,
samples=72

  lat (usec)   : 4=0.01%, 10=0.01%, 20=0.01%, 50=84.82%, 100=8.18%

  lat (usec)   : 250=0.37%, 500=0.09%, 750=0.24%, 1000=0.54%

  lat (msec)   : 2=0.43%, 4=0.46%, 10=1.29%, 20=1.93%, 50=1.30%

  lat (msec)   : 100=0.33%, 250=0.02%

  cpu          : usr=1.76%, sys=16.29%, ctx=14519, majf=0, minf=104

  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%,
>=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%,
>=64=0.0%

     issued rwts: total=524288,0,0,0 short=0,0,0,0 dropped=0,0,0,0

     latency   : target=0, window=0, percentile=100.00%, depth=16



Run status group 0 (all jobs):

   READ: bw=216MiB/s (227MB/s), 216MiB/s-216MiB/s (227MB/s-227MB/s),
io=2048MiB (2147MB), run=9467-9467msec



Disk stats (read/write):

  loop1: ios=177240/0, merge=0/0, ticks=199386/0, in_queue=75984,
util=73.95%



 Fio Test on SquashFs dir:


localhost:~/Downloads/erofs-utils> fio --name=randread --ioengine=libaio
--iodepth=16 --rw=randread --bs=4k --direct=0 --size=512M --numjobs=4
--runtime=240 --group_reporting --directory=/mnt/enwik9_sqsh/
randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=libaio, iodepth=16
...
fio-3.17-90-gd9b7
Starting 4 processes
randread: Laying out IO file (1 file / 512MiB)
fio: pid=0, err=30/file:filesetup.c:150, func=unlink, error=Read-only file
system
randread: Laying out IO file (1 file / 512MiB)
fio: pid=0, err=30/file:filesetup.c:150, func=unlink, error=Read-only file
system
randread: Laying out IO file (1 file / 512MiB)
fio: pid=0, err=30/file:filesetup.c:150, func=unlink, error=Read-only file
system
randread: Laying out IO file (1 file / 512MiB)
fio: pid=0, err=30/file:filesetup.c:150, func=unlink, error=Read-only file
system


Run status group 0 (all jobs):


On Wed, Jan 22, 2020 at 10:07 AM Gao Xiang <hsiangkao at aol.com> wrote:

> On Wed, Jan 22, 2020 at 09:27:45AM +0530, Saumya Panda wrote:
> > Hi Gao,
> >   Thanks for the info. After I enabled the said configuration, I am now
> > able to read the files after mount. But I am seeing Squashfs has better
> > compression ratio compared to Erofs (more than 60% than that of Erofs).
> Am
> > I missing something? I used lz4hc while making the Erofs image.
> >
> > ls -l enwik*
> > -rw-r--r-- 1 saumya users  61280256 Jan 21 03:22 enwik8.erofs.img
> > -rw-r--r-- 1 saumya users  37355520 Jan 21 03:34 enwik8.sqsh
> > -rw-r--r-- 1 saumya users 558133248 Jan 21 03:25 enwik9.erofs.img
> > -rw-r--r-- 1 saumya users 331481088 Jan 21 03:35 enwik9.sqsh
>
> Yes, it's working as expect. Currently EROFS is compressed in 4k
> fixed-sized output compression granularity as mentioned in many
> available materials. That is the use case for our smartphones.
> You should compare with similar block configuration of squashfs.
> and there are some 3rd data by other folks as well [1].
>
> In the future, we will support other compression algorithms and
> larger compressed size (> 4k).
>
> [1] In chinese,
>     https://blog.csdn.net/scnutiger/article/details/102507596
>
> Thanks,
> Gao Xiang
>
>

-- 
Thanks,
Saumya Prakash Panda
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/linux-erofs/attachments/20200129/2af827d7/attachment.htm>

More information about the Linux-erofs mailing list