<div dir="auto">Hi Gao, <div dir="auto"><br></div><div dir="auto">okay. I will make those changes. Yes I would like to do kernel side changes too. Which branch to use for this? kernel's master?</div><div dir="auto"><br></div><div dir="auto">-Pratik</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 15 Oct, 2019, 5:15 AM Gao Xiang, <<a href="mailto:hsiangkao@aol.com">hsiangkao@aol.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Pratik,<br>
<br>
Some nitpick comments... Let me know if you have other thoughts<br>
<br>
On Mon, Oct 14, 2019 at 08:29:43PM +0530, Pratik Shinde wrote:<br>
> Added code for calculating crc of erofs blocks (4K size).for now it calculates<br>
> checksum of first block. but can modified to calculate crc for any no. of blocks<br>
> <br>
> modified patch based on review comments.<br>
> <br>
> Signed-off-by: Pratik Shinde <<a href="mailto:pratikshinde320@gmail.com" target="_blank" rel="noreferrer">pratikshinde320@gmail.com</a>><br>
> ---<br>
> include/erofs/internal.h | 1 +<br>
> include/erofs/io.h | 8 +++++<br>
> lib/io.c | 27 +++++++++++++++++<br>
> mkfs/main.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++<br>
> 4 files changed, 112 insertions(+)<br>
> <br>
> diff --git a/include/erofs/internal.h b/include/erofs/internal.h<br>
> index 5384946..53335bc 100644<br>
> --- a/include/erofs/internal.h<br>
> +++ b/include/erofs/internal.h<br>
> @@ -55,6 +55,7 @@ struct erofs_sb_info {<br>
> u32 feature_incompat;<br>
> u64 build_time;<br>
> u32 build_time_nsec;<br>
> + u32 feature;<br>
> };<br>
> <br>
> /* global sbi */<br>
> diff --git a/include/erofs/io.h b/include/erofs/io.h<br>
> index 9775047..e0ca8d9 100644<br>
> --- a/include/erofs/io.h<br>
> +++ b/include/erofs/io.h<br>
> @@ -19,6 +19,7 @@<br>
> int dev_open(const char *devname);<br>
> void dev_close(void);<br>
> int dev_write(const void *buf, u64 offset, size_t len);<br>
> +int dev_read(void *buf, u64 offset, size_t len);<br>
> int dev_fillzero(u64 offset, size_t len, bool padding);<br>
> int dev_fsync(void);<br>
> int dev_resize(erofs_blk_t nblocks);<br>
> @@ -31,5 +32,12 @@ static inline int blk_write(const void *buf, erofs_blk_t blkaddr,<br>
> blknr_to_addr(nblocks));<br>
> }<br>
> <br>
> +static inline int blk_read(void *buf, erofs_blk_t start,<br>
> + u32 nblocks)<br>
> +{<br>
> + return dev_read(buf, blknr_to_addr(start),<br>
> + blknr_to_addr(nblocks));<br>
> +}<br>
> +<br>
> #endif<br>
> <br>
> diff --git a/lib/io.c b/lib/io.c<br>
> index 7f5f94d..52f9424 100644<br>
> --- a/lib/io.c<br>
> +++ b/lib/io.c<br>
> @@ -207,3 +207,30 @@ int dev_resize(unsigned int blocks)<br>
> return dev_fillzero(st.st_size, length, true);<br>
> }<br>
> <br>
> +int dev_read(void *buf, u64 offset, size_t len)<br>
> +{<br>
> + int ret;<br>
> +<br>
> + if (cfg.c_dry_run)<br>
> + return 0;<br>
> +<br>
> + if (!buf) {<br>
> + erofs_err("buf is NULL");<br>
> + return -EINVAL;<br>
> + }<br>
> + if (offset >= erofs_devsz || len > erofs_devsz ||<br>
> + offset > erofs_devsz - len) {<br>
> + erofs_err("read posion[%" PRIu64 ", %zd] is too large beyond"<br>
> + "the end of device(%" PRIu64 ").",<br>
> + offset, len, erofs_devsz);<br>
> + return -EINVAL;<br>
> + }<br>
> +<br>
> + ret = pread64(erofs_devfd, buf, len, (off64_t)offset);<br>
> + if (ret != (int)len) {<br>
> + erofs_err("Failed to read data from device - %s:[%" PRIu64 ", %zd].",<br>
> + erofs_devname, offset, len);<br>
> + return -errno;<br>
> + }<br>
> + return 0;<br>
> +}<br>
> diff --git a/mkfs/main.c b/mkfs/main.c<br>
> index 91a018f..baaf02a 100644<br>
> --- a/mkfs/main.c<br>
> +++ b/mkfs/main.c<br>
> @@ -22,6 +22,10 @@<br>
> <br>
> #define EROFS_SUPER_END (EROFS_SUPER_OFFSET + sizeof(struct erofs_super_block))<br>
> <br>
> +/* number of blocks for calculating checksum */<br>
> +#define EROFS_CKSUM_BLOCKS 1<br>
> +#define EROFS_FEATURE_SB_CHKSUM 0x0001<br>
<br>
How about Moving EROFS_FEATURE_SB_CHKSUM to erofs_fs.h since it's<br>
an on-disk definition,<br>
<br>
> +<br>
> static void usage(void)<br>
> {<br>
> fprintf(stderr, "usage: [options] FILE DIRECTORY\n\n");<br>
> @@ -85,6 +89,10 @@ static int parse_extended_opts(const char *opts)<br>
> return -EINVAL;<br>
> cfg.c_force_inodeversion = FORCE_INODE_EXTENDED;<br>
> }<br>
> +<br>
> + if (MATCH_EXTENTED_OPT("nocrc", token, keylen)) {<br>
> + sbi.feature &= ~EROFS_FEATURE_SB_CHKSUM;<br>
> + }<br>
> }<br>
> return 0;<br>
> }<br>
> @@ -180,6 +188,7 @@ int erofs_mkfs_update_super_block(struct erofs_buffer_head *bh,<br>
> .meta_blkaddr = sbi.meta_blkaddr,<br>
> .xattr_blkaddr = 0,<br>
> .feature_incompat = cpu_to_le32(sbi.feature_incompat),<br>
> + .checksum = 0<br>
> };<br>
> const unsigned int sb_blksize =<br>
> round_up(EROFS_SUPER_END, EROFS_BLKSIZ);<br>
> @@ -202,6 +211,70 @@ int erofs_mkfs_update_super_block(struct erofs_buffer_head *bh,<br>
> return 0;<br>
> }<br>
> <br>
> +#define CRCPOLY 0x82F63B78<br>
> +static inline u32 crc32c(u32 seed, unsigned char const *in, size_t len)<br>
> +{<br>
> + int i;<br>
> + u32 crc = seed;<br>
> +<br>
> + while (len--) {<br>
> + crc ^= *in++;<br>
> + for (i = 0; i < 8; i++) {<br>
> + crc = (crc >> 1) ^ ((crc & 1) ? CRCPOLY : 0);<br>
> + }<br>
> + }<br>
> + return crc;<br>
> +}<br>
> +<br>
> +/* calculate checksum for first n blocks */<br>
> +u32 erofs_calc_blk_checksum(erofs_blk_t nblks, u32 *crc)<br>
> +{<br>
> + char *buf;<br>
> + int err = 0;<br>
> +<br>
> + buf = malloc(nblks * EROFS_BLKSIZ);<br>
> + err = blk_read(buf, 0, nblks);<br>
> + if (err) {<br>
> + erofs_err("Failed to calculate erofs checksum - %s",<br>
> + erofs_strerror(err));<br>
> + return err;<br>
> + }<br>
> + *crc = crc32c(0, (const unsigned char *)buf, nblks * EROFS_BLKSIZ);<br>
> + free(buf);<br>
> + return 0;<br>
> +}<br>
> +<br>
> +void erofs_write_checksum()<br>
<br>
How about naming write_sb_checksum?<br>
My idea is that this is a checksum in super block (rather than<br>
a checksum only for super block [0th block])<br>
<br>
Let me know if you have another thought...<br>
<br>
> +{<br>
> + struct erofs_super_block *sb;<br>
> + char buf[EROFS_BLKSIZ];<br>
> + int ret = 0;<br>
> + u32 crc;<br>
> +<br>
> + ret = erofs_calc_blk_checksum(EROFS_CKSUM_BLOCKS, &crc);<br>
> + if (ret) {<br>
> + return;<br>
> + }<br>
> + ret = blk_read(buf, 0, 1);<br>
> + if (ret) {<br>
> + erofs_err("error reading super-block structure");<br>
> + return;<br>
> + }<br>
> +<br>
> + sb = (struct erofs_super_block *)((u8 *)buf + EROFS_SUPER_OFFSET);<br>
> + if (le32_to_cpu(sb->magic) != EROFS_SUPER_MAGIC_V1) {<br>
> + erofs_err("not an erofs image");<br>
<br>
As the previous comments, I am little care about these print messages<br>
since users will only see this and "error reading super-block structure"<br>
"not an erofs image" makes confused for them... (They don't know what<br>
the internal process is doing)<br>
<br>
BTW, it looks good to me as a whole... Do you have some time on<br>
kernel side as well? :)<br>
<br>
Thanks,<br>
Gao Xiang<br>
<br>
> + return;<br>
> + }<br>
> + sb->checksum = cpu_to_le32(crc);<br>
> + ret = blk_write(buf, 0, 1);<br>
> + if (ret) {<br>
> + erofs_err("error writing 0th block to disk - %s",<br>
> + erofs_strerror(ret));<br>
> + return;<br>
> + }<br>
> +}<br>
> +<br>
> int main(int argc, char **argv)<br>
> {<br>
> int err = 0;<br>
> @@ -217,6 +290,7 @@ int main(int argc, char **argv)<br>
> <br>
> cfg.c_legacy_compress = false;<br>
> sbi.feature_incompat = EROFS_FEATURE_INCOMPAT_LZ4_0PADDING;<br>
> + sbi.feature = EROFS_FEATURE_SB_CHKSUM;<br>
> <br>
> err = mkfs_parse_options_cfg(argc, argv);<br>
> if (err) {<br>
> @@ -301,6 +375,8 @@ int main(int argc, char **argv)<br>
> err = -EIO;<br>
> else<br>
> err = dev_resize(nblocks);<br>
> + if (sbi.feature & EROFS_FEATURE_SB_CHKSUM)<br>
> + erofs_write_checksum();<br>
> exit:<br>
> z_erofs_compress_exit();<br>
> dev_close();<br>
> -- <br>
> 2.9.3<br>
> <br>
</blockquote></div>