[PATCH] erofs-utils: lib: fix decodedcapacity integer overflow in inflate partial

[Utkal Singh]

decodedcapacity is declared as 'unsigned int' (32 bits). Two code
paths can silently overflow it and corrupt the heap:

1. The initial assignment shifts rq->decodedlength left by 4 bits
   when partial_decoding is set. Values exceeding UINT_MAX >> 4
   wrap to a small integer, causing malloc() to allocate an
   undersized buffer; a subsequent write overflows the heap.

2. The doubling loop left-shifts decodedcapacity by 1. Once it
   exceeds UINT_MAX >> 1 the result wraps to 0. On glibc,
   realloc(ptr, 0) returns a valid non-NULL pointer; the next
   write into that buffer is a silent heap overflow.

Fix both sites: change the type to size_t, cast rq->decodedlength
to size_t before the initial shift to force 64-bit arithmetic, and
add a guard before the doubling shift that returns -EFSCORRUPTED
via out_inflate_end when the value would overflow SIZE_MAX.

Signed-off-by: Utkal Singh <singhutkal015 at gmail.com>
---
 lib/decompress.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/lib/decompress.c b/lib/decompress.c
index f87efd5..ff703ae 100644
--- a/lib/decompress.c
+++ b/lib/decompress.c
@@ -4,6 +4,7 @@
  * Created by Huang Jianan <huangjianan at oppo.com>
  */
 #include <stdlib.h>
+#include <stdint.h>
 
 #include "erofs/decompress.h"
 #include "erofs/err.h"
@@ -251,13 +252,13 @@ static int z_erofs_decompress_deflate(struct z_erofs_decompress_req *rq)
 	unsigned int inputmargin;
 	struct libdeflate_decompressor *inf;
 	enum libdeflate_result ret;
-	unsigned int decodedcapacity;
+	size_t decodedcapacity;
 
 	inputmargin = z_erofs_fixup_insize(src, rq->inputsize);
 	if (inputmargin >= rq->inputsize)
 		return -EFSCORRUPTED;
 
-	decodedcapacity = rq->decodedlength << (4 * rq->partial_decoding);
+	decodedcapacity = (size_t)rq->decodedlength << (4 * rq->partial_decoding);
 	if (rq->decodedskip || rq->partial_decoding) {
 		buff = malloc(decodedcapacity);
 		if (!buff)
@@ -287,7 +288,12 @@ static int z_erofs_decompress_deflate(struct z_erofs_decompress_req *rq)
 			ret = -EFSCORRUPTED;
 			goto out_inflate_end;
 		}
-			decodedcapacity = decodedcapacity << 1;
+			if (decodedcapacity > SIZE_MAX >> 1) {
+				erofs_err("inflate: decompression buffer overflow");
+				ret = -EFSCORRUPTED;
+				goto out_inflate_end;
+			}
+			decodedcapacity <<= 1;
 			dest = realloc(buff, decodedcapacity);
 			if (!dest) {
 				ret = -ENOMEM;
-- 
2.43.0