linux-kernel - Re: [PATCH v3] zstd: fixed possible 'rtbTable' underflow in FSE

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <dfdb2c31-3cdc-43d3-9e5f-0356cf4c1a0d@meta.com>
Date: Tue, 13 Jan 2026 13:58:40 -0500
From: Felix Handte <felixh@...a.com>
To: Ilya Krutskih <devsec@....ru>, Nick Terrell <terrelln@...com>
Cc: David Sterba <dsterba@...e.com>, linux-kernel@...r.kernel.org,
        lvc-project@...uxtesting.org, stable@...r.kernel.org
Subject: Re: [PATCH v3] zstd: fixed possible 'rtbTable' underflow in
 FSE_normalizeCount()

Ilya, can you share any context for this patch? Do you have any evidence 
that `proba` can be negative?

A discussion was just started about this patch on the zstd repo [0]. I'm 
happy to discuss this here or there, whichever is more convenient.

But to my first pass inspection, this seems to be protecting an 
impossible situation. (Separately: if it could happen, the correct 
behavior would to catch it and return an error, not just skip it like 
this patch proposes.)

Thanks,
Felix

[0] https://github.com/facebook/zstd/issues/4567

On 12/11/25 12:19 PM, Ilya Krutskih wrote:
> 'rtbTable' may be underflowed because 'proba' is used without
> checking for a non-negative as index of rtbTable[].
> 
> Add check: proba >= 0
> 
> Cc: stable@...r.kernel.org # v5.10+
> Fixes: e0c1b49f5b67 ("lib: zstd: Upgrade to latest upstream zstd version 1.4.10")
> Signed-off-by: Ilya Krutskih <devsec@....ru>
> ---
>   lib/zstd/compress/fse_compress.c | 7 ++++---
>   1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/lib/zstd/compress/fse_compress.c b/lib/zstd/compress/fse_compress.c
> index 44a3c10becf2..6b83f8bc943a 100644
> --- a/lib/zstd/compress/fse_compress.c
> +++ b/lib/zstd/compress/fse_compress.c
> @@ -492,9 +492,10 @@ size_t FSE_normalizeCount (short* normalizedCounter, unsigned tableLog,
>                   stillToDistribute--;
>               } else {
>                   short proba = (short)((count[s]*step) >> scale);
> -                if (proba<8) {
> -                    U64 restToBeat = vStep * rtbTable[proba];
> -                    proba += (count[s]*step) - ((U64)proba<<scale) > restToBeat;
> +		if ((proba >= 0) && (proba < 8)) {
> +			U64 restToBeat = vStep * rtbTable[proba];
> +
> +			proba += (count[s]*step) - ((U64)proba<<scale) > restToBeat;
>                   }
>                   if (proba > largestP) { largestP=proba; largest=s; }
>                   normalizedCounter[s] = proba;