linux-kernel - Re: [PATCH v12 09/16] iio: afe: rescale: fix accuracy for small fractional scales

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <6901bb1b-b701-9760-1d36-89e8127c1953@axentia.se>
Date:   Sun, 9 Jan 2022 21:07:30 +0100
From:   Peter Rosin <peda@...ntia.se>
To:     Andy Shevchenko <andy.shevchenko@...il.com>,
        Liam Beguin <liambeguin@...il.com>
Cc:     Jonathan Cameron <jic23@...nel.org>,
        Lars-Peter Clausen <lars@...afoo.de>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        linux-iio <linux-iio@...r.kernel.org>,
        devicetree <devicetree@...r.kernel.org>,
        Rob Herring <robh+dt@...nel.org>
Subject: Re: [PATCH v12 09/16] iio: afe: rescale: fix accuracy for small
 fractional scales

Hi!

On 2022-01-09 14:02, Andy Shevchenko wrote:
> On Sat, Jan 8, 2022 at 10:53 PM Liam Beguin <liambeguin@...il.com> wrote:
>>
>> The approximation caused by integer divisions can be costly on smaller
>> scale values since the decimal part is significant compared to the
>> integer part. Switch to an IIO_VAL_INT_PLUS_NANO scale type in such
>> cases to maintain accuracy.
> 

...

> ...
> 
>> +               rem2 = *val % (int)tmp;
>> +               *val = *val / (int)tmp;
>> +
>> +               *val2 = rem / (int)tmp;
> 
> Hmm... You divide s64 by 10^9, which means that the maximum value can
> be ~10^10 / 2 (because 2^64-1 ~= 10^19), but this _might_ be bigger
> than 'int' can hold. Can you confirm that tmp can't be so big?
> 

If tmp is so big that it does not fit in 32-bits, that is indeed a
problem, and it means that the scale has overflowed. However, the
problem with the scale overflowing very much existed prior to this
series. Doing something about that overflow is not strictly related
to improving the accuracy for small scale factors.

Cheers,
Peter