[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAJCx=gmVtxgnWbRKRYMOjVXAVRJciZ2w-5GxUQ4D+1GwMZ4k2Q@mail.gmail.com>
Date: Tue, 11 Apr 2017 16:59:31 -0700
From: Matt Ranostay <matt.ranostay@...sulko.com>
To: Andreas Klinger <ak@...klinger.de>
Cc: Jonathan Cameron <jic23@...nel.org>,
Linus Walleij <linus.walleij@...aro.org>,
Hartmut Knaack <knaack.h@....de>,
Lars-Peter Clausen <lars@...afoo.de>,
Peter Meerwald <pmeerw@...erw.net>,
Vlad Dogaru <ddvlad@...il.com>,
Akinobu Mita <akinobu.mita@...il.com>,
Wei Yongjun <yongjun_wei@...ndmicro.com.cn>,
Aniroop Mathur <a.mathur@...sung.com>,
"linux-iio@...r.kernel.org" <linux-iio@...r.kernel.org>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v2] IIO: bmp280-core.c: fix error in humidity calculation
On Mon, Apr 10, 2017 at 10:00 AM, Andreas Klinger <ak@...klinger.de> wrote:
> While calculating the compensation of the humidity there are negative values
> interpreted as unsigned because of unsigned variables used. These values as
> well as the constants need to be casted to signed as indicated by the
> documentation of the sensor.
>
> Changes in v2:
> - cast also constants to signed; otherwise calculation is not correct
>
> I removed the Acked-By of Linus Walleij because of another change in the
> calculation. Don't know how it should be treated.
>
> Signed-off-by: Andreas Klinger <ak@...klinger.de>
Reviewed-by: Matt Ranostay <matt.ranostay@...sulko.com>
> ---
> drivers/iio/pressure/bmp280-core.c | 11 ++++++-----
> 1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/iio/pressure/bmp280-core.c b/drivers/iio/pressure/bmp280-core.c
> index 4d18826ac63c..d82b788374b6 100644
> --- a/drivers/iio/pressure/bmp280-core.c
> +++ b/drivers/iio/pressure/bmp280-core.c
> @@ -175,11 +175,12 @@ static u32 bmp280_compensate_humidity(struct bmp280_data *data,
> }
> H6 = sign_extend32(tmp, 7);
>
> - var = ((s32)data->t_fine) - 76800;
> - var = ((((adc_humidity << 14) - (H4 << 20) - (H5 * var)) + 16384) >> 15)
> - * (((((((var * H6) >> 10) * (((var * H3) >> 11) + 32768)) >> 10)
> - + 2097152) * H2 + 8192) >> 14);
> - var -= ((((var >> 15) * (var >> 15)) >> 7) * H1) >> 4;
> + var = ((s32)data->t_fine) - (s32)76800;
> + var = ((((adc_humidity << 14) - (H4 << 20) - (H5 * var))
> + + (s32)16384) >> 15) * (((((((var * H6) >> 10)
> + * (((var * (s32)H3) >> 11) + (s32)32768)) >> 10)
> + + (s32)2097152) * H2 + 8192) >> 14);
> + var -= ((((var >> 15) * (var >> 15)) >> 7) * (s32)H1) >> 4;
>
> return var >> 12;
> };
> --
> 2.1.4
>
>
> --
Powered by blists - more mailing lists