linux-kernel - Re: [PATCH v2 1/4] drm/komeda: Add a new helper drm_color_ctm_s31_32_to_qm

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 15 Oct 2019 08:04:26 +0000
From:   "james qian wang (Arm Technology China)" <james.qian.wang@....com>
To:     Ilia Mirkin <imirkin@...m.mit.edu>
CC:     Liviu Dudau <Liviu.Dudau@....com>,
        "airlied@...ux.ie" <airlied@...ux.ie>,
        Brian Starkey <Brian.Starkey@....com>,
        "maarten.lankhorst@...ux.intel.com" 
        <maarten.lankhorst@...ux.intel.com>,
        "sean@...rly.run" <sean@...rly.run>,
        "Jonathan Chai (Arm Technology China)" <Jonathan.Chai@....com>,
        "Julien Yin (Arm Technology China)" <Julien.Yin@....com>,
        "Thomas Sun (Arm Technology China)" <thomas.Sun@....com>,
        "Lowry Li (Arm Technology China)" <Lowry.Li@....com>,
        Ayan Halder <Ayan.Halder@....com>,
        "Tiannan Zhu (Arm Technology China)" <Tiannan.Zhu@....com>,
        "Yiqi Kang (Arm Technology China)" <Yiqi.Kang@....com>,
        nd <nd@....com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "dri-devel@...ts.freedesktop.org" <dri-devel@...ts.freedesktop.org>,
        Ben Davis <Ben.Davis@....com>,
        "Oscar Zhang (Arm Technology China)" <Oscar.Zhang@....com>,
        "Channing Chen (Arm Technology China)" <Channing.Chen@....com>,
        Mihail Atanassov <Mihail.Atanassov@....com>
Subject: Re: [PATCH v2 1/4] drm/komeda: Add a new helper
 drm_color_ctm_s31_32_to_qm_n()

On Mon, Oct 14, 2019 at 11:48:38PM -0400, Ilia Mirkin wrote:
> On Mon, Oct 14, 2019 at 9:16 PM james qian wang (Arm Technology China)
> <james.qian.wang@....com> wrote:
> > On Mon, Oct 14, 2019 at 11:58:48AM -0400, Ilia Mirkin wrote:
> > > On Fri, Oct 11, 2019 at 1:43 AM james qian wang (Arm Technology China)
> > > <james.qian.wang@....com> wrote:
> > > > + *
> > > > + * Convert and clamp S31.32 sign-magnitude to Qm.n 2's complement.
> > > > + */
> > > > +uint64_t drm_color_ctm_s31_32_to_qm_n(uint64_t user_input,
> > > > +                                     uint32_t m, uint32_t n)
> > > > +{
> > > > +       u64 mag = (user_input & ~BIT_ULL(63)) >> (32 - n);
> > > > +       bool negative = !!(user_input & BIT_ULL(63));
> > > > +       s64 val;
> > > > +
> > > > +       /* the range of signed 2s complement is [-2^n+m, 2^n+m - 1] */
> > >
> > > This implies that n = 32, m = 0 would actually yield a 33-bit 2's
> > > complement number. Is that what you meant?
> >
> > Yes, since m doesn't include sign-bit So a Q0.32 is a 33bit value.
> 
> This goes counter to what the wikipedia page says [
> https://en.wikipedia.org/wiki/Q_(number_format) ]:
> 
> (reformatted slightly for text-only consumption):
> 
> """
> For example, a Q15.1 format number:
> 
> - requires 15+1 = 16 bits
> - its range is [-2^14, 2^14 - 2^-1] = [-16384.0, +16383.5] = [0x8000,
> 0x8001 ... 0xFFFF, 0x0000, 0x0001 ... 0x7FFE, 0x7FFF]
> - its resolution is 2^-1 = 0.5
> """
> 
> This suggests that the proper way to represent a standard 32-bit 2's
> complement integer would be Q32.0.
>

Yes you're right, I will send a new version to correct this code
according to the Wiki.

Thanks
James

>   -ilia