linux-kernel - Re: [PATCH] checkpatch: improve handling of email comments

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <8b9beccf81735f2e042447026a1043f8d21c2566.camel@perches.com>
Date:   Fri, 30 Oct 2020 21:14:11 -0700
From:   Joe Perches <joe@...ches.com>
To:     Lukas Bulwahn <lukas.bulwahn@...il.com>
Cc:     Dwaipayan Ray <dwaipayanray1@...il.com>,
        linux-kernel-mentees@...ts.linuxfoundation.org,
        linux-kernel@...r.kernel.org, yashsri421@...il.com
Subject: Re: [PATCH] checkpatch: improve handling of email comments

On Fri, 2020-10-30 at 12:58 +0100, Lukas Bulwahn wrote:
> 
> On Fri, 30 Oct 2020, Joe Perches wrote:
> 
> > On Fri, 2020-10-30 at 14:37 +0530, Dwaipayan Ray wrote:
> > > checkpatch has limited support for parsing email comments. It only
> > > support single name comments or single after address comments.
> > > Whereas, RFC 5322 specifies that comments can be inserted in
> > > between any tokens of the email fields.
> > > 
> > > Improve comment parsing mechanism in checkpatch.
> > > 
> > > What is handled now:
> > > 
> > > - Multiple name/address comments
> > > - Comments anywhere in between name/address
> > > - Nested comments like (John (Doe))
> > > 
> > > A brief analysis of checkpatch output on v5.0..v5.7 showed that
> > > after these modifications, the number of BAD_SIGN_OFF warnings
> > > came down from 2944 to 1424, and FROM_SIGN_OFF_MISMATCH came
> > > down from 2366 to 2330.
> > > 
> > > So, a total of 1556 false positives were resolved in total.
> > 
> > A mere reduction in messages emitted isn't necessarily good.
> > 
> 
> Agree. That is why I also went through the list of those warnings.
> 
> I could not spot any obvious true positive among the reduced ones.
>  
> 
> > Please send me privately a complete list of these nominally
> > false positive messages that are no longer emitted.
> > 
> > I believe one of the relatively common incorrect messages is
> > for the cc: <stable@...r.kernel.org> where a version number is
> > continued on the same line after a #.
> > 
> > CC: stable@...r.kernel.org # for versions x.y.z and above
> > 
> 
> That was one,

It's not just one, it's ~90% of the list that Dwaipayan sent me.

$ wc -l mismatches
831 mismatches

$ grep -v -i stable mismatches | wc -l
98

> another common pattern was just quotes put inconsistently at 
> different places.

Yes, there are some defects there.
But there are also now false negatives.

For instance, this is not appropriate to ignore:

WARNING:BAD_SIGN_OFF: email address 'jacek.anaszewski@...il.com, linux-leds@...r.kernel.org, linux-kernel@...r.kernel.org, dmurphy@...com' might be better as 'jacek.anaszewski@...il.com,linux-leds@...r.kernel.org, linux-kernel@...r.kernel.org, dmurphy@...com'

>From the file that Dwaipayan sent me, all the rest not including the
stable variants, which IMO should be handled separately, are below.

Of these 98 in total, 60+% are unicode which IMO should always be quoted
and most are doubled with BAD_SIGN_OFF doubling FROM_SIGN_OFF_MISMATCH
(and I don't quite understand why it's "From:/" then "Signed-off-by:"

$ grep -v -i stable dwai | sort | uniq -c | sort -rn
     31 WARNING:BAD_SIGN_OFF: email address '周琰杰 (Zhou Yanjie) <zhouyanjie@...yeetech.com>' might be better as '"周琰杰"(Zhou Yanjie) <zhouyanjie@...yeetech.com>'
     30 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: "周琰杰"(Zhou Yanjie) <zhouyanjie@...yeetech.com>' != 'Signed-off-by: 周琰杰 (Zhou Yanjie) <zhouyanjie@...yeetech.com>'
     
These 29 in total would be better stripping any bits in parentheses from
the name portion only when _not_ inside quotes.

     20 WARNING:BAD_SIGN_OFF: email address 'Thomas Hellström (VMware) <thomas_os@...pmail.org>' might be better as '"Thomas Hellström"(VMware) <thomas_os@...pmail.org>'
      5 WARNING:BAD_SIGN_OFF: email address 'H. Peter Anvin (Intel) <hpa@...or.com>' might be better as '"H. Peter Anvin"(Intel) <hpa@...or.com>'
      

      1 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: "Thomas Hellström"(VMware) <thomas_os@...pmail.org>' != 'Signed-off-by: Thomas Hellström (VMware) <thomas_os@...pmail.org>'
      
      1 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: "Srivatsa S. Bhat"(VMware) <srivatsa@...il.mit.edu>' != 'Signed-off-by: Srivatsa S. Bhat (VMware) <srivatsa@...il.mit.edu>'
      1 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: JanNieuwenhuizen(janneke) <janneke@....org>' != 'Signed-off-by: Jan Nieuwenhuizen <janneke@....org>'  
      1 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: "Frédéric Pierret"(fepitre) <frederic.pierret@...es-os.org>' != 'Signed-off-by: Frédéric Pierret (fepitre) <frederic.pierret@...es-os.org>'

So these 8 others are ones where quotes are either oddly placed
or perhaps should always exist and the comment in parentheses
is suggested poorly.  7 of these should be fixed and one should
still be reported.

      1 WARNING:BAD_SIGN_OFF: email address '"Thomas Hellström (VMware)" <thomas_os@...pmail.org>' might be better as '"Thomas Hellström"(VMware) <thomas_os@...pmail.org>'
      1 WARNING:BAD_SIGN_OFF: email address 'Srivatsa S. Bhat (VMware) <srivatsa@...il.mit.edu>' might be better as '"Srivatsa S. Bhat"(VMware) <srivatsa@...il.mit.edu>'
      1 WARNING:BAD_SIGN_OFF: email address '"Rantala, Tommi T. (Nokia - FI/Espoo)" <tommi.t.rantala@...ia.com>' might be better as '"Rantala, Tommi T."(Nokia - FI/Espoo) <tommi.t.rantala@...ia.com>'
      1 WARNING:BAD_SIGN_OFF: email address '"Kai Mäkisara (Kolumbus)" <kai.makisara@...umbus.fi>' might be better as '"Kai Mäkisara"(Kolumbus) <kai.makisara@...umbus.fi>'
      1 WARNING:BAD_SIGN_OFF: email address 'jacek.anaszewski@...il.com, linux-leds@...r.kernel.org, linux-kernel@...r.kernel.org, dmurphy@...com' might be better as 'jacek.anaszewski@...il.com,linux-leds@...r.kernel.org, linux-kernel@...r.kernel.org, dmurphy@...com'
      1 WARNING:BAD_SIGN_OFF: email address 'Frédéric Pierret (fepitre) <frederic.pierret@...es-os.org>' might be better as '"Frédéric Pierret"(fepitre) <frederic.pierret@...es-os.org>'
      1 WARNING:BAD_SIGN_OFF: email address 'David.Laight@...lab.com (big endian system concerns)' might be better as 'David.Laight@...lab.com(big endian system concerns)'
      1 WARNING:BAD_SIGN_OFF: email address 'apenwarr@...net.net (Avery Pennarun)' might be better as 'apenwarr@...net.net(Avery Pennarun)'