linux-kernel - Re: [PATCH v4 00/12] [dt-bindings] [media] Add document file and driver for Sony CXD2880 DVB-T2/T tuner + demodulator

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAOFm3uEYfMH8Zj8uEx-D9yYrTyDMTG_j02619esHu-j0brQKaA@mail.gmail.com>
Date:   Thu, 14 Dec 2017 15:25:13 +0100
From:   Philippe Ombredanne <pombredanne@...b.com>
To:     Mauro Carvalho Chehab <mchehab@...pensource.com>
Cc:     "Takiguchi, Yasunari" <Yasunari.Takiguchi@...y.com>,
        "akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "devicetree@...r.kernel.org" <devicetree@...r.kernel.org>,
        "linux-media@...r.kernel.org" <linux-media@...r.kernel.org>,
        "tbird20d@...il.com" <tbird20d@...il.com>,
        "frowand.list@...il.com" <frowand.list@...il.com>,
        "Yamamoto, Masayuki" <Masayuki.Yamamoto@...y.com>,
        "Nozawa, Hideki (STWN)" <Hideki.Nozawa@...y.com>,
        "Yonezawa, Kota" <Kota.Yonezawa@...y.com>,
        "Matsumoto, Toshihiko" <Toshihiko.Matsumoto@...y.com>,
        "Watanabe, Satoshi (SSS)" <Satoshi.C.Watanabe@...y.com>,
        "Bird, Timothy" <Tim.Bird@...y.com>
Subject: Re: [PATCH v4 00/12] [dt-bindings] [media] Add document file and
 driver for Sony CXD2880 DVB-T2/T tuner + demodulator

Dear Mauro,

On Thu, Dec 14, 2017 at 11:55 AM, Mauro Carvalho Chehab
<mchehab@...pensource.com> wrote:

> SPDX is a new requirement that started late on Kernel 4.14 development
> cycle (and whose initial changes were merged directly at Linus tree).
> Not all existing files have it yet, as identifying the right license
> on existing files is a complex task, but if you do a:
>
>         $ git grep SPDX $(find . -name Makefile) $(find . -name Kconfig)
>
> You'll see that lot of such files have it already.

FWIW, short of having SPDX tags, identifying the right license on
existing files is not a super complex task: if boils down to running
many diffs.

Take the ~60K files in kernel, and about 6K license and notices
reference texts. Then compute a pairwise diff of each of the 60K file
against the 6K reference texts. Repeat the pairwise diff a few more
times, say 10 times, as multiple licenses may appear in any given
kernel file. And keep the diffs that have the fewest
difference/highest similarity with the reference texts as the detected
license. Done!

The only complex thing is that if you have a fast diff that runs at
0.1 millisec end-to-end per diff, you still have 3.6B diffs to do and
this would take about 250 days on one thread. Even with a beefy 250
core CPU, that would still be a day (and quite few kilo watts) . So
the whole trick is to avoid doing a diffs if not really needed. This
is what I do in my scancode-toolkit (that I used/use to help Greg and
Thomas with kernel license scans). Net effect is that on a laptop on 8
threads it takes ~20 minutes to scan a whole kernel using this
diff-based approach and obtain a fairly accurate license detection.
-- 
Cordially
Philippe Ombredanne