linux-kernel - Re: [RFC PATCH v1 2/6] kernel-doc: replace kernel-doc perl parser with a pure python one (WIP)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 25 Jan 2017 11:35:38 +0100
From:   Daniel Vetter <daniel@...ll.ch>
To:     Jani Nikula <jani.nikula@...el.com>
Cc:     Jonathan Corbet <corbet@....net>,
        Markus Heiser <markus.heiser@...marit.de>,
        Mauro Carvalho Chehab <mchehab@...radead.org>,
        Daniel Vetter <daniel.vetter@...ll.ch>,
        Matthew Wilcox <mawilcox@...rosoft.com>,
        "linux-doc @ vger . kernel . org List" <linux-doc@...r.kernel.org>,
        "linux-kernel @ vger . kernel . org List" 
        <linux-kernel@...r.kernel.org>
Subject: Re: [RFC PATCH v1 2/6] kernel-doc: replace kernel-doc perl parser
 with a pure python one (WIP)

On Wed, Jan 25, 2017 at 12:24:31PM +0200, Jani Nikula wrote:
> Finally, while I'd love to see scripts/kernel-doc go, I do have to ask
> if changing roughly 3k lines of Perl to roughly 3k lines of Python (*)
> really makes everything better? They both still parse everything using a
> large pile of regular expressions and a clunky state machine. When I
> look at the code, I'm afraid I do not get that liberating feeling of
> throwing out old junk in favor of something small or elegant or even
> obviously more maintainable than the old one. The new one offers more
> features, but repeatedly we face the problem that it's all lumped in
> together with the parser change. We should be able to look at the parser
> change and the other improvements separately.

I share this concern a lot. The kernel-doc perl is a horror show, but it's
a horror show that 3-4 people now somewhat understand. Simply translating
the entire script into python leaves us with the same horror show, but in
a different language. And personally I'm not versed at all in either of
them (and I think that applies to many kernel hackers), so seems a wash.

If the new script would implement the state machinery in some
parser-combinator library to make it much easier to maintain, while still
being bug-for-bug compatible, then I'd be much, much more in favour of
doing this. And once we go to that amount of effort, then rewriting it in
python for more consistency with sphinx is definitely a good idea.

> That said, perhaps having an elegant parser (perhaps based on a compiler
> plugin) is incompatible with the idea of making it a bug-for-bug drop-in
> replacement of the old one, and it's something we need to think about.

Yeah, I fear we'll always need our own parser to avoid breaking the world.
But there's definitely better ways out there to write parsers than
cobbling together regexes in a state machine that uses globals :-)
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch