linux-kernel - Re: [PATCH v2 00/11] Documentation: Add ABI to the admin guide

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20170421112221.58d1379b@vento.lan>
Date:   Fri, 21 Apr 2017 11:22:21 -0300
From:   Mauro Carvalho Chehab <mchehab@...pensource.com>
To:     Markus Heiser <markus.heiser@...marit.de>
Cc:     Jonathan Corbet <corbet@....net>,
        Linux Doc Mailing List <linux-doc@...r.kernel.org>,
        Mauro Carvalho Chehab <mchehab@...radead.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Kees Cook <keescook@...omium.org>,
        Anton Vorontsov <anton@...msg.org>,
        Colin Cross <ccross@...roid.com>,
        Tony Luck <tony.luck@...el.com>
Subject: Re: [PATCH v2 00/11] Documentation: Add ABI to the admin guide

Em Fri, 21 Apr 2017 08:37:45 +0200
Markus Heiser <markus.heiser@...marit.de> escreveu:

> Am 21.04.2017 um 01:21 schrieb Mauro Carvalho Chehab <mchehab@...pensource.com>:
> 
> > - I'm not a python programmer ;-) I just took Markus "generic" kernel-cmd
> >  code, hardcoding there a call to the script.
> > 
> >  With (a lot of) time, I would likely be able to find a solution to add
> >  the entire ABI logic there, but, in this case, we would lose the
> >  capability of calling the script without Sphinx.  
> 
> Hi Mauro,
> 
> I have no problem with calling the perl script, but your last sentence
> is not correct: We can implement a python module, which is used by sphinx
> and we can add a command line as well.

Markus,

Yeah, I guess technically it would be possible to make a Sphinx plugin 
that would also allow have a command line interface. I don't like this
kind of option, because the code can be come messy.

A better design would be to create a library and make two interfaces
for it, one for the ReST plugin and another one to be called via a
command line, but, in this case, we'll need to maintain 3 interdependent
components (library, command line, Sphinx plugin) instead of two (almost)
independent ones (a script and a Sphinx plugin).

So, IMO, the design I took is a good one, and it has a big advantage:
writing in perl is a way easier for me, and I can benefit from my
knowledge to write a script that performs well. On my desktop, it can
parse the entire ABI, search for a string there and output its result
in 100ms:

$ time ./scripts/get_abi.pl search usbip_status

/sys/devices/platform/usbip-vudc.%d/usbip_status
------------------------------------------------

Kernel version:		4.6
Date:			April 2016
Contact:		Krzysztof Opasiak <k.opasiak@...sung.com>
Defined on file:	Documentation/ABI/testing/sysfs-platform-usbip-vudc

Description:

Current status of the device.
Allowed values:
1 - Device is available and can be exported
2 - Device is currently exported
3 - Fatal error occurred during communication
  with peer


real	0m0.112s
user	0m0.106s
sys	0m0.005s

---

With regards to the decision of using perl instead of python,
see below. One might thing it is rant. It isn't. It is just a
matter of optimizing my development time.

I have lots of bad experiences with patchwork and even with
cgit (running on an old LTS debian machine) due to the lack of 
a consistent backward compatible API on python.

Most of my bad experiences on python scripts is related to how badly
it handles a file input with an unknown charset and unsigned chars > 0x7f. 
If Python can't recognize the a char as a valid ascii character 
(or the charset was not explicity defined - or the script doesn't
 know what's the original encoding charset),  the script crashes 
(ok, one could add a "try" block, but it is very painful to do that
all over the code). Also, the way the charset is specified on a python 
script changed several times during 2.x development (causing incompatible
changes), and again on 3.x. It is even worse if a python script is called 
by some other script, as, in such case, Python (not sure if all versions)
ignores script headers like:
	# -*- coding: utf-8 -*-

with would otherwise tell what's the default encoding.

If you look at patchwork git tree, you'll see that it took a really long
time since when the first patch trying to address those issues until the
last one was merged.

That's the first patch there[1]:

	commit ea39a9952e3fa647ebcb4bf16981ce941ec5236a
	Author: Mauro Carvalho Chehab <mchehab@...radead.org>
	Date:   Tue Nov 18 23:00:32 2008 -0200

	    Fix non-ascii character encodings on xmlrpc interface

That seems to be the last one[2]:
	commit 880fc52d2d4ccdcbf4a7b76f1b4ba6b9e7482dff
	Author: Siddhesh Poyarekar <siddhesh@...hat.com>
	Date:   Mon Jul 14 10:21:32 2014 +0800

	    parsemail: Fallback to common charsets when charset is None or x-unknown

AFAIKT, none of those charset fixes are due to a code regression, but,
instead, fixing parsing on different places of the code. So, it took
at least 6 years to get it right there.

So, my decision of writing it in perl is basically due to the fact
that I can write a reliable script it in a few hours, and won't 
need to be concerned that some weird char inside a file or some
new scripting interpreter version would cause my script to crash.

[1] git://github.com/getpatchwork/patchwork
[2] I guess I hit other patchwork charset bugs after 2014.

Thanks,
Mauro