linux-kernel - Re: [PATCH 1/1] lib: remove leading spaces before tabs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YMHW80wUyFpW6Utf@smile.fi.intel.com>
Date:   Thu, 10 Jun 2021 12:10:11 +0300
From:   Andy Shevchenko <andy.shevchenko@...il.com>
To:     Joe Perches <joe@...ches.com>
Cc:     "Leizhen (ThunderTown)" <thunder.leizhen@...wei.com>,
        Andy Shevchenko <andy@...nel.org>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        linux-kernel <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 1/1] lib: remove leading spaces before tabs

On Wed, Jun 09, 2021 at 02:44:54PM -0700, Joe Perches wrote:
> On Wed, 2021-06-09 at 13:30 +0300, Andy Shevchenko wrote:
> > On Wed, Jun 9, 2021 at 9:21 AM Leizhen (ThunderTown)
> > <thunder.leizhen@...wei.com> wrote:
> > > On 2021/6/9 13:15, Leizhen (ThunderTown) wrote:
> > > > On 2021/6/9 0:03, Joe Perches wrote:
> > > > > On Tue, 2021-06-08 at 17:00 +0800, Leizhen (ThunderTown) wrote:
> > > > > > On 2021/6/8 16:44, Andy Shevchenko wrote:
> > > > > > > On Tue, Jun 8, 2021 at 10:14 AM Zhen Lei <thunder.leizhen@...wei.com> wrote:
> > > > > > > > 
> > > > > > > > 1) Run the following command to find and remove the leading spaces before
> > > > > > > >    tabs:
> > > > > > > >    find lib/ -type f | xargs sed -r -i 's/^[ ]+\t/\t/'
> > > > > > > 
> > > > > > > Hint for the future, try to use what Git provides, for example `git
> > > > > > > ls-files -- lib/`.
> > > > > > 
> > > > > > Okay, thanks. I learned a new trick.
> > 
> > > > > For instance:
> > > > > 
> > > > > $ git diff --stat lib
> > > > > $ git ls-files lib/ | \
> > > > >   xargs ./scripts/checkpatch.pl --types=SPACE_BEFORE_TAB --fix-inplace
> > > 
> > > I just tried it. It's too slow.
> > 
> > If checkpath accepts infinite positional arguments, then proper way of
> > use (that's how I do with simple perl regexps) is to
> > 
> > scripts/checkpatch.pl --types=SPACE_BEFORE_TAB --fix-inplace -- $(git ls-files lib/)
> 
> That won't always work:
> 
> $ git ls-files | xargs | wc -c
> 2716737
> 
> Nothing accepts infinite positional arguments.
> You are always limited by the maximum length of a command line
> 
> $ getconf ARG_MAX
> 2097152
> 
> xargs has:
> 
> $ xargs --show-limits
> Your environment variables take up 3517 bytes
> POSIX upper limit on argument length (this system): 2091587
> POSIX smallest allowable upper limit on argument length (all systems): 4096
> Maximum length of command we could actually use: 2088070
> Size of command buffer we are actually using: 131072
> Maximum parallelism (--max-procs must be no greater): 2147483647

If you care about parallelism, the checkpatch should be doing it itself.

Currently it's one perl interpreter with a single queue vs. zillions of perl
invocations (with all downsides of this). And I'm not sure which one is a
winning.

And yes, I'm aware of why xargs is ever exists (including handling of the
special file names).

> > Seems like we have a shell lesson :-)
> 
> Maybe so.
> 
> Using xargs allows use of -P to invoke parallelism.
> Or you could just use gnu parallel instead of xargs.
> 
> Using './scripts/checkpatch.pl -- $(git ls-files <path>)' will only
> allow a single process to be invoked for the files to be scanned.

-- 
With Best Regards,
Andy Shevchenko