linux-kernel - Re: [GIT PULL for v5.8-rc1] media updates

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200604081302.1a36f358@coco.lan>
Date:   Thu, 4 Jun 2020 08:13:02 +0200
From:   Mauro Carvalho Chehab <mchehab+huawei@...nel.org>
To:     Linus Torvalds <torvalds@...ux-foundation.org>
Cc:     Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Linux Media Mailing List <linux-media@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [GIT PULL for v5.8-rc1] media updates

Em Wed, 3 Jun 2020 21:13:21 -0700
Linus Torvalds <torvalds@...ux-foundation.org> escreveu:

> On Wed, Jun 3, 2020 at 1:06 AM Mauro Carvalho Chehab
> <mchehab+huawei@...nel.org> wrote:
> >
> > PS.: The diffstat is so big that I almost dropped it, as it is almost
> > useless for humans to read. I ended by not doing it just because perhaps
> > you could be using some sort of script to check diffstat.  
> 
> No, but I do compare the basics, and you don't have to more than scan
> it to see that "ok, it only touches area xyz".

Ok.

> And it turns out that it is huge for you partly because you have the
> default (fairly low) git rename detection limits, in order to avoid
> using a lot of CPU or memory for rename detection.
> 
> So you get:
> 
> >  2181 files changed, 260633 insertions(+), 106012 deletions(-)  
> 
> while I get
> 
>  1698 files changed, 161922 insertions(+), 7301 deletions(-)
> 
> which is a noticeable difference. Still a big diffstat, but quite a
> bit smaller than yours.
> 
> You also get a _lot_ more noise in the form of "create mode xyz" and
> "delete mode abc" notices, while for me a lot of them are just "rename
> abc => xyz". So there's a double whammy for you.
> 
> The reason is that your diff only has renames for the 100% matches like this:
> 
> >  rename Documentation/{media/v4l-drivers => admin-guide/media}/au0828-cardlist.rst (100%)  
> 
> which git can detect purely by seeing "oh, same exact SHA1".
> 
> But you don't have any non-100% renames.
> 
> In contrast, the diffstat I see also has the inexact renames like
> 
>  rename Documentation/{media/v4l-drivers =>
> admin-guide/media}/bttv-cardlist.rst (99%)
>  rename Documentation/{media/v4l-drivers => admin-guide/media}/bttv.rst (79%)
> 
> because I have done
> 
>    git config diff.renamelimit 0
> 
> to make the rename detection limit be infinite (alternatively, just
> edit your ~/.gitconfig file manually - it's often easier than
> remembering what the "git config" syntax is).
> 
> You want to see
> 
>   [diff]
>         renamelimit = 0
> 
> in your ~/.gitconfig file (or, alternatively, if you want the setting
> to be per-repo, in your .git/config file in your repository).
> 
> The default git limits for "should I spend CPU time and memory on
> detecting inexact renames" are fairly low, because people use git on
> fairly low-end machines.

I'm using renamelimit = 0 on one of my trees (the one I'm using for
the ReST conversion), and I even use -M1 there when sending patches
to docs (as some of the conversions require lots of indentation 
changes, for example, on files with lots of ascii artwork), but on
my merge tree, I was using some limit, as it is not common to have
this huge amount of changes.

> I bet your development machine isn't some kind of low-end toy, and
> rename detection is not _that_ expensive.

Probably not as nice as yours, but it is a comfortable machine,
with 32 GB RAM, an i7-8705G CPU and a fast SSD disk. 

Changing it to unlimited limit costed almost nothing here:

	real	0m1,210s
	user	0m1,009s
	sys	0m0,190s

I'll use this from now on. Thanks for the tip!

Thanks,
Mauro