linux-kernel - Re: [PACTH v2 0/3] Implement /proc/<pid>/totmaps

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160822164554.GA7043@dhcp22.suse.cz>
Date:   Mon, 22 Aug 2016 18:45:54 +0200
From:   Michal Hocko <mhocko@...nel.org>
To:     Minchan Kim <minchan@...nel.org>
Cc:     Sonny Rao <sonnyrao@...omium.org>, Jann Horn <jann@...jh.net>,
        Robert Foss <robert.foss@...labora.com>, corbet@....net,
        Andrew Morton <akpm@...ux-foundation.org>,
        Vlastimil Babka <vbabka@...e.cz>,
        Konstantin Khlebnikov <koct9i@...il.com>,
        Hugh Dickins <hughd@...gle.com>,
        Naoya Horiguchi <n-horiguchi@...jp.nec.com>,
        John Stultz <john.stultz@...aro.org>,
        ross.zwisler@...ux.intel.com, jmarchan@...hat.com,
        Johannes Weiner <hannes@...xchg.org>,
        Kees Cook <keescook@...omium.org>,
        Al Viro <viro@...iv.linux.org.uk>,
        Cyrill Gorcunov <gorcunov@...nvz.org>,
        Robin Humble <plaguedbypenguins@...il.com>,
        David Rientjes <rientjes@...gle.com>,
        eric.engestrom@...tec.com, Janis Danisevskis <jdanis@...gle.com>,
        calvinowens@...com, Alexey Dobriyan <adobriyan@...il.com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        ldufour@...ux.vnet.ibm.com, linux-doc@...r.kernel.org,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Ben Zhang <benzh@...omium.org>,
        Bryan Freed <bfreed@...omium.org>,
        Filipe Brandenburger <filbranden@...omium.org>,
        Mateusz Guzik <mguzik@...hat.com>
Subject: Re: [PACTH v2 0/3] Implement /proc/<pid>/totmaps

On Mon 22-08-16 23:12:41, Minchan Kim wrote:
> On Mon, Aug 22, 2016 at 09:40:52AM +0200, Michal Hocko wrote:
> > On Mon 22-08-16 09:07:45, Minchan Kim wrote:
> > [...]
> > > #!/bin/sh
> > > ./smap_test &
> > > pid=$!
> > > 
> > > for i in $(seq 25)
> > > do
> > >         awk '/^Rss/{rss+=$2} /^Pss/{pss+=$2} END {}' \
> > >          /proc/$pid/smaps
> > > done
> > > kill $pid
> > > 
> > > root@...x:/home/barrios/test/smap# time ./s.sh 
> > > pid:21973
> > > 
> > > real    0m17.812s
> > > user    0m12.612s
> > > sys     0m5.187s
> > 
> > retested on the bare metal (x86_64 - 2CPUs)
> >         Command being timed: "sh s.sh"
> >         User time (seconds): 0.00
> >         System time (seconds): 18.08
> >         Percent of CPU this job got: 98%
> >         Elapsed (wall clock) time (h:mm:ss or m:ss): 0:18.29
> > 
> > multiple runs are quite consistent in those numbers. I am running with
> > $ awk --version
> > GNU Awk 4.1.3, API: 1.1 (GNU MPFR 3.1.4, GNU MP 6.1.0)
> > 
> > > > like a problem we are not able to address. And I would even argue that
> > > > we want to address it in a generic way as much as possible.
> > > 
> > > Sure. What solution do you think as generic way?
> > 
> > either optimize seq_printf or replace it with something faster.
> 
> If it's real culprit, I agree. However, I tested your test program on
> my 2 x86 machines and my friend's machine.
> 
> Ubuntu, Fedora, Arch
> 
> They have awk 4.0.1 and 4.1.3.
> 
> Result are same. Userspace speand more times I mentioned.
> 
> [root@...ptop smap_test]# time awk '/^Rss/{rss+=$2} /^Pss/{pss+=$2} END {printf "rss:%d pss:%d\n", rss, pss}' /proc/3552/smaps
> rss:263484 pss:262188
> 
> real    0m0.770s
> user    0m0.574s
> sys     0m0.197s
> 
> I will attach my test progrma source.
> I hope you guys test and repost the result because it's the key for direction
> of patchset.

Hmm, this is really interesting. I have checked a different machine and
it shows different results. Same code, slightly different version of awk
(4.1.0) and the results are different
        Command being timed: "awk /^Rss/{rss+=$2} /^Pss/{pss+=$2} END {printf "rss:%d pss:%d\n", rss, pss} /proc/48925/smaps"
        User time (seconds): 0.43
        System time (seconds): 0.27

I have no idea why those numbers are so different on my laptop
yet. It surely looks suspicious. I will try to debug this further
tomorrow. Anyway, the performance is just one side of the problem. I
have tried to express my concerns about a single exported pss value in
other email. Please try to step back and think about how useful is this
information without the knowing which resource we are talking about.
-- 
Michal Hocko
SUSE Labs