linux-kernel - Re: [PACTH v2 0/3] Implement /proc/<pid>/totmaps

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160822141241.GA3464@blaptop>
Date:   Mon, 22 Aug 2016 23:12:41 +0900
From:   Minchan Kim <minchan@...nel.org>
To:     Michal Hocko <mhocko@...nel.org>
Cc:     Minchan Kim <minchan@...nel.org>,
        Sonny Rao <sonnyrao@...omium.org>, Jann Horn <jann@...jh.net>,
        Robert Foss <robert.foss@...labora.com>, corbet@....net,
        Andrew Morton <akpm@...ux-foundation.org>,
        Vlastimil Babka <vbabka@...e.cz>,
        Konstantin Khlebnikov <koct9i@...il.com>,
        Hugh Dickins <hughd@...gle.com>,
        Naoya Horiguchi <n-horiguchi@...jp.nec.com>,
        John Stultz <john.stultz@...aro.org>,
        ross.zwisler@...ux.intel.com, jmarchan@...hat.com,
        Johannes Weiner <hannes@...xchg.org>,
        Kees Cook <keescook@...omium.org>,
        Al Viro <viro@...iv.linux.org.uk>,
        Cyrill Gorcunov <gorcunov@...nvz.org>,
        Robin Humble <plaguedbypenguins@...il.com>,
        David Rientjes <rientjes@...gle.com>,
        eric.engestrom@...tec.com, Janis Danisevskis <jdanis@...gle.com>,
        calvinowens@...com, Alexey Dobriyan <adobriyan@...il.com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        ldufour@...ux.vnet.ibm.com, linux-doc@...r.kernel.org,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        Ben Zhang <benzh@...omium.org>,
        Bryan Freed <bfreed@...omium.org>,
        Filipe Brandenburger <filbranden@...omium.org>,
        Mateusz Guzik <mguzik@...hat.com>
Subject: Re: [PACTH v2 0/3] Implement /proc/<pid>/totmaps

On Mon, Aug 22, 2016 at 09:40:52AM +0200, Michal Hocko wrote:
> On Mon 22-08-16 09:07:45, Minchan Kim wrote:
> [...]
> > #!/bin/sh
> > ./smap_test &
> > pid=$!
> > 
> > for i in $(seq 25)
> > do
> >         awk '/^Rss/{rss+=$2} /^Pss/{pss+=$2} END {}' \
> >          /proc/$pid/smaps
> > done
> > kill $pid
> > 
> > root@...x:/home/barrios/test/smap# time ./s.sh 
> > pid:21973
> > 
> > real    0m17.812s
> > user    0m12.612s
> > sys     0m5.187s
> 
> retested on the bare metal (x86_64 - 2CPUs)
>         Command being timed: "sh s.sh"
>         User time (seconds): 0.00
>         System time (seconds): 18.08
>         Percent of CPU this job got: 98%
>         Elapsed (wall clock) time (h:mm:ss or m:ss): 0:18.29
> 
> multiple runs are quite consistent in those numbers. I am running with
> $ awk --version
> GNU Awk 4.1.3, API: 1.1 (GNU MPFR 3.1.4, GNU MP 6.1.0)
> 
> > > like a problem we are not able to address. And I would even argue that
> > > we want to address it in a generic way as much as possible.
> > 
> > Sure. What solution do you think as generic way?
> 
> either optimize seq_printf or replace it with something faster.

If it's real culprit, I agree. However, I tested your test program on
my 2 x86 machines and my friend's machine.

Ubuntu, Fedora, Arch

They have awk 4.0.1 and 4.1.3.

Result are same. Userspace speand more times I mentioned.

[root@...ptop smap_test]# time awk '/^Rss/{rss+=$2} /^Pss/{pss+=$2} END {printf "rss:%d pss:%d\n", rss, pss}' /proc/3552/smaps
rss:263484 pss:262188

real    0m0.770s
user    0m0.574s
sys     0m0.197s

I will attach my test progrma source.
I hope you guys test and repost the result because it's the key for direction
of patchset.

Thanks.

View attachment "smap_test.c" of type "text/plain" (317 bytes)