linux-ext4 - Re: [PATCH] mm, oom: report compaction/migration stats for higher order requests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <201608182049.42261.a.miskiewicz@gmail.com>
Date:   Thu, 18 Aug 2016 20:49:42 +0200
From:   Arkadiusz Miskiewicz <a.miskiewicz@...il.com>
To:     Michal Hocko <mhocko@...nel.org>
Cc:     linux-ext4@...r.kernel.org, linux-mm@...ck.org,
        Jan Kara <jack@...e.cz>
Subject: Re: [PATCH] mm, oom: report compaction/migration stats for higher order requests

On Wednesday 17 of August 2016, Michal Hocko wrote:
> On Wed 17-08-16 10:34:54, Arkadiusz Miśkiewicz wrote:
> [...]
> 
> > With "[PATCH] mm, oom: report compaction/migration stats for higher order
> > requests" patch:
> > https://ixion.pld-linux.org/~arekm/p2/ext4/log-20160817.txt
> > 
> > Didn't count much - all counters are 0
> > compaction_stall:0 compaction_fail:0 compact_migrate_scanned:0
> > compact_free_scanned:0 compact_isolated:0 pgmigrate_success:0
> > pgmigrate_fail:0
> 
> Dohh, COMPACTION counters are events and those are different than other
> counters we have. They only have per-cpu representation and so we would
> have to do
> +       for_each_online_cpu(cpu) {
> +               struct vm_event_state *this = &per_cpu(vm_event_states,
> cpu); +               ret += this->event[item];
> +       }
> 
> which is really nasty because, strictly speaking, we would have to do
> {get,put}_online_cpus around that loop and that uses locking and we do
> not want to possibly block in this path just because something is in the
> middle of the hotplug. So let's scratch that patch for now and sorry I
> haven't realized that earlier.
> 
> > two processes were killed by OOM (rm and cp), the rest of rm/cp didn't
> > finish
> > 
> > and I'm interrupting it to try that next patch:
> > > Could you try to test with
> > > patch from
> > > http://lkml.kernel.org/r/20160816031222.GC16913@js1304-P5Q-DELUXE
> > > please? Ideally on top of linux-next. You can add both the compaction
> > > counters patch in the oom report and high order atomic reserves patch
> > > on top.
> > 
> > Uhm, was going to use it on top of 4.7.[01] first.
> 
> OK

So with  http://lkml.kernel.org/r/20160816031222.GC16913@js1304-P5Q-DELUXE
OOM no longer happens (all 10x rm/cp processes finished).

https://ixion.pld-linux.org/~arekm/p2/ext4/log-20160818.txt

On Wednesday 17 of August 2016, Jan Kara wrote:
> Just one more debug idea to add on top of what Michal said: Can you enable
> mm_shrink_slab_start and mm_shrink_slab_end tracepoints (via
> /sys/kernel/debug/tracing/events/vmscan/mm_shrink_slab_{start,end}/enable)
> and gather output from /sys/kernel/debug/tracing/trace_pipe while the copy
> is running?

Here it is:

https://ixion.pld-linux.org/~arekm/p2/ext4/log-trace_pipe-20160818.txt.gz

-- 
Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html