netdev - Re: [PATCH] softirq: let ksoftirqd do its job

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160901153042.465b864e@redhat.com>
Date:   Thu, 1 Sep 2016 15:30:42 +0200
From:   Jesper Dangaard Brouer <brouer@...hat.com>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     Eric Dumazet <eric.dumazet@...il.com>,
        David Miller <davem@...emloft.net>,
        Rik van Riel <riel@...hat.com>,
        Paolo Abeni <pabeni@...hat.com>,
        Hannes Frederic Sowa <hannes@...hat.com>,
        linux-kernel <linux-kernel@...r.kernel.org>,
        netdev <netdev@...r.kernel.org>,
        Jonathan Corbet <corbet@....net>, brouer@...hat.com
Subject: Re: [PATCH] softirq: let ksoftirqd do its job

On Thu, 1 Sep 2016 14:48:39 +0200
Peter Zijlstra <peterz@...radead.org> wrote:

> On Thu, Sep 01, 2016 at 02:38:59PM +0200, Jesper Dangaard Brouer wrote:
> > On Thu, 1 Sep 2016 14:29:25 +0200
> > Jesper Dangaard Brouer <brouer@...hat.com> wrote:
> >   
> > > On Thu, 1 Sep 2016 13:53:56 +0200
> > > Peter Zijlstra <peterz@...radead.org> wrote:
> > >   
> > > > On Thu, Sep 01, 2016 at 01:02:31PM +0200, Jesper Dangaard Brouer wrote:    
> > > > >    PID  S  %CPU     TIME+  COMMAND
> > > > >      3  R  50.0  29:02.23  ksoftirqd/0
> > > > >  10881  R  10.7   1:01.61  udp_sink
> > > > >  10837  R  10.0   1:05.20  udp_sink
> > > > >  10852  S  10.0   1:01.78  udp_sink
> > > > >  10862  R  10.0   1:05.19  udp_sink
> > > > >  10844  S   9.7   1:01.91  udp_sink
> > > > > 
> > > > > This is strange, why is ksoftirqd/0 getting 50% of the CPU time???      
> > > > 
> > > > Do you run your udp_sink thingy in a cpu-cgroup?    
> > > 
> > > That was also Paolo's feedback (IRC).  I'm not aware of it, but it
> > > might be some distribution (Fedora 22) default thing.  
> > 
> > Correction, on the server-under-test, I'm actually running RHEL7.2
> > 
> >   
> > > How do I verify/check if I have enabled a cpu-cgroup?  
> > 
> > Hannes says I can look in "/proc/self/cgroup"
> > 
> >  $ cat /proc/self/cgroup
> >  7:net_cls:/
> >  6:blkio:/
> >  5:devices:/
> >  4:perf_event:/
> >  3:cpu,cpuacct:/
> >  2:cpuset:/
> >  1:name=systemd:/user.slice/user-1000.slice/session-c1.scope
> >  
> > And that "/" indicate I've not enabled cgroups, right?  
> 
> Mostly so. I think RHEL/Fedora has SCHED_AUTOGROUP enabled, and you can
> find that through:
> 
> cat /proc/self/autogroup

$ cat /proc/self/autogroup
/autogroup-88 nice 0

> And disable with the noautogroup boot param, or:
> 
> echo 0 > /proc/sys/kernel/sched_autogroup_enabled

Looks like it is enabled on my system:

$ grep -H . /proc/sys/kernel/sched_autogroup_enabled
/proc/sys/kernel/sched_autogroup_enabled:1


> although this latter will leave the current state intact while avoiding
> creation of any further autogroups iirc.

$ sudo sh -c 'echo 0 > /proc/sys/kernel/sched_autogroup_enabled'
$ grep -H . /proc/sys/kernel/sched_autogroup_enabled
/proc/sys/kernel/sched_autogroup_enabled:0

$ sudo systemctl restart sshd

Starting new SSH login:

$ cat /proc/self/autogroup
/autogroup-153 nice 0

Hmmm, still enabled...

$ sudo systemctl stop sshd
$ sudo systemctl start sshd
$ grep -H . /proc/sys/kernel/sched_autogroup_enabled
/proc/sys/kernel/sched_autogroup_enabled:0
$ cat /proc/self/autogroup
/autogroup-158 nice 0

Still... enabled!
Hmmm.. more idea how to disable this???

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer