netdev - Re: [Patch net v2] cgroup: fix cgroup_sk_alloc() for sk_clone

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAM_iQpWWXmrNzNZc5-N=bXo2_o58V0=3SeFkPzmJaDL3TVUeEA@mail.gmail.com>
Date:   Wed, 8 Jul 2020 12:32:36 -0700
From:   Cong Wang <xiyou.wangcong@...il.com>
To:     Guenter Roeck <linux@...ck-us.net>
Cc:     Linux Kernel Network Developers <netdev@...r.kernel.org>,
        Cameron Berkenpas <cam@...-zeon.de>,
        Peter Geis <pgwipeout@...il.com>,
        Lu Fengqi <lufq.fnst@...fujitsu.com>,
        Daniël Sonck <dsonck92@...il.com>,
        Zhang Qiang <qiang.zhang@...driver.com>,
        Thomas Lamprecht <t.lamprecht@...xmox.com>,
        Daniel Borkmann <daniel@...earbox.net>,
        Zefan Li <lizefan@...wei.com>, Tejun Heo <tj@...nel.org>,
        Roman Gushchin <guro@...com>
Subject: Re: [Patch net v2] cgroup: fix cgroup_sk_alloc() for sk_clone_lock()

On Wed, Jul 8, 2020 at 12:07 PM Guenter Roeck <linux@...ck-us.net> wrote:
>
> On 7/8/20 11:34 AM, Cong Wang wrote:
> > Hi,
> >
> > On Wed, Jul 8, 2020 at 8:33 AM Guenter Roeck <linux@...ck-us.net> wrote:
> >> This patch causes all my s390 boot tests to crash. Reverting it fixes
> >> the problem. Please see bisect results and and crash log below.
> >>
> > ...
> >> Crash log:
> >
> > Interesting. I don't see how unix socket is any special here, it creates
> > a peer sock with sk_alloc(), but this is not any different from two separated
> > sockets.
> >
> > What is your kernel config? Do you enable CONFIG_CGROUP_NET_PRIO
> > or CONFIG_CGROUP_NET_CLASSID? I can see there might be a problem
> > if you don't enable either of them but enable CONFIG_CGROUP_BPF.
> >
>
> cgroup specific configuration bits:
>
> CONFIG_CGROUPS=y
> CONFIG_BLK_CGROUP=y
> CONFIG_CGROUP_WRITEBACK=y
> CONFIG_CGROUP_SCHED=y
> CONFIG_CGROUP_PIDS=y
> CONFIG_CGROUP_RDMA=y
> CONFIG_CGROUP_FREEZER=y
> CONFIG_CGROUP_HUGETLB=y
> CONFIG_CGROUP_DEVICE=y
> CONFIG_CGROUP_CPUACCT=y
> CONFIG_CGROUP_PERF=y
> CONFIG_CGROUP_BPF=y
> # CONFIG_CGROUP_DEBUG is not set
> CONFIG_SOCK_CGROUP_DATA=y
> CONFIG_BLK_CGROUP_RWSTAT=y
> CONFIG_BLK_CGROUP_IOLATENCY=y
> CONFIG_BLK_CGROUP_IOCOST=y
> # CONFIG_BFQ_CGROUP_DEBUG is not set
> # CONFIG_NETFILTER_XT_MATCH_CGROUP is not set
> CONFIG_NET_CLS_CGROUP=y
> CONFIG_CGROUP_NET_PRIO=y
> CONFIG_CGROUP_NET_CLASSID=y
>
> This originates from arch/s390/configs/defconfig; I don't touch
> any cgroup specific configuration in my tests.

Good to know you enable everything related here.

>
> > And if you have the full kernel log, it would be helpful too.
> >
>
> https://kerneltests.org/builders/qemu-s390-pending-fixes/builds/222/steps/qemubuildcommand/logs/stdio

It looks like cgroup_sk_alloc_disabled is always false in your case.
This makes the bug even more weird. Unless there is a refcnt bug
prior to my commit, I don't see how it could happen.

>
> Interestingly, enabling CONFIG_BFQ_CGROUP_DEBUG makes the problem disappear.

Yeah, I guess there might be some cgroup refcnt bug which could
just paper out with CONFIG_BFQ_CGROUP_DEBUG=y.

If you can test patches, I can send you a debugging patch for you
to collect more data. I assume this is 100% reproducible on your
side?

Thanks.