netdev - Re: nfs NULL-dereferencing in net-next

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20161031152550.10acb7e0@jkicinski-Precision-T1700>
Date:   Mon, 31 Oct 2016 15:25:50 +0000
From:   Jakub Kicinski <kubakici@...pl>
To:     Anna Schumaker <Anna.Schumaker@...app.com>
Cc:     Yotam Gigi <yotamg@...lanox.com>, Andy Adamson <andros@...app.com>,
        "linux-nfs@...r.kernel.org" <linux-nfs@...r.kernel.org>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        Trond Myklebust <trond.myklebust@...marydata.com>,
        "Yotam Gigi" <yotam.gi@...il.com>, mlxsw <mlxsw@...lanox.com>
Subject: Re: nfs NULL-dereferencing in net-next

On Thu, 27 Oct 2016 06:50:22 +0000, Yotam Gigi wrote:
> >-----Original Message-----
> >From: Anna Schumaker [mailto:Anna.Schumaker@...app.com]
> >Sent: Wednesday, October 26, 2016 9:17 PM
> >To: Jakub Kicinski <kubakici@...pl>
> >Cc: Yotam Gigi <yotamg@...lanox.com>; Andy Adamson <andros@...app.com>;
> >linux-nfs@...r.kernel.org; netdev@...r.kernel.org; Trond Myklebust
> ><Trond.Myklebust@...app.com>; Yotam Gigi <yotam.gi@...il.com>; mlxsw
> ><mlxsw@...lanox.com>
> >Subject: Re: nfs NULL-dereferencing in net-next
> >
> >On 10/26/2016 02:08 PM, Jakub Kicinski wrote:  
> >> On Wed, 26 Oct 2016 16:15:24 +0000, Yotam Gigi wrote:  
> >>>> -----Original Message-----
> >>>> From: Anna Schumaker [mailto:Anna.Schumaker@...app.com]
> >>>> Sent: Wednesday, October 26, 2016 5:40 PM
> >>>> To: Yotam Gigi <yotamg@...lanox.com>; Jakub Kicinski <kubakici@...pl>;  
> >Andy  
> >>>> Adamson <andros@...app.com>; Anna Schumaker
> >>>> <Anna.Schumaker@...app.com>; linux-nfs@...r.kernel.org
> >>>> Cc: netdev@...r.kernel.org; Trond Myklebust  
> ><Trond.Myklebust@...app.com>;  
> >>>> Yotam Gigi <yotam.gi@...il.com>; mlxsw <mlxsw@...lanox.com>
> >>>> Subject: Re: nfs NULL-dereferencing in net-next
> >>>>
> >>>> On 10/25/2016 01:19 PM, Yotam Gigi wrote:  
> >>>>>  
> >>>>>> -----Original Message-----
> >>>>>> From: netdev-owner@...r.kernel.org [mailto:netdev-  
> >owner@...r.kernel.org]  
> >>>> On  
> >>>>>> Behalf Of Jakub Kicinski
> >>>>>> Sent: Monday, October 17, 2016 10:20 PM
> >>>>>> To: Andy Adamson <andros@...app.com>; Anna Schumaker
> >>>>>> <Anna.Schumaker@...app.com>; linux-nfs@...r.kernel.org
> >>>>>> Cc: netdev@...r.kernel.org; Trond Myklebust  
> >>>> <Trond.Myklebust@...app.com>  
> >>>>>> Subject: nfs NULL-dereferencing in net-next
> >>>>>>
> >>>>>> Hi!
> >>>>>>
> >>>>>> I'm hitting this reliably on net-next, HEAD at 3f3177bb680f
> >>>>>> ("fsl/fman: fix error return code in mac_probe()").  
> >>>>>
> >>>>>
> >>>>> I see the same thing. It happens constantly on some of my machines, making  
> >>>> them  
> >>>>> completely unusable.
> >>>>>
> >>>>> I bisected it and got to the commit:
> >>>>>
> >>>>> commit 04ea1b3e6d8ed4978bb608c1748530af3de8c274
> >>>>> Author: Andy Adamson <andros@...app.com>
> >>>>> Date:   Fri Sep 9 09:22:27 2016 -0400
> >>>>>
> >>>>>     NFS add xprt switch addrs test to match client
> >>>>>
> >>>>>     Signed-off-by: Andy Adamson <andros@...app.com>
> >>>>>     Signed-off-by: Anna Schumaker <Anna.Schumaker@...app.com>  
> >>>>
> >>>> Thanks for reporting on this everyone!  Does this patch help?  
> >>>
> >>> Actually, I still see the same bug with the same trace.  
> >
> >Well, it was worth a shot.  I'll keep poking at it.
> >  
> >>
> >> I rebuild the latest net-next and I'm not seeing the trace any more...
> >> I'm only seeing this (with or without your patch):
> >>
> >> [   23.465877] NFS: set_pnfs_layoutdriver: cl_exchange_flags 0x0
> >> [   23.473784] NFS: set_pnfs_layoutdriver: cl_exchange_flags 0x0
> >> [   23.588890] NFS: set_pnfs_layoutdriver: cl_exchange_flags 0x0
> >> [   23.596746] NFS: set_pnfs_layoutdriver: cl_exchange_flags 0x0
> >> [   23.781574] NFS: set_pnfs_layoutdriver: cl_exchange_flags 0x0
> >> [   23.789599] NFS: set_pnfs_layoutdriver: cl_exchange_flags 0x0  
> >
> >Interesting, I get that too when I try to use NFS v4.1.  It's weird that the crash would
> >stop happening like that, so maybe something is racy in this area.
> >
> >Thanks for testing, Yotam and Jakub!
> >Anna  
> 
> I just found out that it happens on any of my machines, once I put two nfs entries in
> my fstab. If I put only one, I don't see the problem. 
> 
> I hope it might be helpful :)

Hi Anna,

any updates on this one?  The crash came back half an hour after I
reported that it was gone...

Over the weekend David Miller rebased net-next on top of 4.9.0-rc3 and
the bug is still there :(  FWIW I also have multiple nfs mounts on my
setup, 2 in fstab and one in a startup script.  Following Yotam I
dropped one of the fstab entries and things seem to be working (even
though I still have multiple mounts, the other one just comes a bit
later).