Message-ID: <7b94cc3f.4acf.15c32faad0b.Coremail.zhanglin496@163.com>
Date:   Tue, 23 May 2017 09:45:08 +0800 (CST)
From:   zhanglin496 <zhanglin496@....com>
To:     stefan@....samsung.com
Cc:     aar@...gutronix.de, davem@...emloft.net,
        linux-wpan@...r.kernel.org, netdev@...r.kernel.org,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH] net: ieee802154: fix net_device reference release too
 early

Hello.

Sorry too late to reply.
>
> Hello.
>
> On Thu, 2017-05-18 at 15:14, Stefan Schmidt wrote:
> > Hello.
> >
> > On Thu, 2017-05-18 at 15:50, linzhang wrote:
> > > This patch fixes the kernel oops when release net_device reference in
> > > advance. In function raw_sendmsg(i think the dgram_sendmsg has the same
> > > problem), there is a race condition between dev_put and dev_queue_xmit
> > > when the device is gong that maybe lead to dev_queue_ximt to see
> > > an illegal net_device pointer.
> > >
> >
> > You have a test case to reproduce this oops? I fear I have not seen
> > one.
>
> If you have a test case handy adding it to the commit would be handy. If you do
> not have one around we can do without.
>

My test kernel is 3.13.0-32.
Becasue i am not have a real 802154 device, so i change lowpan_newlink
function to this:

        /* find and hold real wpan device */
        real_dev = dev_get_by_index(src_net, nla_get_u32(tb[IFLA_LINK]));
        if (!real_dev)
                return -ENODEV;
//      if (real_dev->type != ARPHRD_IEEE802154) {
//              dev_put(real_dev);
//              return -EINVAL;
//      }
        lowpan_dev_info(dev)->real_dev = real_dev;
        lowpan_dev_info(dev)->fragment_tag = 0;
        mutex_init(&lowpan_dev_info(dev)->dev_list_mtx);

Also, in order to simulate preempt, i change the raw_sendmsg function to this:

        skb->dev = dev;
        skb->sk  = sk;
        skb->protocol = htons(ETH_P_IEEE802154);
        dev_put(dev);
        //simulate preempt
        schedule_timeout_uninterruptible(30 * HZ);
        err = dev_queue_xmit(skb);
        if (err > 0)
                err = net_xmit_errno(err);

and this is my userspace test code named test_send_data:

#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
int main(int argc, char **argv)
{
        char buf[127];
        int sockfd;
        sockfd = socket(AF_IEEE802154, SOCK_RAW, 0);
        if (sockfd < 0) {
                printf("create sockfd error: %s\n", strerror(errno));
                return -1;
        }
        send(sockfd, buf, sizeof(buf), 0);
        return 0;
}

This is my test case:
root@...nglin-x-computer:~/develop/802154# uname -a
Linux zhanglin-x-computer 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15
03:51:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
root@...nglin-x-computer:~/develop/802154# ip link add link eth0 name
lowpan0 type lowpan
root@...nglin-x-computer:~/develop/802154#
//keep the lowpan0 device down
root@...nglin-x-computer:~/develop/802154# ./test_send_data &
//wait a while
root@...nglin-x-computer:~/develop/802154# ip link del link dev lowpan0
//the device is gone
//oops
[381.303307] general protection fault: 0000 [#1]SMP
[381.303407] Modules linked in: af_802154 6lowpan bnep rfcomm
bluetooth nls_iso8859_1 snd_hda_codec_hdmi snd_hda_codec_realtek
rts5139(C) snd_hda_intel
snd_had_codec snd_hwdep snd_pcm snd_page_alloc snd_seq_midi
snd_seq_midi_event snd_rawmidi snd_req intel_rapl snd_seq_device
coretemp i915 kvm_intel
kvm snd_timer snd crct10dif_pclmul crc32_pclmul ghash_clmulni_intel
cypted drm_kms_helper drm i2c_algo_bit soundcore video mac_hid
parport_pc ppdev ip parport hid_generic
usbhid hid ahci r8169 mii libahdi
[381.304286] CPU:1 PID: 2524 Commm: 1 Tainted: G C 0 3.13.0-32-generic
#57-Ubuntu
[381.304409] Hardware name: Haier Haier DT Computer/Haier DT Codputer,
BIOS FIBT19H02_X64 06/09/2014
[381.304546] tasks: ffff000096965fc0 ti: ffffB0013779c000 task.ti:
ffffB8013779c000
[381.304659] RIP: 0010:[<ffffffff01621fe1>] [<ffffffff81621fe1>]
__dev_queue_ximt+0x61/0x500
[381.304798] RSP: 0018:ffffB8013779dca0 EFLAGS: 00010202
[381.304880] RAX: 272b031d57565351 RBX: 0000000000000000 RCX: ffff8800968f1a00
[381.304987] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8800968f1a00
[381.305095] RBP: ffff8e013773dce0 R08: 0000000000000266 R09: 0000000000000004
[381.305202] R10: 0000000000000004 R11: 0000000000000005 R12: ffff88013902e000
[381.305310] R13: 000000000000007f R14: 000000000000007f R15: ffff8800968f1a00
[381.305418] FS:  00007fc57f50f740(0000) GS: ffff88013fc80000(0000)
knlGS: 0000000000000000
[381.305540] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[381.305627] CR2: 00007fad0841c000 CR3: 00000001368dd000 CR4: 00000000001007e0
[361.905734] Stack:
[381.305768]  00000000002052d0 000000003facb30a ffff88013779dcc0
ffff880137764000
[381.305898]  ffff88013779de70 000000000000007f 000000000000007f
ffff88013902e000
[381.306026]  ffff88013779dcf0 ffffffff81622490 ffff88013779dd39
ffffffffa03af9f1
[381.306155] Call Trace:
[381.306202]  [<ffffffff81622490>] dev_queue_xmit+0x10/0x20
[381.306294]  [<ffffffffa03af9f1>] raw_sendmsg+0x1b1/0x270 [af_802154]
[381.306396]  [<ffffffffa03af054>] ieee802154_sock_sendmsg+0x14/0x20 [af_802154]
[381.306512]  [<ffffffff816079eb>] sock_sendmsg+0x8b/0xc0
[381.306600]  [<ffffffff811d52a5>] ? __d_alloc+0x25/0x180
[381.306687]  [<ffffffff811a1f56>] ? kmem_cache_alloc_trace+0x1c6/0x1f0
[381.306791]  [<ffffffff81607b91>] SYSC_sendto+0x121/0x1c0
[381.306878]  [<ffffffff8109ddf4>] ? vtime_account_user+x54/0x60
[381.306975]  [<ffffffff81020d45>] ? syscall_trace_enter+0x145/0x250
[381.307073]  [<ffffffff816086ae>] SyS_sendto+0xe/0x10
[381.307156]  [<ffffffff8172c87f>] tracesys+0xe1/0xe6
[381.307233] Code: c6 a1 a4 ff 41 8b 57 78 49 8b 47 20 85 d2 48 8b 80
78 07 00 00 75 21 49 8b 57 18 48 85 d2 74 18 48 85 c0 74 13 8b 92 ac
01 00 00 <3b> 50 10 73 08 8b 44 90 14 41 89 47 78 41 f6 84 24 d5 00 00
00
[381.307801] RIP [<ffffffff81621fe1>] _dev_queue_xmit+0x61/0x500
[381.307901]  RSP <ffff88013779dca0>
[381.347512] Kernel panic - not syncing: Fatal exception in interrupt
[381.347747] drm_kms_helper: panic occurred, switching back to text console


> > > So i think that dev_put should be behind of the dev_queue_xmit.
> > >
> > > Also, explicit set skb->sk is needless, sock_alloc_send_skb is
> > > already set it.
> >
> > You could have put this fixup in a different patch.
>

Thanks, i will repost it.

> I actually would request you to split this into two patches. One for the
> removal of the sk setting and one for the race condition fix.
>
> > > Signed-off-by: linzhang <xiaolou4617@...il.com>
> >
> > This looks more like a username instead of a real name. If you have Lin
> > Zhang as you English real name that would be better here. :)
>
> This would be also appreciated.

Yes, my real name is Lin Zhang, sorry to make you confusion.

> > > ---
> > >  net/ieee802154/socket.c | 10 ++++------
> > >  1 file changed, 4 insertions(+), 6 deletions(-)
> > >
> > > diff --git a/net/ieee802154/socket.c b/net/ieee802154/socket.c
> > > index eedba76..a60658c 100644
> > > --- a/net/ieee802154/socket.c
> > > +++ b/net/ieee802154/socket.c
> > > @@ -301,15 +301,14 @@ static int raw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size)
> > >             goto out_skb;
> > >
> > >     skb->dev = dev;
> > > -   skb->sk  = sk;
> > >     skb->protocol = htons(ETH_P_IEEE802154);
> > >
> > > -   dev_put(dev);
> > > -
> > >     err = dev_queue_xmit(skb);
> > >     if (err > 0)
> > >             err = net_xmit_errno(err);
> > >
> > > +   dev_put(dev);
> > > +
> > >     return err ?: size;
> > >
> > >  out_skb:
> > > @@ -690,15 +689,14 @@ static int dgram_sendmsg(struct sock *sk, struct msghdr *msg, size_t size)
> > >             goto out_skb;
> > >
> > >     skb->dev = dev;
> > > -   skb->sk  = sk;
> > >     skb->protocol = htons(ETH_P_IEEE802154);
> > >
> > > -   dev_put(dev);
> > > -
> > >     err = dev_queue_xmit(skb);
> > >     if (err > 0)
> > >             err = net_xmit_errno(err);
> > >
> > > +   dev_put(dev);
> > > +
> > >     return err ?: size;
> >
> > Going to give this a test ride here now.
>
> I gave it a ride in my testbed and I encountered no problems. While I have never
> seen the race and oops myself doing the dev_put before the xmit can surely lead to
> such a race and the fix is valid.
>
> Once you have done the changes requested above and re-submit your two patches you can
> add my
>
> Acked-by: Stefan Schmidt <stefan@....samsung.com>
>
> to both of them.
>
> regards
> Stefan Schmidt

In my opinion, there is always exist a chance that the device is gong
before call dev_queue_xmit.

I think the latest kernel is have the same problem. If you have a real 802154 device, 
maybe use the test case as above, thanks.

Please forgive me for my weak English.
Thanks for reviewing!
regards

Powered by blists - more mailing lists