linux-kernel - Re: [PATCH v3] Drivers: hv: vmbus: fix the race when querying & updating the percpu list

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87h9ddffov.fsf@vitty.brq.redhat.com>
Date:	Wed, 01 Jun 2016 10:58:56 +0200
From:	Vitaly Kuznetsov <vkuznets@...hat.com>
To:	Dexuan Cui <decui@...rosoft.com>
Cc:	"gregkh\@linuxfoundation.org" <gregkh@...uxfoundation.org>,
	"linux-kernel\@vger.kernel.org" <linux-kernel@...r.kernel.org>,
	"driverdev-devel\@linuxdriverproject.org" 
	<driverdev-devel@...uxdriverproject.org>,
	"olaf\@aepfle.de" <olaf@...fle.de>,
	"apw\@canonical.com" <apw@...onical.com>,
	"jasowang\@redhat.com" <jasowang@...hat.com>,
	KY Srinivasan <kys@...rosoft.com>,
	Haiyang Zhang <haiyangz@...rosoft.com>,
	"rolf.neugebauer\@docker.com" <rolf.neugebauer@...ker.com>,
	"dave.scott\@docker.com" <dave.scott@...ker.com>,
	"ian.campbell\@docker.com" <ian.campbell@...ker.com>
Subject: Re: [PATCH v3] Drivers: hv: vmbus: fix the race when querying & updating the percpu list

Dexuan Cui <decui@...rosoft.com> writes:

>> From: Vitaly Kuznetsov [mailto:vkuznets@...hat.com]
>> Sent: Wednesday, June 1, 2016 0:27
>> To: Dexuan Cui <decui@...rosoft.com>
>> Cc: gregkh@...uxfoundation.org; linux-kernel@...r.kernel.org; driverdev-
>> devel@...uxdriverproject.org; olaf@...fle.de; apw@...onical.com;
>> jasowang@...hat.com; KY Srinivasan <kys@...rosoft.com>; Haiyang
>> Zhang <haiyangz@...rosoft.com>; rolf.neugebauer@...ker.com;
>> dave.scott@...ker.com; ian.campbell@...ker.com
>> Subject: Re: [PATCH v3] Drivers: hv: vmbus: fix the race when querying &
>> updating the percpu list
>> 
>> Dexuan Cui <decui@...rosoft.com> writes:
>> 
>> > There is a rare race when we remove an entry from the global list
>> > hv_context.percpu_list[cpu] in hv_process_channel_removal() ->
>> > percpu_channel_deq() -> list_del(): at this time, if vmbus_on_event() ->
>> > process_chn_event() -> pcpu_relid2channel() is trying to query the list,
>> > we can get the kernel fault.
>> >
>> > Similarly, we also have the issue in the code path: vmbus_process_offer()
>> ->
>> > percpu_channel_enq().
>> >
>> > We can resolve the issue by disabling the tasklet when updating the list.
>> >
>> > The patch also moves vmbus_release_relid() to a later place where
>> > the channel has been removed from the per-cpu and the global lists.
>> >
>> > Reported-by: Rolf Neugebauer <rolf.neugebauer@...ker.com>
>> > Cc: Vitaly Kuznetsov <vkuznets@...hat.com>
>> > Signed-off-by: Dexuan Cui <decui@...rosoft.com>
>> 
>> Tested 4.7-rc1 with this path applied and kernel always crashes on boot
>> (WS2016TP5, 12 CPU SMP guest, Generation 2):
>> 
>> [    5.464251] hv_vmbus: Hyper-V Host Build:14300-10.0-1-0.1006; Vmbus
>> version:4.0
>> [    5.471666] hv_vmbus: Unknown GUID: f8e65716-3cb3-4a06-9a60-
>> 1889c5cccab5
>> [    5.472143] BUG: unable to handle kernel paging request at
>> 000000079fff5288
>> [    5.477107] IP: [<ffffffffa0004b91>] vmbus_onoffer+0x311/0x570
>> [hv_vmbus]
>> ...
>>   Vitaly
>
> I can't reproduce the panic somehow, but I did find a bug in vmbus_process_offer():
>
> "hv_event_tasklet_disable(channel) and hv_event_tasklet_enable(channel)"
> are buggy: the 'channel' parameter should be 'newchannel'.
>
> This was a copy-and-paste bug... Sorry!
> Can you fix this and see if the panic will disappear in your side?

This fixes the issue I'm seeing, thanks!

-- 
  Vitaly