netdev - Re: reproducable panic eviction work queue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <55ACFF62.6000006@cumulusnetworks.com>
Date:	Mon, 20 Jul 2015 16:02:10 +0200
From:	Nikolay Aleksandrov <nikolay@...ulusnetworks.com>
To:	Frank Schreuder <fschreuder@...nsip.nl>,
	Johan Schuijt <johan@...nsip.nl>
CC:	Eric Dumazet <eric.dumazet@...il.com>,
	"nikolay@...hat.com" <nikolay@...hat.com>,
	"davem@...emloft.net" <davem@...emloft.net>,
	"fw@...len.de" <fw@...len.de>,
	"chutzpah@...too.org" <chutzpah@...too.org>,
	Robin Geuze <robing@...nsip.nl>,
	netdev <netdev@...r.kernel.org>
Subject: Re: reproducable panic eviction work queue

On 07/20/2015 02:47 PM, Frank Schreuder wrote:
> 
> On 7/18/2015  05:32 PM, Nikolay Aleksandrov wrote:
>> On 07/18/2015 05:28 PM, Johan Schuijt wrote:
>>> Thx for your looking into this!
>>>
>>>> Thank you for the report, I will try to reproduce this locally
>>>> Could you please post the full crash log ?
>>> Of course, please see attached file.
>>>
>>>> Also could you test
>>>> with a clean current kernel from Linus' tree or Dave's -net ?
>>> Will do.
>>>
>>>> These are available at:
>>>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
>>>> git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
>>>> respectively.
>>>>
>>>> One last question how many IRQs do you pin i.e. how many cores
>>>> do you actively use for receive ?
>>> This varies a bit across our systems, but we’ve managed to reproduce this with IRQs pinned on as many as 2,4,8 or 20 cores.
>>>
>>> I won’t have access to our test-setup till Monday again, so I’ll be testing 3 scenario’s then:
>>> - Your patch
>> -----
>>> - Linux tree
>>> - Dave’s -net tree
>> Just one of these two would be enough. I couldn't reproduce it here but
>> I don't have as many machines to test right now and had to improvise with VMs. :-)
>>
>>> I’ll make sure to keep you posted on all the results then. We have a kernel dump of the panic, so if you need me to extract any data from there just let me know! (Some instructions might be needed)
>>>
>>> - Johan
>>>
>> Great, thank you!
>>
> I'm able to reproduce this panic on the following kernel builds:
> - 3.18.7
> - 3.18.18
> - 3.18.18 + patch from Nikolay Aleksandrov
> - 4.1.0
> 
> Would you happen to have any more suggestions we can try?
> 
> Thanks,
> Frank
> 

Unfortunately I was wrong about my theory because I mixed qp and qp_in, the new frag
doesn't make the chainlist if that codepath is hit so it couldn't mix the flags.
I'm still trying (unsuccessfully) to reproduce this, I've tried with up to 4 cores
and 4 different pinned irqs but no luck so far.
Anyway, I'll keep looking into this and will let you know if I get anywhere.

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html