Message-ID: <560A7CAB.6060105@gmail.com>
Date:	Tue, 29 Sep 2015 07:57:31 -0400
From:	Austin S Hemmelgarn <ahferroin7@...il.com>
To:	Theodore Ts'o <tytso@....edu>, Jeff Epler <jepler@...ythonic.net>,
	Andi Kleen <andi@...stfloor.org>, linux-kernel@...r.kernel.org,
	kirill.shutemov@...ux.intel.com, herbert@...dor.apana.org.au,
	Andi Kleen <ak@...ux.intel.com>
Subject: Re: [PATCH 1/3] Make /dev/urandom scalable

On 2015-09-25 15:07, Austin S Hemmelgarn wrote:
> On 2015-09-25 07:41, Austin S Hemmelgarn wrote:
>> On 2015-09-24 16:14, Theodore Ts'o wrote:
>>> On Thu, Sep 24, 2015 at 03:11:23PM -0400, Austin S Hemmelgarn wrote:
>>>>> That is a startling result.  Please say what architecture, kernel
>>>>> version, dieharder version and commandline arguments you are using to
>>>>> get 10% WEAK or FAILED assessments from dieharder on /dev/urandom.
>>>>
>>>> I do not remember what exact dieharder version or command-line
>>>> arguments
>>>> (this was almost a decade ago), except that I compiled it from source
>>>> myself, I do remember it was a 32-bit x86 processor (as that was
>>>> sadly all I
>>>> had to run Linux on at the time), and an early 2.6 series kernel
>>>> (which if I
>>>> remember correctly was already EOL by the time I was using it).
>>>
>>> It might have been nice if you had said this from the beginning
>>> instead of making an unqualified statement with the assumption that it
>>> was applicable to kernels likely to be used today in non-obsolete
>>> systems.  Otherwise it risks generating a click-bait article on
>>> Phoronix that would get people really worried for no good reason...
>> I sincerely apologize about this, I should have been more specific right
>> from the beginning (I need to get better about that when talking to
>> people, I'm so used to dealing with some of my friends who couldn't
>> event tell you the difference between RAM and a hard drive, think a bus
>> is only something you use for transportation, and get confused when I
>> try to properly explain even relatively simple CS and statistics
>> concepts).
>>>
>>> There was a bug a long, long time ago (which where we weren't doing
>>> sufficient locking and if two processes raced reading from
>>> /dev/urandom at the same time, it was possible that the two processes
>>> would get the same value read out from /dev/urandom).  This was fixed
>>> a long time ago, though, and in fact the scalability problem which
>>> Andi is trying to fix was caused by that extra locking that was
>>> added.  :-)
>>>
>>> It's possible that is what you saw.  I don't know, since there was no
>>> reproduction information to back up your rather startling claim.
>> I don't think this was what I hit, I'm pretty sure I had serialized the
>> dieharder runs.
>>>
>>> If you can reproduce consistent Dieharder failures, please do let us
>>> know with detailed reproduction instructures.
>> Will do.
> OK, just started a couple of runs in parallel using different generators
> using the following command line:
> dieharder -a -m 32 -k 1 -Y 1 -g XXX
> with one each for:
> /dev/urandom (502)
> AES_OFB (205)
> glibc random() (039)
> mt19937 (013)
> The above command line will run all dieharder tests with 12800 psamples,
> using a higher than default precision, and re-running tests that return
> WEAK until it gets a PASS or FAIL.  Even on the relatively fast (at
> least, fast for a desktop) system I'm running them on, I expect it will
> take quite some time to finish (although regardless of that I'm probably
> not going to be getting back to it until Monday).
>
> Interestingly, based on what dieharder is already saying about
> performance, /dev/urandom is slower than AES_OFB (at least, on this
> particular system, happy to provide hardware specs if someone wants).
>
Apologies for not replying yesterday like I said I would.

I actually didn't get a chance to run the tests to completion as the 
wifi card in the system I was running the tests on lost it's mind about 
55 hours in and I had to cold reboot the system to reset it.  I would 
give the results here, except that I have a feeling that people probably 
don't want 110kb of data in the e-mail body, and thunderbird is for some 
reason choking on trying to attach files.  In general, the results were 
pretty typical of a good PRNG, performance differences not withstanding. 
  In other words, don't use /dev/urandom except for seeding other 
PRNG's, but because of the speed, not the quality.


Download attachment "smime.p7s" of type "application/pkcs7-signature" (3019 bytes)

Powered by blists - more mailing lists