lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <Pine.LNX.4.64.0702231206290.31281@alien.or.mcafeemobile.com>
Date:	Fri, 23 Feb 2007 12:43:54 -0800 (PST)
From:	Davide Libenzi <davidel@...ilserver.org>
To:	Evgeniy Polyakov <johnpol@....mipt.ru>
cc:	Ingo Molnar <mingo@...e.hu>, Ulrich Drepper <drepper@...hat.com>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Arjan van de Ven <arjan@...radead.org>,
	Christoph Hellwig <hch@...radead.org>,
	Andrew Morton <akpm@....com.au>,
	Alan Cox <alan@...rguk.ukuu.org.uk>,
	Zach Brown <zach.brown@...cle.com>,
	"David S. Miller" <davem@...emloft.net>,
	Suparna Bhattacharya <suparna@...ibm.com>,
	Jens Axboe <jens.axboe@...cle.com>,
	Thomas Gleixner <tglx@...utronix.de>
Subject: Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

On Fri, 23 Feb 2007, Evgeniy Polyakov wrote:

> I was not clear - I meant why do we need to do that when we can run the
> same code in userspace? And better if we can have non-blocking dataflows
> and number of threads equal to number of processors...

I've a userspace library that does exactly that (GUASI - GPL code avail if 
you want, but no man page written yet). It uses a pool of threads and 
queue requests. I've a bench that crawl through a directory a read files. 
The sync against async performance sucks. You can't do the cachehit 
optimization in userspace. With network stuff could prolly do better 
(since network is more heavily towards async), but still. 




> I started a week of writing without russian-english dictionary, so
> expect some troubles in communications with me :)
> 
> I said that about kernel design - when we have thousand(s) of threads,
> which do the work - if number of context switches is small (i.e. when
> operations mostly do not block), then it is ok (although 'ps' output 
> with threads can scary a grandma).
> It is also ok to say - 'hey, Linux has so easy AIO model, so that
> everyone should switch and start using it and do not care about problems 
> associated with multi-threaded programming with high concurrency', 
> but, in my opinion, both that cases can not cover all (and most of) 
> the usage cases.
> 
> To eat my hat (or force others to do the same) I'm preparing a tree for 
> threadlet test - I plan to write a trivial web server
> (accept/recv/send/sendfile in one threadlet function) and give it a try
> soon.

Funny, I lazily started doing the same thing last weekend (than I had to 
stop, since real job kicked in ;). I wanted to compare a fully MT trivial 
HTTP server:

http://www.xmailserver.org/thrhttp.c

with one that is event-driven (epoll) and coroutine based. This one will 
only be compared for memory-content delivery, since it has no async vfs 
capabilities. They both support the special "/mem-XXXX" url, that allows 
an HTTP loader to request a given content size.
I also have a epoll+coroutine HTTP loader (that works around httperf 
limitations).
Then, I wanted to compare the above, with one that is epoll+GUASI+coroutine 
based (basically a userpace-only thingy).
I've the code for all the above.
Finally, with one that is epoll+syslet+coroutine based (no code for this 
yet - but it should be a easy port from the GUASI one).
Keep in mind though, that a threadlet solution doing accept/recv/send/sendfile
is becoming blazingly similar to a full MT solution.
I can only immagine the thunders and flames that Larry would throw at us 
for using all those threads :D




- Davide


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ