linux-kernel - Re: Syslets, Threadlets, generic AIO support, v6

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20070531092640.GA2504@elte.hu>
Date:	Thu, 31 May 2007 11:26:40 +0200
From:	Ingo Molnar <mingo@...e.hu>
To:	Eric Dumazet <dada1@...mosbay.com>
Cc:	Linus Torvalds <torvalds@...ux-foundation.org>,
	Davide Libenzi <davidel@...ilserver.org>,
	Ulrich Drepper <drepper@...hat.com>,
	Jeff Garzik <jeff@...zik.org>,
	Zach Brown <zach.brown@...cle.com>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	Arjan van de Ven <arjan@...radead.org>,
	Christoph Hellwig <hch@...radead.org>,
	Andrew Morton <akpm@....com.au>,
	Alan Cox <alan@...rguk.ukuu.org.uk>,
	Evgeniy Polyakov <johnpol@....mipt.ru>,
	"David S. Miller" <davem@...emloft.net>,
	Suparna Bhattacharya <suparna@...ibm.com>,
	Jens Axboe <jens.axboe@...cle.com>,
	Thomas Gleixner <tglx@...utronix.de>
Subject: Re: Syslets, Threadlets, generic AIO support, v6


* Eric Dumazet <dada1@...mosbay.com> wrote:

> > speedup: i suggested O_ANY 6 years ago as a speedup to Apache - 
> > non-linear fds are cheaper to allocate/map:
> > 
> >   http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg23820.html
> > 
> > (i definitely remember having written code for that too, but i 
> > cannot find that in the archives. hm.) In theory we could avoid 
> > _all_ fd-bitmap overhead as well and use a per-process list/pool of 
> > struct file buffers plus a maximum-fd field as the 'non-linear fd 
> > allocator' (at the price of only deallocating them at process exit 
> > time).
> 
> Only very few apps need to open more than 100.000 files.

yes. I did not list it as a primary reason for private fds, it's just a 
nice side-effect. As long as the other apps are not hurt, i see no 
problem in improving the >100K open files case.

> As these files are likely sockets, O_ANY is not a solution.

why not? It would be a natural thing to extend sys_socket() with a 
'flags' parameter and pass in O_ANY (along with any other possible fd 
parameter like O_NDELAY, which could be inherited over connect()).

> A trick is to try to keep first 64 handles freed, so that kernel wont 
> consume too much cpu time and cache in get_unused_fd()
> 
> http://lkml.org/lkml/2005/9/15/307

this is basically a user-space front-end cache to fd allocation - which 
duplicates data needlessly. I dont see any problem with doing this in 
the kernel. (Also, obviously 'first 64 handles' could easily break with 
certain types of apps so glibc cannot do this.)

	Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/