linux-kernel - Re: [patch 05/11] syslets: core code

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 15 Feb 2007 14:34:04 -0800
From:	"Michael K. Edwards" <medwards.linux@...il.com>
To:	"Linus Torvalds" <torvalds@...ux-foundation.org>
Cc:	"Evgeniy Polyakov" <johnpol@....mipt.ru>,
	"Ingo Molnar" <mingo@...e.hu>,
	"Linux Kernel Mailing List" <linux-kernel@...r.kernel.org>,
	"Arjan van de Ven" <arjan@...radead.org>,
	"Christoph Hellwig" <hch@...radead.org>,
	"Andrew Morton" <akpm@....com.au>,
	"Alan Cox" <alan@...rguk.ukuu.org.uk>,
	"Ulrich Drepper" <drepper@...hat.com>,
	"Zach Brown" <zach.brown@...cle.com>,
	"David S. Miller" <davem@...emloft.net>,
	"Benjamin LaHaise" <bcrl@...ck.org>,
	"Suparna Bhattacharya" <suparna@...ibm.com>,
	"Davide Libenzi" <davidel@...ilserver.org>,
	"Thomas Gleixner" <tglx@...utronix.de>
Subject: Re: [patch 05/11] syslets: core code

On 2/15/07, Linus Torvalds <torvalds@...ux-foundation.org> wrote:
> Would it make the interface less cool? Yeah. Would it limit it to just a
> few linked system calls (to avoid memory allocation issues in the kernel)?
> Yes again. But it would simplify a lot of the interface issues.

Only in toy applications.  Real userspace code that lives between
networks+disks and impatient humans is 80% exception handling,
logging, and diagnostics.  If you can't do any of that between stages
of an async syscall chain, you're fscked when it comes to performance
analysis (the "which 10% of the traffic do we not abort under
pressure" kind, not the "cut overhead by 50%" kind).  Not that real
userspace code could get any simpler by using this facility anyway,
since you can't jump the queue, cancel in bulk, or add cleanup hooks.

Efficiently interleaved execution of high-latency I/O chains would be
nice.  Low overhead for cache hits would be nicer.  But least for the
workloads that interest me, neither is anywhere near as important as
the ability to say, "This 10% (or 90%) of my requests are going to
take forever?  Nevermind -- but don't cancel the 1% I can't do
without."

This is not a scheduling problem, it is a caching problem.  Caches are
data structures, not thread pools.  Assume that you need to design for
dynamic reprioritization, speculative fetch, and opportunistic flush,
even if you don't implement them at first.  Above all, stay out of the
way when a synchronous request misses cache -- and when application
code decides that a bunch of its outstanding requests are no longer
interesting, take the hint!

Oh, and while you're at it: I'd like to program AIO facilities using a
C compiler with an explicitly parallel construct -- something along
the lines of:

try (my_aio_batch, initial_priority, ...) {
} catch {
} finally {
}

Naturally the compiler will know how to convert synchronous syscalls
to their asynchronous equivalent, will use an analogue of IEEE NaNs to
minimize the hits to the exception path, and won't let you call
functions that aren't annotated as safe in IO completion context.  I
would also like five acres in town and a pony.

Cheers,
- Michael
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/