linux-kernel - Re: [RFC] Unify KVM kernel-space and user-space code into a single project

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <g6y0y1gaeew5bcwbUYAxe124vaj_firegpg@mail.gmail.com>
Date:	Thu, 18 Mar 2010 13:39:17 -0700 (PST)
From:	drepper@...il.com
To:	Ingo Molnar <mingo@...e.hu>
Cc:	Anthony Liguori <anthony@...emonkey.ws>,
	Avi Kivity <avi@...hat.com>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	linux-kernel@...r.kernel.org, kvm@...r.kernel.org,
	Marcelo Tosatti <mtosatti@...hat.com>,
	Joerg Roedel <joro@...tes.org>,
	Jes Sorensen <Jes.Sorensen@...hat.com>,
	Gleb Natapov <gleb@...hat.com>,
	Zachary Amsden <zamsden@...hat.com>,
	Arnaldo Carvalho de Melo <acme@...hat.com>,
	Fr?d?ric Weisbecker <fweisbec@...il.com>
Subject: Re: [RFC] Unify KVM kernel-space and user-space code into a single project

On Thu, Mar 18, 2010 at 13:18, Ingo Molnar <mingo@...e.hu> wrote:
> Where 'reasonable' is defined by you, right?

Not only by me.  For some of the AIO approaches which happened there were also glibc patches other people wrote.  It's pretty simple.

> As i said, the KAIO situation is mostly the kernel's fault, but you are a
> pretty passive and unhelpful entity in this matter too, arent you?

How'd you guess?  I've always been been willing to discuss interface requirements with whoever showed interest in implementing things.  Again, ask Zach.  I think Christoph Lameter also was involved as were various SGI people over the years.

Short of actually doing all the work myself I've done what can be expected.

> For example, just to state the obvious: libaio has been written 8 years ago in
> 2002 and has been used in apps early on. Why arent those kernel APIs, while
> not being a full/complete solution, supported by glibc, and wrapped to
> pthreads based emulation on kernels that dont support it?

You never looked at the glibc code in use and didn't read what I wrote before.  We do have an implementation of libaio using those interfaces.  They exist  in the Fedora/RHEL glibc and are probably copied elsewhere, too.  The code is not upstream because it is not general enough.  It simply doesn't work in all situations.

The problem with using it (among others) is that certain operations cannot be implemented.  And that's not a kernel interface problem.  I cannot just switch to using the pthread-based code when coming across something that's not implementable because then the requests have already been sent to the kernel.  Only code that knows about the limitations ahead of time can use the KAIO code.

> Why isnt glibc itself making use of those AIO capabilities internally? (even
> if it's not possible to support full POSIX AIO)

For what?  glibc doesn't implement anything requiring AIO.  The only non-trivial file handling is in nscd and nscd uses memory mapped files.

> I checked today's glibc repo, and there's no sign of any of that:

Check the Fedora/RHEL/... source files.

> Getting _something_ into glibc would certainly help move the situation.

No it won't as the 7+ since Jakub wrote the code nothing came out of it.  And before you again make groundless claims, there was plenty of discussions with kernel people at the time when the code was written.

> it's certainly capable enough to run DB servers. glibc
> using it would create further demand (and pressure, and incentives) for
> improvements.

There simply is no need for AIO in glibc internally.  Well, there might be, if it could be used on sockets.  But that's not the case.

> Charming argumentation style, i really missed it.

Well, then this last mail should show you.  Without knowing the subject matter, just based on flawed lookups you try to spread the blame to make sure that no mud ever sticks to development process you are so fond of.  Sorry to disappoint you.
Download attachment "signature.asc" of type "application/pgp-signature" (273 bytes)