linux-kernel - Re: [PATCH V8 00/33] loop: Issue O_DIRECT aio using bio

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130731004303.GA7118@dastard>
Date:	Wed, 31 Jul 2013 10:43:03 +1000
From:	Dave Chinner <david@...morbit.com>
To:	Andrew Morton <akpm@...ux-foundation.org>
Cc:	Dave Kleikamp <dave.kleikamp@...cle.com>,
	linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org,
	"Maxim V. Patlasov" <mpatlasov@...allels.com>,
	Zach Brown <zab@...bo.net>
Subject: Re: [PATCH V8 00/33] loop: Issue O_DIRECT aio using bio_vec

On Tue, Jul 30, 2013 at 02:28:20PM -0700, Andrew Morton wrote:
> On Thu, 25 Jul 2013 12:50:26 -0500 Dave Kleikamp <dave.kleikamp@...cle.com> wrote:
> 
> > This patch series adds a kernel interface to fs/aio.c so that kernel code can
> > issue concurrent asynchronous IO to file systems.  It adds an aio command and
> > file system methods which specify io memory with pages instead of userspace
> > addresses.
> > 
> > This series was written to reduce the current overhead loop imposes by
> > performing synchronus buffered file system IO from a kernel thread.  These
> > patches turn loop into a light weight layer that translates bios into iocbs.
> 
> Do you have any performance numbers?
> 
> Does anyone care much about loop performance?  What's the value here?

Yes. Anyone using loopback devices for file-backed devices exposed
to containers and VMs cares about the memory and CPU overhead
the double caching the existing loop device has.

Or those of us how use loopback devices to examine metadumps from
broken filesystems and run repair/fsck on the filesystem image via
loopback devices. When I'm dealing with images containing tens to
hundreds of gigabytes of metadata, caching a second time in the
backing file is a significant overhead. It's especially annoying
when the application is already using direct IO because the kernel
based block device caching isn't at all efficient and just consumes
needless amount of memory holding on to pages that are never going
to be read again.

And on small memory machines xfstests can trigger OOM killer when it
uses loopback devices in certain tests. Thats generally caused by
the writeback of a dirty page causing a new page to be allocated and
dirtied and so writeback of dirty memory under memory pressure can
a) increase memory usage, and b) increase the percentage of dirty
memory that can't be reclaimed immediately....

Cheers,

Dave.
-- 
Dave Chinner
david@...morbit.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/