lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening PHC | |
Open Source and information security mailing list archives
| ||
|
Date: Mon, 3 Nov 2008 09:45:13 +1100 From: Dave Chinner <david@...morbit.com> To: Chad Talbott <ctalbott@...gle.com> Cc: linux-kernel@...r.kernel.org, Andrew Morton <akpm@...gle.com>, Michael Rubin <mrubin@...gle.com> Subject: Re: Metadata in sys_sync_file_range and fadvise(DONTNEED) On Fri, Oct 31, 2008 at 01:54:14PM -0700, Chad Talbott wrote: > We are looking at adding calls to posix_fadvise(DONTNEED) to various > data logging routines. This has two benefits: > > - frequent write-out -> shorter queues give lower latency, also disk > is more utilized as writeout begins immediately > > - less useless stuff in page cache > > One problem with fadvise() (and ext2, at least) is that associated > metadata isn't scheduled with the data. So, for a large log file with > a high append rate, hundreds of indirect blocks are left to be written > out by periodic writeback. This metadata consists of single blocks > spaced by 4MB, leading to spikes of very inefficient disk utilization, > deep queues and high latency. Sounds like a filesystem bug to me, not a problem with posix_fadvise(DONTNEED). > Andrew suggests a new SYNC_FILE_RANGE_METADATA flag for > sys_sync_file_range(), and leaving posix_fadvise() alone. What is the interface that a filesystem will see? No filesystem has a "metadata sync" method - is this going to fall through to some new convoluted combination of writeback flags to an inode/mapping that more filesystems than not can get wrong? FWIW, sys_sync_file_range() is fundamentally broken for data integrity writeback - at no time does it call a filesystem method that can result in a barrier I/O being issued to disk after writeback is complete. So, unlike fsync() or fdatasync(), the data can still be lost after completion due to power failure on drives with volatile write caches.... Cheers, Dave. -- Dave Chinner david@...morbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists