linux-kernel - Re: Linux 2.6.29

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20090402094247.9d7ac19f.akpm@linux-foundation.org>
Date:	Thu, 2 Apr 2009 09:42:47 -0700
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	David Rees <drees76@...il.com>
Cc:	Janne Grunau <j@...nau.net>,
	Lennart Sorensen <lsorense@...lub.uwaterloo.ca>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Theodore Tso <tytso@....edu>, Jesper Krogh <jesper@...gh.cc>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: Linux 2.6.29

On Thu, 2 Apr 2009 09:29:59 -0700 David Rees <drees76@...il.com> wrote:

> On Thu, Apr 2, 2009 at 4:05 AM, Janne Grunau <j@...nau.net> wrote:
> >> Current kernels just have too much IO latency
> >> with ext3 it seems.
> >
> > MythTV calls fsync every few seconds on ongoing recordings to prevent
> > stalls due to large cache writebacks on ext3.
> 
> Personally that is also one of my MythTV pet peeves.  A hack added to
> MythTV to work around a crappy ext3 latency bug that also causes these
> large files to get heavily fragmented.  That and the fact that yo have
> to patch MythTV to eliminate those forced fdatasyncs - there is no
> knob to turn it off if you're running MythTV on a filesystem which
> doesn't suffer from ext3's data=ordered fsync stalls.
> 

For any filesystem it is quite sensible for an application to manage
the amount of dirty memory which the kernel is holding on its behalf,
and based upon the application's knowledge of its future access
patterns.

But MythTV did it the wrong way.

A suitable design for the streaming might be, every 4MB:

- run sync_file_range(SYNC_FILE_RANGE_WRITE) to get the 4MB underway
  to the disk

- run fadvise(POSIX_FADV_DONTNEED) against the previous 4MB to
  discard it from pagecache.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/