| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Tue, 21 Jul 2009 07:45:58 -0700 From: Curt Wohlgemuth <curtw@...gle.com> To: Eric Sandeen <sandeen@...hat.com> Cc: Xiang Wang <xiangw@...gle.com>, linux-ext4@...r.kernel.org Subject: Re: Using O_DIRECT in ext4 On Mon, Jul 20, 2009 at 8:41 PM, Eric Sandeen<sandeen@...hat.com> wrote: > Xiang Wang wrote: >> Hi, >> >> Recently I've been experimenting with O_DIRECT in ext4 to get a >> feeling of how much file fragmentation will be generated. >> >> On a newly formatted ext4 partition(no-journal), I created a top-level >> directory and under this top-level directory I ran a test program to >> generate some files. >> >> The test program does the following: >> -- create multiple threads(in my test case: 16 threads) >> -- each thread creates a file with the O_DIRECT flag and keeps >> extending the file to 1MB >> Since these threads run concurrently, they compete in block allocation. >> >> After the program ran to a completion, I ran filefrag on each file and >> measure how many extents there are in the file. >> And here is a sample result: >> file0: 6 extents found >> file1: 20 extents found >> file2: 7 extents found >> file3: 6 extents found >> file4: 6 extents found >> file5: 5 extents found >> file6: 6 extents found >> file7: 20 extents found >> file8: 20 extents found >> file9: 20 extents found >> file10: 20 extents found >> file11: 20 extents found >> file12: 20 extents found >> file13: 19 extents found >> file14: 19 extents found >> file15: 19 extents found >> >> Looks like these files are quite heavily fragmented. > > Multiple parallel extending DIOs in a single dir is a tough case for a > filesystem - it has no hints about what to do, and can't use delalloc to > wait to see what's happening; it just has to allocate things as they > come, more or less. > >> For comparison, I did the same experiment on an ext2 partition, >> resulting in each file having only 1 extent. > > Interestinng, not sure I would have expected that. Same with us; we're looking into more variables to understand it. >> I also did the experiments of using buffered writes(by removing the >> O_DIRECT flag) on ext2 and ext4, both resulting in each file having >> only 1 extent. > > delayed allocation at work I suppose. > >> I am wondering whether this kind of file fragmentation is already a >> known issue in ext4 when O_DIRECT is used? Is it something by design? >> Since it seems like ext2 does not have this issue under my test case, >> is it necessary that we make the behavior of ext4 similar to ext2 >> under situations like this? > > Is this representative of a real workload? Not exactly perhaps, but we do have apps that are showing significantly more fragmentation in their files on ext4 than with ext2, while using O_DIRECT (e.g., 8 extents on ext4 vs 1 on ext2, as reported by filefrag). The experiment above is synthetic, but fairly representative. (Hence the related questions about fallocate, since this is one possible, though ugly, workaround.) Curt -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists