linux-ext4 - Re: [PATCH] ext3,4:fdatasync should skip metadata writeout

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <6.0.0.20.2.20071120142849.03f5f5e0@172.19.0.2>
Date:	Tue, 20 Nov 2007 16:20:56 +0900
From:	Hisashi Hifumi <hifumi.hisashi@....ntt.co.jp>
To:	Andrew Morton <akpm@...ux-foundation.org>,
	Bryan Henderson <hbryan@...ibm.com>,
	J��n Engel <joern@...fs.org>
Cc:	linux-ext4@...r.kernel.org, linux-fsdevel@...r.kernel.org
Subject: Re: [PATCH] ext3,4:fdatasync should skip metadata writeout


At 11:59 07/11/16, Andrew Morton wrote:
 >
 >I suppose so.  Although one wonders what earthly point there is in syncing
 >a file's data if we haven't yet written out the metadata which is required
 >for locating that data.
 >
 >IOW, fdatasync() is only useful if the application knows that it is overwriting
 >already-instantiated blocks.
 >
 >In which case it might as well have used fsync().  For ext2-style filesystems,
 >anyway.
 >
 >hm.  It needs some thought.

I did a test to measure the file overwriting performance difference between
original fdatasync and one that skips journal flush.
The test program and obtained result is as follows:

Test program source code:

#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <time.h>

#define BUFSIZE 8192
#define LOOP 1024*1024

main(void)
{
	int i;
	int fd;
	char buf[BUFSIZE];
	time_t t1,t2;

	memset(buf,0,BUFSIZE);

	fd = open("testfile", O_CREAT|O_RDWR);
	if (fd < 0)
		perror("cannot open file\n");

	for (i = 0; i < LOOP; i++)
		write(fd,buf,BUFSIZE);
	fsync(fd);

	lseek(fd, 0, SEEK_SET);
	time(&t1);
	for (i = 0; i < LOOP; i++) {
		write(fd,buf,BUFSIZE);
		fdatasync(fd);
	}
	time(&t2);

	printf("%d sec\n",t2-t1);
}

Result:

  2.6.24-rc3:

	264 sec

  2.6.23-rc3-fdatasync-skips-journal-flush-patched

	253 sec

Hardware environment:
  Dell Poweredge 850
  CPU Pentium D  3GHz
  memory 4GB
  HDD Maxtor 6L160M0


I got somewhat better result from the patched ext3 skipping journal flush.

Some DBMS such as PostgreSQL can use fdatasync. So I think skipping
journal flush on overwriting leads to performance improvement for
these application.

I am for the notion that skipping metadata writeout unconditionally is wrong,
and "important metadata" such as i_size, block-bitmap etc should be
synched even if fdatasync is issued , but unimportant meta such as
mtime and ctime update can be ignored when a file is overwritten.


-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html