linux-kernel - Re: [RFC][PATCH] ChunkFS: fs fission for faster fsck

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20070428100319.GA25493@lazybastard.org>
Date:	Sat, 28 Apr 2007 12:03:20 +0200
From:	Jörn Engel <joern@...ybastard.org>
To:	Valerie Henson <val_henson@...ux.intel.com>
Cc:	Jan Kara <jack@...e.cz>, David Chinner <dgc@....com>,
	Amit Gud <gud@....edu>, Nikita Danilov <nikita@...sterfs.com>,
	David Lang <david.lang@...italinsight.com>,
	linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org,
	riel@...riel.com, zab@...bo.net, arjan@...radead.org,
	suparna@...ibm.com, brandon@...p.org, karunasagark@...il.com
Subject: Re: [RFC][PATCH] ChunkFS: fs fission for faster fsck

On Fri, 27 April 2007 23:50:06 -0700, Valerie Henson wrote:
> 
> Any mapping structure will have to be pre-allocated.

How much space would you allocate for it then?  The required size surely
depends on the file size and fragmentation.

> So in my secret heart of hearts, I do indeed hope that cnodes are rare
> enough that we don't actually have to do anything smart to make them
> go fast.  Either having no fast lookup structure or creating it in
> memory as needed would be the nicest solution.  However, since I can't
> guarantee this will be the case, it's nice to have some idea of what
> we'll do if this does become important.

You go slow and slower.  Or you can defragment/clean the filesystem.
Fragmented filesystems being slow is hardly news to anyone.  ChunkFS
will be slower than others, so the reasons to fight fragmentation weigh
heavier.

What you can do:
- If two files 1 and 2 contain blocks in chunks A and B, move file 1
  blocks to chunk A and file 2 blocks to chunk B until one of the files
  only spans a single chunk.
- Same as above with longer chains like file 1 in chunks (A,B), file 2
  in chunks (B,C), file 3 in chunks (A,C).
- When writing to file 1 in chunk A but running out of space in chunk A,
  move some cross-chunk file over instead of writing file 1 blocks to
  chunk B.

The first two can hope to use some idle times (if they exist), the third
will make writes go slower in order to speed up future reads.

There ain't no such thing as a free lunch. :)

Jörn

-- 
I can say that I spend most of my time fixing bugs even if I have lots
of new features to implement in mind, but I give bugs more priority.
-- Andrea Arcangeli, 2000
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/