linux-kernel - Re: How to handle >16TB devices on 32 bit hosts ??

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090718074811.GA2682@basil.fritz.box>
Date:	Sat, 18 Jul 2009 09:48:11 +0200
From:	Andi Kleen <andi@...stfloor.org>
To:	Andreas Dilger <adilger@....com>
Cc:	Andi Kleen <andi@...stfloor.org>,
	device-mapper development <dm-devel@...hat.com>,
	Neil Brown <neilb@...e.de>, linux-fsdevel@...r.kernel.org,
	linux-raid@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: How to handle >16TB devices on 32 bit hosts ??

On Sat, Jul 18, 2009 at 02:52:13AM -0400, Andreas Dilger wrote:
> If you aren't running a 32-bit system with this config, you shouldn't
> really care.  For those systems that need to run in this mode they
> would rather have it work a few percent slower instead of not at all.

Well, it doesn't work at all anyways due to the fsck problem.

> The last test numbers I saw were 5GB of RAM for a 20TB filesystem,
> but since the bitmaps used are fully-allocated arrays that isn't
> surprising.  We are planning to replace this with a tree, since the
> majority of bitmaps used by e2fsck have large contiguous ranges of
> set or unset bits and can be represented much more efficiently.

You would need to get <~2.5GB for 32bit. In practice that's
the limit you have there.

> Also, for filesystems like btrfs or ZFS the checking can be done
> online and incrementally without storing a full representation of
> the state in memory.

You could, but I suspect it would be cheaper to just use a
64bit system than to rewrite fsck. 64bit is available
for a lot of embedded setups these days too.

-Andi

-- 
ak@...ux.intel.com -- Speaking for myself only.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/