linux-kernel - Re: How to handle >16TB devices on 32 bit hosts ??

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 21 Jul 2009 23:59:04 -0700
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	Neil Brown <neilb@...e.de>
Cc:	linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org,
	linux-raid@...r.kernel.org, dm-devel@...hat.com
Subject: Re: How to handle >16TB devices on 32 bit hosts ??

On Sat, 18 Jul 2009 10:08:10 +1000 Neil Brown <neilb@...e.de> wrote:

>  It has recently come to by attention that Linux on a 32 bit host does
>  not handle devices beyond 16TB particularly well.
> 
>  In particular, any access that goes through the page cache for the
>  block device is limited to a pgoff_t number of pages.
>  As pgoff_t is "unsigned long" and hence 32bit, and as page size is
>  4096, this comes to 16TB total.

I expect that the VFS could be made to work with 64-bit pgoff_t fairly
easily.  The generated code will be pretty damn sad.

radix-trees use a ulong index, so we would need a new
lib/radix_tree64.c or some other means of fixing that up.

The bigger problem is filesystems - they'll each need to be checked,
tested, fixed and enabled.  It's probably not too bad for the
mainstream filesystems which mostly bounce their operations into VFS
libarary functions anyway.

There's perhaps a middle ground - support >16TB devices, but not >16TB
partitions.  That way everything remains 32-bit and we just have to get
the offsetting right (probably already the case).

So now /dev/sda1, /dev/sda2 etc are all <16TB.  The remaining problem
is that /dev/sda is >16TB.  I expect that we could arrange for the
kernel to error out if userspace tries to access /dev/sda beyond the
16TB point, and those very very few applications which want to touch
that part of the disk will need to be written using direct-io, (or
perhaps sgio) or run on 64-bit machines.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/