linux-kernel - Re: [Lsf-pc] [LSF/MM TOPIC] really large storage sectors

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1390419019.2372.89.camel@dabdike.int.hansenpartnership.com>
Date:	Wed, 22 Jan 2014 11:30:19 -0800
From:	James Bottomley <James.Bottomley@...senPartnership.com>
To:	Ric Wheeler <rwheeler@...hat.com>
Cc:	"linux-ide@...r.kernel.org" <linux-ide@...r.kernel.org>,
	"linux-scsi@...r.kernel.org" <linux-scsi@...r.kernel.org>,
	Chris Mason <clm@...com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"mgorman@...e.de" <mgorman@...e.de>,
	"linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
	"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
	"lsf-pc@...ts.linux-foundation.org" 
	<lsf-pc@...ts.linux-foundation.org>
Subject: Re: [Lsf-pc] [LSF/MM TOPIC] really large storage sectors - going
 beyond 4096 bytes

On Wed, 2014-01-22 at 13:39 -0500, Ric Wheeler wrote:
> On 01/22/2014 01:35 PM, James Bottomley wrote:
> > On Wed, 2014-01-22 at 13:17 -0500, Ric Wheeler wrote:
[...]
> >> I think that the key to having the file system work with larger
> >> sectors is to
> >> create them properly aligned and use the actual, native sector size as
> >> their FS
> >> block size. Which is pretty much back the original challenge.
> > Only if you think laying out stuff requires block size changes.  If a 4k
> > block filesystem's allocation algorithm tried to allocate on a 16k
> > boundary for instance, that gets us a lot of the performance without
> > needing a lot of alteration.
> 
> The key here is that we cannot assume that writes happen only during 
> allocation/append mode.

But that doesn't matter at all, does it?  If the file is sector aligned,
then the write is aligned.  If the write is short on a large block fs,
well we'd just have to do the RMW in the OS anyway ... is that any
better than doing it in the device?

> Unless the block size enforces it, we will have non-aligned, small
> block IO done 
> to allocated regions that won't get coalesced.

We always get that if it's the use pattern ... the question merely
becomes who bears the burden of RMW.

> > It's not even obvious that an ignorant 4k layout is going to be so
> > bad ... the RMW occurs only at the ends of the transfers, not in the
> > middle.  If we say 16k physical block and average 128k transfers,
> > probabalistically we misalign on 6 out of 31 sectors (or 19% of the
> > time).  We can make that better by increasing the transfer size (it
> > comes down to 10% for 256k transfers.
> 
> This really depends on the nature of the device. Some devices could
> produce very 
> erratic performance

Yes, we get that today with misaligned writes to the 4k devices.

>  or even (not today, but some day) reject the IO.

I really doubt this.  All 4k drives today do RMW ... I don't see that
changing any time soon.

> >> Teaching each and every file system to be aligned at the storage
> >> granularity/minimum IO size when that is larger than the physical
> >> sector size is
> >> harder I think.
> > But you're making assumptions about needing larger block sizes.  I'm
> > asking what can we do with what we currently have?  Increasing the
> > transfer size is a way of mitigating the problem with no FS support
> > whatever.  Adding alignment to the FS layout algorithm is another.  When
> > you've done both of those, I think you're already at the 99% aligned
> > case, which is "do we need to bother any more" territory for me.
> >
> 
> I would say no, we will eventually need larger file system block sizes.
> 
> Tuning and getting 95% (98%?) of the way there with alignment and IO
> scheduler 
> does help a lot. That is what we do today and it is important when
> looking for 
> high performance.
> 
> However, this is more of a short term work around for a lack of a
> fundamental 
> ability to do the right sized file system block for a specific class
> of device. 
> As such, not a crisis that must be solved today, but rather something
> that I 
> think is definitely worth looking at so we can figure this out over
> the next 
> year or so.

But this, I think, is the fundamental point for debate.  If we can pull
alignment and other tricks to solve 99% of the problem is there a need
for radical VM surgery?  Is there anything coming down the pipe in the
future that may move the devices ahead of the tricks?

James



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/