linux-kernel - Re: [Lsf-pc] [LSF/MM TOPIC] multi-stream IO hint implementation proposal for LSF/MM 2016

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160217152155.GA14140@quack.suse.cz>
Date:	Wed, 17 Feb 2016 16:21:55 +0100
From:	Jan Kara <jack@...e.cz>
To:	Changho Choi-SSI <changho.c@....samsung.com>
Cc:	"lsf-pc@...ts.linux-foundation.org" 
	<lsf-pc@...ts.linux-foundation.org>, linux-fsdevel@...r.kernel.org,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: [Lsf-pc] [LSF/MM TOPIC] multi-stream IO hint implementation
 proposal for LSF/MM 2016

On Sat 13-02-16 01:50:09, Changho Choi-SSI wrote:
> Dear Program committee,
> 
> I wanted to propose a technical discussion.
> Please let me know if there is anything else that I have to submit and/or
> prepare.

As a side note: It is good to CC other relevant mailing lists so that
corresponding developers can react to the proposal.

> ==
> Linux Kernel Multi-stream I/O Hint Implementation
> 
> Enterprise, datacenter, and client systems increasingly deploy NAND
> flash-based SSDs. However, in use, SSDs cannot avoid inevitable garbage
> collection that deterministically causes write amplification which
> decreases device performance. Unfortunately, write amplification also
> decreases SSD lifetime. However, with multi-stream, unavoidable garbage
> collection overhead (e.g., write amplification) can be significantly
> reduced. For multi-stream devices, the host tags device I/O write
> requests with a stream ID (e.g., I/O hint). The SSD controller places the
> data in media erase blocks according to the stream ID. For example, a SSD
> controller stores data with same stream ID in an associated physical
> location inside SSD. In this way, the multi-stream depends on host I/O
> hints. So it is useful to develop how to implement multi-stream I/O hints
> under limited protocol constraints. The T10 SCSI standard group has
> already standardized the multi-stream feature and NVMe standardization is
> an ticipated in March, 2016. Many Linux users want to leverage
> multi-stream as a mainstream Linux feature since they have seen
> performance improvement and SSD lifetime extension when evaluating
> multi-stream enabled devices. Hence, the multi-stream feature is a good
> Linux community development candidate and should be discussed within the
> community.  I propose this multi-stream topic (i.e., I/O write hint
> implementation) in a discussion session. I can briefly present the
> multi-stream system architecture and answer any technical questions.

So a key question for a feature like this is: How many stream IDs are
devices going to support? Because AFAIR so far the answer was "it depends
on the device". However the design how stream IDs can be used greatly
differs between "a couple of stream IDs" and e.g. 2^32 stream IDs. Without
this information I don't think the discussion would be very useful. So can
you provide some rough numbers?

								Honza
-- 
Jan Kara <jack@...e.com>
SUSE Labs, CR