linux-kernel - Re: [PATCH RFC] Support for write stream IDs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <551199FE.7060901@kernel.dk>
Date:	Tue, 24 Mar 2015 11:08:14 -0600
From:	Jens Axboe <axboe@...nel.dk>
To:	Jeff Moyer <jmoyer@...hat.com>, Jens Axboe <axboe@...com>
CC:	linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org,
	ming.l@....samsung.com
Subject: Re: [PATCH RFC] Support for write stream IDs

On 03/24/2015 11:03 AM, Jeff Moyer wrote:
> Jens Axboe <axboe@...com> writes:
>
>> Hi,
>>
>> One of the things that exacerbates write amplification on flash
>> based devices is that fact that data with different lifetimes get
>> grouped together on media. Currently we have no interface that
>> applications can use to separate different types of writes. This
>> patch set adds support for that.
>>
>> The kernel has no knowledge of what stream ID is what. The idea is
>> that writes with identical stream IDs have similar life times, not
>> that stream ID 'X' has a shorter lifetime than stream ID 'X+1'.
>>
>> There are basically two interfaces that could be used for this. One
>> is fcntl, the other is fadvise. This patchset uses fadvise, with a
>> new POSIX_FADV_STREAMID hint. The 'offset' field is used to pass
>> the relevant stream ID. Switching to fcntl (with a SET/GET_STREAMID)
>> would be trivial.
>>
>> The patchset wires up the block parts, adds buffered and O_DIRECT
>> support, and modifies btrfs/xfs too. It should be trivial to extend
>> this to all other file systems, I just used xfs and btrfs for testing.
>>
>> No block drivers are wired up yet. Patches are against current -git.
>
> Can you give an idea of how the stream id would be communicated to the
> device?  NVMe doesn't appear to have any notion of a data stream ID.

It doesn't have it, yet, completely. Ming Lin can expand on what it 
looks like for the Samsung nvme devices. Current nvme does have vague 
support for it, however. The write command does have bits for frequent 
vs infrequent vs one-time writes.

-- 
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/