linux-kernel - Re: [PATCH v2] Support for write stream IDs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 05 May 2015 16:51:11 -0400
From:	Jeff Moyer <jmoyer@...hat.com>
To:	Jens Axboe <axboe@...com>
Cc:	<linux-kernel@...r.kernel.org>, <linux-fsdevel@...r.kernel.org>,
	<adilger@...ger.ca>, <david@...morbit.com>
Subject: Re: [PATCH v2] Support for write stream IDs

Jens Axboe <axboe@...com> writes:

> Hi,
>
> Changes since the last posting:
>
> - Added a specific per-file fadvise setting. POSIX_FADV_STREAMID sets
>   the inode and file stream ID, POSIX_FADV_STREAMID_FILE sets just the
>   file stream ID.
>
> - Addressed review comments.
>
> I've since run some testing with write streams. Test case was a RocksDB
> overwrite benchmark, using 3 billion keys of 400B in size (numbers set
> use the full size of the device). WAL/LOG was assigned to stream 1, and
> each RocksDB compaction level used a separate stream. With streams
> enabled, user write to device writes (write amplification) was at 2.33.
> Without streams, the write amplification was 3.05. That is roughly 20%
> less written NAND, and the streams test subsequently also had 20%
> higher throughput.
>
> Unless there are any grave concerns here, I'd like to merge this for
> 4.2.

I have a few concerns.  You've added POSIX_FADV_* definitions that do
not exist in the SUS/POSIX spec.  Do we care?  We (poor reviewers) still
have no idea what the driver side of this will look like.  Do streams
need to be opened and closed?  Is that going to be handled transparently
by the kernel, or exposed to userspace?  If in the kernel, where in the
kernel?  You've also added a user-visible api without cc-ing linux-api.

My preference would be to wait for the spec to finalize before pushing
in changes that depend on it.

Cheers,
Jeff
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/