linux-ext4 - Re: [fstests PATCH v3] generic: add test for DAX MAP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20171025215638.GA30335@dastard>
Date:   Thu, 26 Oct 2017 08:56:38 +1100
From:   Dave Chinner <david@...morbit.com>
To:     Ross Zwisler <ross.zwisler@...ux.intel.com>
Cc:     Amir Goldstein <amir73il@...il.com>, Josef Bacik <jbacik@...com>,
        Eryu Guan <eguan@...hat.com>,
        fstests <fstests@...r.kernel.org>,
        linux-kernel <linux-kernel@...r.kernel.org>,
        Alasdair Kergon <agk@...hat.com>,
        Dan Williams <dan.j.williams@...el.com>,
        Jan Kara <jack@...e.cz>, Mike Snitzer <snitzer@...hat.com>,
        Shaohua Li <shli@...nel.org>, dm-devel@...hat.com,
        linux-nvdimm@...ts.01.org, linux-raid@...r.kernel.org,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>,
        Ext4 <linux-ext4@...r.kernel.org>,
        linux-xfs <linux-xfs@...r.kernel.org>,
        Christoph Hellwig <hch@...radead.org>
Subject: Re: [fstests PATCH v3] generic: add test for DAX MAP_SYNC support

On Wed, Oct 25, 2017 at 02:47:04PM -0600, Ross Zwisler wrote:
> Add a test that exercises DAX's new MAP_SYNC flag.
> 
> This test creates a file and writes to it via an mmap(), but never syncs
> via fsync/msync.  This process is tracked via dm-log-writes, then replayed.
> 
> If MAP_SYNC is working the dm-log-writes replay will show the test file
> with 1MiB of on-media block allocations.  This is because each allocating
> page fault included an implicit metadata sync.  If MAP_SYNC isn't working
> (which you can test by fiddling with the parameters to mmap()) the file
> will be smaller or missing entirely.
> 
> Note that dm-log-writes doesn't track the data that we write via the
> mmap(), so we can't do any data integrity checking.  We can only verify
> that the metadata writes for the page faults happened.
> 
> Signed-off-by: Ross Zwisler <ross.zwisler@...ux.intel.com>
.....
> --- /dev/null
> +++ b/src/t_map_sync.c
> @@ -0,0 +1,92 @@
> +#include <errno.h>
> +#include <fcntl.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <sys/mman.h>
> +#include <sys/stat.h>
> +#include <sys/types.h>
> +#include <unistd.h>
> +
> +#define MiB(a) ((a)*1024*1024)
> +
> +/*
> + * These two defines were added to the kernel via commits entitled
> + * "mm: Define MAP_SYNC and VM_SYNC flags" and
> + * "mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap
> + * flags", respectively.
> + */
> +#ifndef MAP_SYNC
> +#define MAP_SYNC 0x80000
> +#endif
> +
> +#ifndef MAP_SHARED_VALIDATE
> +#define MAP_SHARED_VALIDATE 0x3
> +#endif

Autoconf rules for detecting supported functionality, please...

> +
> +void err_exit(char *op)
> +{
> +	fprintf(stderr, "%s: %s\n", op, strerror(errno));
> +	exit(1);
> +}
> +
> +void mark_log(char *logwrites_name, char *mark_name)
> +{
> +	char command[256];
> +
> +	snprintf(command, 256, "dmsetup message %s 0 mark %s",
> +			logwrites_name, mark_name);
> +
> +	if (system(command))
> +		err_exit("mark_log");
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +	int page_size = getpagesize();
> +	int len = MiB(1);
> +	int i, fd, err;
> +	char *data;
> +
> +	if (argc < 4) {
> +		printf("Usage: %s <file> <logwrites_name> <mark_name>\n",
> +				basename(argv[0]));
> +		exit(0);
> +	}
> +
> +	fd = open(argv[1], O_RDWR|O_CREAT|O_TRUNC, S_IRUSR|S_IWUSR);
> +	if (fd < 0)
> +		err_exit("fd");
> +
> +	ftruncate(fd, len);
> +
> +	data = mmap(NULL, len, PROT_READ|PROT_WRITE,
> +			MAP_SHARED_VALIDATE|MAP_SYNC, fd, 0);
> +	if (data == MAP_FAILED)
> +		err_exit("mmap");

As I say to all these sorts of one-off test prgrams: please add the
new MAP_SYNC flag to xfs_io rather than writing a one-off
test program to set it and write some data.

And if we're going to be adding special custom tests just because
we need to insert dm-log marks, add that functionality to xfs_io,
too.

That way we can create complex custom dm logwrite tests without
needing one-off test programs for them all...

> +#! /bin/bash
> +# FS QA Test No. 466
> +#
> +# Use md_log_writes to verify that MAP_SYNC actually syncs metadata during

dm_log_writes?

> +# We should see $SCRATCH_MNT/test as having 1MiB in block allocations
> +du -sh $SCRATCH_MNT/test | _filter_scratch | _filter_spaces

Perhaps stat -c %b $SCRATCH_MNT/test ?

Cheers,

Dave.
-- 
Dave Chinner
david@...morbit.com