| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Wed, 10 Apr 2019 14:12:19 +0200
From: Lukas Czerner <lczerner@...hat.com>
To: Karel Zak <kzak@...hat.com>
Cc: zhenwei pi <pizhenwei@...edance.com>,
Theodore Ts'o <tytso@....edu>, linux-ext4@...r.kernel.org,
util-linux@...r.kernel.org
Subject: Re: [PATCH v2] fiemap : add fiemap misc tool
On Wed, Apr 10, 2019 at 11:59:47AM +0200, Karel Zak wrote:
> On Wed, Apr 10, 2019 at 04:12:20PM +0800, zhenwei pi wrote:
> > Add fiemap to dump file extent mappings. Typically we can recognize a
> > file is sparse or not.
>
> It's good idea to export FIEMAP to command line, but it seems we
> already have xfs_io and filefrag.
>
> The command filefrag seems pretty usable and with "-e" it provides
> exactly the same output like your "fiemap" tool.
>
> The problem I see is that filefrag is in e2fsprogs package. It would
> be better to have it in some more generic package and FS independent
> package.
>
> Maybe we can do another move from e2fsprogs to util-linux. Ted, your
> opinion?
Looking at the filefrag code the only ext specific part I can identify
is checking for non-extent based inode and changing the output slightly
as well as taking direct/indirect mapping block layout into consideration
in case FIBMAP is used.
Other than that it's purely generic and non ext[234] specific code that
can easily live in util-linux and personally I do not think I'd miss it :)
-Lukas
>
> Karel
>
>
> # filefrag -e /var/log/wtmp
> Filesystem type is: ef53
> File size of /var/log/wtmp is 9032064 (2206 blocks of 4096 bytes)
> ext: logical_offset: physical_offset: length: expected: flags:
> 0: 0.. 0: 11567426.. 11567426: 1:
> 1: 1.. 190: 11730944.. 11731133: 190: 11567427:
> 2: 191.. 380: 11732109.. 11732298: 190: 11731134:
> 3: 381.. 570: 11731883.. 11732072: 190: 11732299:
> 4: 571.. 633: 11731501.. 11731563: 63: 11732073:
> 5: 634.. 761: 11731584.. 11731711: 128: 11731564:
> 6: 762.. 763: 11731198.. 11731199: 2: 11731712:
> 7: 764.. 764: 11732073.. 11732073: 1: 11731200:
> 8: 765.. 955: 11731200.. 11731390: 191: 11732074:
> 9: 956.. 1145: 11732705.. 11732894: 190: 11731391:
> 10: 1146.. 1254: 11733137.. 11733245: 109: 11732895:
> 11: 1255.. 1444: 11732895.. 11733084: 190: 11733246:
> 12: 1445.. 1824: 11733246.. 11733625: 380: 11733085:
> 13: 1825.. 2014: 11737497.. 11737686: 190: 11733626:
> 14: 2015.. 2205: 11896890.. 11897080: 191: 11737687: last,eof
> /var/log/wtmp: 15 extents found
>
>
> >
> > For example:
> > #./fiemap /var/log/syslog /var/log/syslog.2.gz
> > File /var/log/syslog has 19 extent(s):
> > Logical Physical Length Flag
> > 0: 0000000000000000 00000010d63a8000 0000000000001000 0000
> > 1: 0000000000001000 00000010d63c4000 0000000000001000 0000
> > 2: 0000000000002000 00000010d63d6000 0000000000001000 0000
> > 3: 0000000000003000 00000010d723e000 0000000000001000 0000
> > 4: 0000000000004000 000000076c377000 0000000000001000 0000
> > 5: 0000000000005000 00000010d550f000 0000000000001000 0000
> > 6: 0000000000006000 00000010d7290000 0000000000001000 0000
> > 7: 0000000000007000 00000010d57c9000 0000000000001000 0000
> > 8: 0000000000008000 00000010d57f5000 0000000000001000 0000
> > 9: 0000000000009000 000000076d6b3000 0000000000001000 0000
> > 10: 000000000000a000 00000010d558d000 0000000000001000 0000
> > 11: 000000000000b000 00000010d77ff000 0000000000001000 0000
> > 12: 000000000000c000 00000010d67fb000 0000000000001000 0000
> > 13: 000000000000d000 00000010d73fc000 0000000000001000 0000
> > 14: 000000000000e000 00000005d2bb2000 0000000000001000 0000
> > 15: 000000000000f000 00000007705a1000 0000000000001000 0000
> > 16: 0000000000010000 0000000c8bf10000 0000000000010000 0000
> > 17: 0000000000020000 0000001070020000 0000000000060000 0000
> > 18: 0000000000080000 00000011aa580000 0000000000014000 0001
> >
> > File /var/log/syslog.2.gz has 1 extent(s):
> > Logical Physical Length Flag
> > 0: 0000000000000000 0000000995c40000 0000000000023000 0001
> >
> > Signed-off-by: zhenwei pi <pizhenwei@...edance.com>
> > ---
> > AUTHORS | 1 +
> > configure.ac | 3 +
> > misc-utils/Makemodule.am | 5 ++
> > misc-utils/fiemap.c | 155 +++++++++++++++++++++++++++++++++++++++++++++++
> > 4 files changed, 164 insertions(+)
> > create mode 100644 misc-utils/fiemap.c
> >
> > diff --git a/AUTHORS b/AUTHORS
> > index d7dbbd6..a0cecf7 100644
> > --- a/AUTHORS
> > +++ b/AUTHORS
> > @@ -20,6 +20,7 @@ AUTHORS (merged projects & commands):
> > fallocate: Eric Sandeen <sandeen@...hat.com>
> > Karel Zak <kzak@...hat.com>
> > Matěj Cepl <mcepl@...hat.com>
> > + fiemap: zhenwei pi <pizhenwei@...edance.com>
> > fincore: Masatake YAMATO <yamato@...hat.com>
> > findmnt: Karel Zak <kzak@...hat.com>
> > flock: H. Peter Anvin <hpa@...or.com>
> > diff --git a/configure.ac b/configure.ac
> > index bbf07db..ccb1a12 100644
> > --- a/configure.ac
> > +++ b/configure.ac
> > @@ -1769,6 +1769,9 @@ UL_REQUIRES_LINUX([fincore])
> > UL_REQUIRES_BUILD([fincore], [libsmartcols])
> > AM_CONDITIONAL([BUILD_FINCORE], [test "x$build_fincore" = xyes])
> >
> > +UL_BUILD_INIT([fiemap], [yes])
> > +AM_CONDITIONAL([BUILD_FIEMAP], [test "x$build_fiemap" = xyes])
> > +
> > UL_BUILD_INIT([fsfreeze], [check])
> > UL_REQUIRES_LINUX([fsfreeze])
> > AM_CONDITIONAL([BUILD_FSFREEZE], [test "x$build_fsfreeze" = xyes])
> > diff --git a/misc-utils/Makemodule.am b/misc-utils/Makemodule.am
> > index f56a819..8bdc6fa 100644
> > --- a/misc-utils/Makemodule.am
> > +++ b/misc-utils/Makemodule.am
> > @@ -228,3 +228,8 @@ hardlink_CFLAGS += $(PCRE_CFLAGS)
> > endif
> > dist_man_MANS += misc-utils/hardlink.1
> > endif
> > +
> > +if BUILD_FIEMAP
> > +usrbin_exec_PROGRAMS += fiemap
> > +fiemap_SOURCES = misc-utils/fiemap.c
> > +endif
> > diff --git a/misc-utils/fiemap.c b/misc-utils/fiemap.c
> > new file mode 100644
> > index 0000000..b77eb14
> > --- /dev/null
> > +++ b/misc-utils/fiemap.c
> > @@ -0,0 +1,155 @@
> > +/*
> > + * Copyright (C) 2019 zhenwei pi <pizhenwei@...edance.com>
> > + *
> > + * This file may be redistributed under the terms of the GNU Public
> > + * License.
> > + */
> > +#include <stdlib.h>
> > +#include <stdio.h>
> > +#include <string.h>
> > +#include <unistd.h>
> > +#include <fcntl.h>
> > +#include <getopt.h>
> > +#include <sys/ioctl.h>
> > +#include <linux/fs.h>
> > +
> > +#include "c.h"
> > +#include "nls.h"
> > +#include "closestream.h"
> > +
> > +#ifdef FS_IOC_FIEMAP
> > +#include <linux/fiemap.h>
> > +
> > +static struct fiemap *read_fiemap(int fd)
> > +{
> > + struct fiemap *fiemap;
> > + int extents_size;
> > +
> > + fiemap = (struct fiemap *)malloc(sizeof(struct fiemap));
> > + if (fiemap == NULL)
> > + err(EXIT_FAILURE, _("malloc"));
> > +
> > + memset(fiemap, 0, sizeof(struct fiemap));
> > +
> > + fiemap->fm_start = 0;
> > + fiemap->fm_length = ~0;
> > + fiemap->fm_flags = 0;
> > + fiemap->fm_extent_count = 0;
> > + fiemap->fm_mapped_extents = 0;
> > +
> > + /* count how many extents there are */
> > + if (ioctl(fd, FS_IOC_FIEMAP, fiemap) < 0)
> > + err(EXIT_FAILURE, _("fiemap ioctl() failed"));
> > +
> > + /* read in the extents */
> > + extents_size = sizeof(struct fiemap_extent) *
> > + (fiemap->fm_mapped_extents);
> > +
> > + /* resize fiemaps for all extents */
> > + fiemap = (struct fiemap *)realloc(fiemap, sizeof(struct fiemap) +
> > + extents_size);
> > + if (fiemap == NULL)
> > + err(EXIT_FAILURE, _("realloc for extents memory"));
> > +
> > + memset(fiemap->fm_extents, 0, extents_size);
> > + fiemap->fm_extent_count = fiemap->fm_mapped_extents;
> > + fiemap->fm_mapped_extents = 0;
> > +
> > + if (ioctl(fd, FS_IOC_FIEMAP, fiemap) < 0) {
> > + err(EXIT_FAILURE, _("ioctl() FS_IOC_FIEMAP failed"));
> > + return NULL;
> > + }
> > +
> > + return fiemap;
> > +}
> > +
> > +static void show_fiemap(struct fiemap *fiemap, char *filename)
> > +{
> > + unsigned int i = 0;
> > +
> > + printf("File %s has %d extent(s):\n", filename,
> > + fiemap->fm_mapped_extents);
> > + printf("#\tLogical Physical Length Flag\n");
> > + for (i = 0; i < fiemap->fm_mapped_extents; i++) {
> > + printf("%d:\t%-16.16llx %-16.16llx %-16.16llx %-4.4x\n", i,
> > + fiemap->fm_extents[i].fe_logical,
> > + fiemap->fm_extents[i].fe_physical,
> > + fiemap->fm_extents[i].fe_length,
> > + fiemap->fm_extents[i].fe_flags);
> > + }
> > +
> > + printf("\n");
> > +}
> > +
> > +static void __attribute__((__noreturn__)) usage(void)
> > +{
> > + FILE *out = stdout;
> > +
> > + fputs(USAGE_HEADER, out);
> > + fprintf(out, _(" %s <file>...\n"), program_invocation_short_name);
> > +
> > + fputs(USAGE_SEPARATOR, out);
> > + exit(EXIT_SUCCESS);
> > +}
> > +
> > +int main(int argc, char **argv)
> > +{
> > + int c;
> > + static const struct option longopts[] = {
> > + { "version", no_argument, NULL, 'V' },
> > + { "help", no_argument, NULL, 'h' },
> > + { NULL, 0, NULL, 0 },
> > + };
> > +
> > + setlocale(LC_ALL, "");
> > + bindtextdomain(PACKAGE, LOCALEDIR);
> > + textdomain(PACKAGE);
> > + atexit(close_stdout);
> > +
> > + while ((c = getopt_long (argc, argv, "Vh", longopts, NULL)) != -1) {
> > + switch (c) {
> > + case 'V':
> > + printf(UTIL_LINUX_VERSION);
> > + return EXIT_SUCCESS;
> > +
> > + case 'h':
> > + usage();
> > +
> > + default:
> > + errtryhelp(EXIT_FAILURE);
> > + }
> > + }
> > +
> > + if (optind == argc) {
> > + warnx(_("no file specified"));
> > + errtryhelp(EXIT_FAILURE);
> > + }
> > +
> > + for ( ; optind < argc; optind++) {
> > + int fd = 0;
> > +
> > + fd = open(argv[optind], O_RDONLY);
> > + if (fd < 0) {
> > + err(EXIT_FAILURE, _("open file failed"));
> > + } else {
> > + struct fiemap *fiemap = NULL;
> > +
> > + fiemap = read_fiemap(fd);
> > + if (fiemap != NULL) {
> > + show_fiemap(fiemap, argv[optind]);
> > + free(fiemap);
> > + }
> > + close(fd);
> > + }
> > + }
> > +
> > + return 0;
> > +}
> > +#else
> > +int main(int argc, char **argv)
> > +{
> > + err(EXIT_FAILURE, _("FS_IOC_FIEMAP not supported"));
> > +
> > + return 0;
> > +}
> > +#endif
> > --
> > 2.7.4
> >
>
> --
> Karel Zak <kzak@...hat.com>
> http://karelzak.blogspot.com
Powered by blists - more mailing lists