linux-kernel - Re: [PATCH 6/6] Add dockerfile

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160819232953.GB12834@thunk.org>
Date:   Fri, 19 Aug 2016 19:29:53 -0400
From:   Theodore Ts'o <tytso@....edu>
To:     Dmitry Monakhov <dmonakhov@...nvz.org>
Cc:     linux-kernel@...r.kernel.org
Subject: Re: [PATCH 6/6] Add dockerfile

On Fri, Aug 19, 2016 at 12:27:43PM +0300, Dmitry Monakhov wrote:
> Actually main idea is to have build environment for xfstests-bld development
> plus automatic builds gives us CI. This is important because
> xfstests-blk build is silently broken since May 30 due to 82c3a179
> (fix proposed: https://github.com/dmonakhov/xfsprogs/commit/5dc7a0805d6b7a265863413a8e1b7acb191d5280)
> But this was uncovered because most people do not build xfstests-bld
> from scratch very often, we all too busy. Let's machine do that :)
> In fact other good candidate for CI is https://travis-ci.com/

Actually, I'm rebuilding all the time, but I'm not upgrading to the
latest bleeding edge version of xfsprogs all the time.  In fact, I'm
currently still using xfsprogs v4.5.0, since that's plenty for the
testing I'm doing.  I have historically pinned the commit Id's for
certain projects where newer versions were either (a) known to be
flaky, or (b) known to be broken in some way -- that's why we
currently have explicit commit ID's for fio and quota.  Perhaps I
should add one for xfsprogs as well, so that we only upgrade to newer
versions when it known to be needed and where it's helpful.

> Once we have Dockerfile which encapsulates all dependencies
> one can create xfstets-blk from scratch simply typing
> $ docker build -t my-xfstests-bld-image --file Dockerfile
> Or use precreated environment
> $ docker pull XXX/xfstests-bld

If you use the precreated environment that will have a fixed commit
ID.  So the advantage of doing continuous integration of the bleeding
edge version of xfsprogs only works if you are using the "docker
build" line, right.

> Once image is created 
> Log in to container and start hacking xfstest or e2fsprogs etc.
> $ docker run -i -t XXX/xfstests-bld
> $ emacs xfstest/tests/ext4/023 ....
> $ kvm-xfstests --update-xfstest ext4/023
> ...

I'm not sure I see the advantage of doing this in a container, I
guess.  I just do in my standard laptop environment today.

> Other useful feature is ability to speed xfstests-bld by splitting
> full test-set to groups and run kvm-xfstests instances in parallel.
> For example:
> docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
>        kvm-xfstests.sh --kernel /my-kernel/bzImage -c 1k > 1k.log &
> docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
>        kvm-xfstests.sh --kernel /my-kernel/bzImage -c 4k > 4k.log &
> docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
>        kvm-xfstests.sh --kernel /my-kernel/bzImage -c dax > dax.log &
> .....
> w/o sand box they will conflict due to net ports (localhost:7500 etc)

... and because you will need separate scratch disks for each of the
docker runs.

OK, I can see how that's certainly an advantage..  This is what I use
gce-xfstests for, myself.  :-)

The main thing I don't really like about the Dockerfile is that with
the exception of the last use case, everyone would really need to have
their own customized version of the Docker file, right?  And for
someone who isn't Docker aware, I'm concerned that it might be very
confusing to use, especially compared to all of the documentation I
currently have for kvm-xfstests and gce-xfstests.

Hmm, I'll have to think about this and play with it a bit more...

     	       	  	      	       	    - Ted