linux-kernel - Re: [PATCH 6/6] Add dockerfile

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 19 Aug 2016 12:27:43 +0300
From:   Dmitry Monakhov <dmonakhov@...nvz.org>
To:     Theodore Ts'o <tytso@....edu>
Cc:     linux-kernel@...r.kernel.org
Subject: Re: [PATCH 6/6] Add dockerfile

Theodore Ts'o <tytso@....edu> writes:

> On Fri, Aug 19, 2016 at 12:54:11AM +0400, Dmitry Monakhov wrote:
>> Dockerfile is good way to create build environments
>> Let's keep it as a reference build script.
>> 
>> XXX: Currently Ted does not have hub.docker account
>>      so this docker file points to my hub. Should be updated.
>
> I guess I'm a little confused about how this Dockerfile works.  First
> of all, says "From Debian", so I guess the idea is to be based on an
> unspecified version of Debian (why not debian:jessie?).  But then it
> has apt-get commands?  How does that work?
Yes, this is an official debian image https://hub.docker.com/_/debian .
plain 'debian' means 'debian:jessie', but you can change it to 'debian:sid'
This image already has apt-get  (otherwise it will be not very useful)
>
> And if the idea is to fetch a prebuilt docker image, and we're running
> it with docker run --privilege, I'm not sure what value using docker
> run is really providing.  We're not using the docker container
> features.   And typing
>
> docker run -i -t --privileged ... \
> 	"kvm-xfstests.sh --kernel /tmp/bzImage --update-files --update-xfstests-tar  smoke"
>
> Is just *awkward*.  The whole point is to let kvm-xfstests read the
> kernel from your ~/.config/gce-xfstests, so your work flow can be
> something like this:
>
Actually main idea is to have build environment for xfstests-bld development
plus automatic builds gives us CI. This is important because
xfstests-blk build is silently broken since May 30 due to 82c3a179
(fix proposed: https://github.com/dmonakhov/xfsprogs/commit/5dc7a0805d6b7a265863413a8e1b7acb191d5280)
But this was uncovered because most people do not build xfstests-bld
from scratch very often, we all too busy. Let's machine do that :)
In fact other good candidate for CI is https://travis-ci.com/

Once we have Dockerfile which encapsulates all dependencies
one can create xfstets-blk from scratch simply typing
$ docker build -t my-xfstests-bld-image --file Dockerfile
Or use precreated environment
$ docker pull XXX/xfstests-bld

Once image is created 
Log in to container and start hacking xfstest or e2fsprogs etc.
$ docker run -i -t XXX/xfstests-bld
$ emacs xfstest/tests/ext4/023 ....
$ kvm-xfstests --update-xfstest ext4/023
...

Or run push your kernel patches to container via bindmount
and then submit it to gce.
$ docker run -v my-stuff:/my-stuff XXX/xfstests-bld \
                git clone kernel.git && \
                cd kernel && \
                git am -s /my-stuff/patch.mbox && \
                make O=/build/ext4 -j16 && \
                cp /build/ext4/arch/x86/boot/bzImage /my-stuff && \
                cp my-stuff/gce-config ~/.config/gce-config && \
                gce-xfstests.sh --kernel /my-stuff/bzImage smoke

Other useful feature is ability to speed xfstests-bld by splitting
full test-set to groups and run kvm-xfstests instances in parallel.
For example:
docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
       kvm-xfstests.sh --kernel /my-kernel/bzImage -c 1k > 1k.log &
docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
       kvm-xfstests.sh --kernel /my-kernel/bzImage -c 4k > 4k.log &
docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
       kvm-xfstests.sh --kernel /my-kernel/bzImage -c dax > dax.log &
.....
w/o sand box they will conflict due to net ports (localhost:7500 etc)
In my case speed up is about x10
> git am -s ~/mbox/patch.mbox
> ../make-ext4      # shortcut for something like "make O=/build/ext4 -j16"
> kvm-xfstests smoke   # ~/.config/gce-xfststs sets GCE_KERNEL=/build/ext4
>
> needing to type a long docker run command doesn't seem to add any
> value.
>
> Confused,
>
> 					- Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html