Message-ID: <YVizJvi/7+e3n55Q@marsc.168.1.7>
Date:   Sat, 2 Oct 2021 16:29:42 -0300
From:   Marcelo Schmitt <marcelo.schmitt1@...il.com>
To:     Daniel Latypov <dlatypov@...gle.com>
Cc:     brendanhiggins@...gle.com, andersonreisrosa@...il.com,
        linux-kernel@...r.kernel.org, kunit-dev@...glegroups.com
Subject: Re: [PATCH] kunit: mock: add support for function mocks with no
 parameters

Removed Andy from the CC list as we're not talking about that bug anymore.
[...]

> > > We can work on something else besides mocking if it makes more sense to the
> > > project.
> >
> > Mocking doesn't feel like an area where we can expect to see progress right now.
> > In terms of other KUnit features we know would be useful now, I think
> > it's mostly just [1] and [2], which hopefully will land in 5.16.
> 
> To be clear, if anyone thinks up a useful feature, that'd be great.
> I personally am just out of ideas at the moment, and I think so are
> Brendan and David.
> 
> We'd want to prioritize features that can improve existing tests or
> unblock known new tests.
> Mocking in the alpha version of KUnit is a case where a feature
> sounded really good on paper and had a bunch of bells and whistles
> (e.g. strict/nice/naggy mocks support, etc.) but was perhaps
> overengineered and thus failed to find a home upstream.
> 
> But I just thought of a few more things we could do in the kunit.py script.
> I think we have more room for improvement there than in the in-kernel
> part of KUnit right now, but I assume it's the more boring part for
> most people.
> 
> One thing I'd really like to see is getting code coverage to work in
> kunit.py while using QEMU.
> We have a process for doing so under UML here:
> https://lore.kernel.org/linux-kselftest/20210901190623.315736-1-rmoar@google.com/
> UML actually uses a different coverage implementation than normal, so
> there's a few things that would need to change.
> 
> We can build and run against "normal" coverage kernels pretty easily:
> 
> $ cat >qemu_coverage_kunitconfig <<EOF
> CONFIG_KUNIT=y
> CONFIG_KUNIT_EXAMPLE_TEST=y
> CONFIG_GCOV_KERNEL=y
> CONFIG_DEBUG_FS=y
> CONFIG_GCOV_PROFILE_ALL=y
> EOF
> $ ./tools/testing/kunit/kunit.py run --arch=x86_64
> --kunitconfig=qemu_coverage_kunitconfig
> 
> The problem is we'd need to copy the coverage data off the VM instead
> of just letting it shutdown when tests are done.
> If we had a userspace running, we'd basically do something like
> $ scp -r user@vm:/sys/kernel/debug/gcov .
> <some stuff to get these files in the right spot under .kunit/>
> <then we'd run lcov and genhtml, just like we do for UML>
> 
> Normal KUnit tests definitely don't want to have to have the overhead
> of running a userspace, so the implementation might look like a
> "--qemu_coverage" flag, or maybe a set of generic flags that would
> give a user enough control over the VM to do this.
> Or maybe the right answer is to not involve kunit.py at all.
> 
> Not sure if that sounds interesting to you or anyone.
> 

Sounds cool. We'll try to make it.

Thanks

> >
> > I think right now we probably need more tests written to have a better
> > idea of what else we could/should do.
> > Partly because of that, David is trying to get the ball rolling on
> > testing ext4. We're also hopeful that it'll be easier to add tests if
> > adjacent code is already tested (sharing fakes, conventions, ability
> > to copy-paste, etc.).
> >
> > [1] https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/commit/?h=kunit&id=3b29021ddd10cfb6b2565c623595bd3b02036f33
> > [2] https://lore.kernel.org/linux-kselftest/20210909001037.2842954-1-dlatypov@google.com/
> >
> >

Powered by blists - more mailing lists