linux-kernel - Re: [PATCH 7/8] USB: testusb: imported David Brownell's USB testing application

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <201004140241.47445.david-b@pacbell.net>
Date:	Wed, 14 Apr 2010 02:41:47 -0700
From:	David Brownell <david-b@...bell.net>
To:	Michal Nazarewicz <m.nazarewicz@...sung.com>
Cc:	linux-usb@...r.kernel.org, Peter Korsgaard <jacmet@...site.dk>,
	Rupesh Gujare <rupeshgujare@...il.com>,
	linux-kernel@...r.kernel.org,
	David Brownell <dbrownell@...rs.sourceforge.net>,
	Kyungmin Park <kyungmin.park@...sung.com>,
	Marek Szyprowski <m.szyprowski@...sung.com>
Subject: Re: [PATCH 7/8] USB: testusb: imported David Brownell's USB testing application

On Wednesday 07 April 2010, Michal Nazarewicz wrote:
> 
> The testusb application can be used to test various USB gadget
> that implment the source/sink intrerface.

That comment is woefully incomplete.  It's not just gadgets it exercises,
and a lot of thought went into testing streaming modes too (within limitations
of a the trivial test harness).

And more significantly ... It's part of a fairly complete test suite which exercises

- all four types of USB 2.0 data transfer,

          on both peripheral and host sides

       - - Good coverage of observed hardware and software failure modes,
           (to help ensure routine fauilts are handled well)

    -  Decent coverage of Linux-USB programmming interfaces ... both
       in-kernel and user-visible.

   - - Stress test modes

For more info, whvih *SHOULD* be referenced from wherever the
kernel tree includes any of this code:

See: http://www.linux-usb.org/usbtest/

Just throwing tools at someone without instructions can be rather
counter-productive .... they get misused, important issues ignored,
results mis-interpreteed, etc.

Note that there's a basic test plan, letting folk put
drivers (and hardware) through their paces.  THe evidence is that when
drivers behave on that whole suite, Linux won't misbehave much at all.

In fact, without such tools and a test plan, it'd be hard to have mch faith
in the drivr quality ... except as a weak and scattershot "this combination of
drivers and hardware seems OK for now".
futile

At one point there were allegedly folk working on Linux testng frameworks, but
they never seem to have looked at this (even on specific request); I'm not sure
if it was just general weakness in driver testing efforts (they're not easy to test),
lack of background (/interest?) in USB, or something else.   (As many of us know, it's
a sad truth that testing isn't all that glamorous, for all that it's essential).
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/