linux-kernel - Re: [PATCH] input: Introduce light-weight contact tracking

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 08 Nov 2010 15:57:15 +0100
From:	Henrik Rydberg <rydberg@...omail.se>
To:	Rafi Rubin <rafi@...s.upenn.edu>
CC:	Dmitry Torokhov <dmitry.torokhov@...il.com>,
	linux-input@...r.kernel.org, linux-kernel@...r.kernel.org,
	Chris Bagwell <chris@...bagwell.com>,
	Chase Douglas <chasedouglas@...il.com>,
	Takashi Iwai <tiwai@...e.de>,
	Stéphane Chatty <chatty@...c.fr>
Subject: Re: [PATCH] input: Introduce light-weight contact tracking

On 11/07/2010 11:14 PM, Rafi Rubin wrote:

> On 11/07/2010 03:18 PM, Henrik Rydberg wrote:
>> The synaptics patches are currently cooking at Bagwell's, and we
>> already discussed the API a bit. Something ntrig-ish is cooking at
>> Rubin's, and there is also the ntrig driver in Ubuntu 10.10 to
>> consider. All-in-all, it is my hope that posting this patch will
>> simplify the synchronization of our efforts.
> 
> Since Henrik brought it up, here's the tracking code I've been working on.  I
> have not run performance tests.  My goals for this code are perhaps a little
> different.
> 
> - efficient for the common case where contact ordering stays consistent
> - arbitrary number of contacts
> - motion estimation to improve tracking
> - leveraging tracking for error filtering
> 
> Also for fun, I was playing with smoothing in this particular version of the code.
> 
> For now, I'm sending this just for review and discussion.

Hi Rafi,

what tree does this patch apply to?

I think the situation with this driver is slightly cumbersome. Perhaps there is
a small conflict of interest here, but I hope we will be able to resolve it
quickly. I see little benefit in duplicating the tracking efforts. Since greedy
matching algorithms are not provably suited to the task, I think any such
algorithm should first be tested and benchmarked thoroughly, preferably in the
mtdev context.

Regarding the ntrig driver, fact is it does not work very well. Contacts are
occasionally dropped, and spurious ghosts make multitouch gestures not work
properly. There are also many parameters in the driver, which are difficult to
use in practice. Therefore, in the Ubuntu 10.10 kernel, a rather large ntrig
patch has been applied, improving those issues.

The way the data is extracted in 2.6.36 leads to many dropped contacts. The
ubuntu patch remedies this problem, so no particular logic for drops is needed.
Instead, more effort is put into removing ghost contacts, which greatly improves
the actual experience with multitouch gestures.

In order for ghost removal to work, the contacts need to be tracked. Because of
the deficiencies of the ntrig firmware, this needs to be done in software. The
ubuntu patch contains a simple greedy tracking mechanism, which did not seem
good enough to upstream, so I never upstreamed it.

With the proposed matcher (http://lkml.org/lkml/2010/11/7/122), the situation is
different. The ubuntu patch can be modified to use the matcher, which I believe
will make it ready for upstream. Since the patch also removes all parameters
from the driver, the usability aspect is improved.

With the above said, it seems to me that the simplest way forward is to modify
and upstream the ubuntu patches. Would you agree?

Regarding support for more than four fingers, there is a single device which
reports six contacts from the firmware. Getting four good contacts out of it
will be a major improvement, and IMO good enough. It seems likely that future
firmware will handle tracking properly, so this should all be a stop-gap measure
anyways.

We talked earlier about the possibility to perform automatic calibration of the
driver. This would be the last major step towards a fully functional,
maintenance-free driver.

I hope we will be able to agree on the direction of this driver, such that
duplicated work is minimized and the focus stays with the user experience of the
ntrig devices.

Thanks,
Henrik
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/