linux-kernel - xhci_reset_endpoint() doesn't reset endpoint

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <7baa1154-5c0e-4c07-88e7-3e532404b69a@default>
Date:   Wed, 14 Dec 2016 02:58:47 -0800 (PST)
From:   Michal Necasek <michal.necasek@...cle.com>
To:     <mathias.nyman@...el.com>
Cc:     <linux-usb@...r.kernel.org>, <linux-kernel@...r.kernel.org>,
        Dennis Wassenberg <dennis.wassenberg@...unet.com>
Subject: xhci_reset_endpoint() doesn't reset endpoint

    Hi Mathias,

 We have run into a problem with a USB printer which we're quite confident is a bug in the Linux xHCI driver. There is no problem when the same printer is plugged into a port managed by the EHCI driver.

 The core problem is that xhci_reset_endpoint() doesn't do anything, and more specifically does not reset the xHC's data toggle/sequence number. That is not normally an issue, because the reset does happen in response to a STALL; in our scenario, there is no STALL or any other error. That can lead to the data toggle getting out of sync and the host dropping a packet sent by the device.

 Now a detailed problem description. We have a USB printer passed through to a VM. The VM runs Windows 8.1 or 10 (other versions may be affected too), and uses Microsoft's standard usbprint.sys to talk to the printer. The vendor printer driver tries to query the printer's configuration, using the control endpoint, one OUT endpoint, and one IN endpoint. The query always times out/fails when printer is plugged into a port managed by xHCI, yet works in EHCI ports.

 The usbprint.sys driver is a bit funny and in many cases (though not always) queues up URBs on the IN endpoint in advance, and once it decides that it has received the entire response, cancels the last URB and resets the IN endpoint (issuing SetFeature(CLEAR_HALT)). After much head scratching, we realized, and later confirmed with a USB analyzer, that the next IN packet that the printer sends is not seen by the host's USB stack at all, let alone the guest OS. Other packets arrive just fine, but the guest OS keeps waiting for more data to arrive, eventually loses patience and fails.

 We cannot observe the data toggle state of the xHC but we are fairly certain that things go wrong when the data toggle is set (on both ends) prior to the endpoint reset. SetFeature(CLEAR_HALT) resets the toggle on the device, but not on the host. But we know for a fact that the device sends a packet (with data toggle 0) which the host USB stack never sees, and a data toggle mismatch explains that quite well.

 We are using USBFS to talk to the printer, but that shouldn't matter much. I will note that the available documentation<1> explicitly says that USBDEVFS_RESETEP and USBDEVFS_CLEAR_HALT both reset the data toggle. That is indeed the case for the Linux EHCI driver but not xHCI. Both of the USBFS IOCTLs call into xhci_reset_endpoint() which does nothing.

 We believe that xhci_reset_endpoint() needs to reset the data toggle/sequence number to match the documentation and for compatibility with the EHCI driver. We tried but failed to find a workaround which would reset the data toggle without side effects (e.g. USBDEVFS_SETINTERFACE does reset the toggle on the IN endpoint, but also resets it on the OUT endpoint and talks to the device, so that's no good).

 The data toggle management is not terribly well documented in the xHCI spec so we hope you know about it more than we do. Based on our understanding of the xHCI specification, xhci_reset_endpoint() should issue either a Reset Endpoint command with TSP=0 or a dummy Configure Endpoint command dropping/re-adding the specified endpoint (as the xHCI 1.1 spec suggests at the end of 4.6.8). Please confirm if that should solve the problem.

 We don't know how many devices this problem affects. We suspect it affects many USB printers and could in theory affect more or less any device, but few drivers reset endpoints when there are no errors. The problem scenario can probably be artificially reproduced with more or less any USB device (when data toggle is set, issue USBDEVFS_CLEAR_HALT, see if next packet arrives at destination).

      Regards,
         Michal

1:
  https://www.kernel.org/doc/htmldocs/usb/usbfs-ioctl.html