linux-kernel - [PATCH] USB: serial: cp210x: Fix error 32 when hardware flow control is enabled.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <658ABD80-572C-44BD-8DBD-79396109A175@silabs.com>
Date:   Tue, 19 Jan 2021 09:43:13 +0000
From:   Pho Tran <Pho.Tran@...abs.com>
To:     "johan@...nel.org" <johan@...nel.org>,
        "gregkh@...uxfoundation.org" <gregkh@...uxfoundation.org>
CC:     "inux-usb@...r.kernel.org" <inux-usb@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: [PATCH] USB: serial: cp210x: Fix error 32 when hardware flow control
  is enabled.

Fix error 32 returned by CP210X_SET_MHS when hardware flow control is enabled.

The root cause of error 32 is that user application (CoolTerm, linux-serial-test)
opened cp210x device with hardware flow control then attempt to control RTS/DTR pins.
In hardware flow control, RTS/DTR pins will be controlled by hardware only,
any attempt to control those pin will cause error 32 from the device.
This fix will block MHS command(command to control RTS/DTR pins) to the device
if hardware flow control is being used.

Signed-off-by: Pho Tran <pho.tran@...abs.com>
---
 drivers/usb/serial/cp210x.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/drivers/usb/serial/cp210x.c b/drivers/usb/serial/cp210x.c
index fbb10dfc56e3..1835ccf2aa2f 100644
--- a/drivers/usb/serial/cp210x.c
+++ b/drivers/usb/serial/cp210x.c
@@ -1211,6 +1211,11 @@ static int cp210x_tiocmset_port(struct usb_serial_port *port,
 		unsigned int set, unsigned int clear)
 {
 	u16 control = 0;
+	struct cp210x_flow_ctl flow_ctl;
+	u32 ctl_hs = 0;
+	u32 flow_repl = 0;
+	bool auto_dtr = false;
+	bool auto_rts = false;
 
 	if (set & TIOCM_RTS) {
 		control |= CONTROL_RTS;
@@ -1230,6 +1235,25 @@ static int cp210x_tiocmset_port(struct usb_serial_port *port,
 	}
 
 	dev_dbg(&port->dev, "%s - control = 0x%.4x\n", __func__, control);
+	//Don't send MHS command if device in hardware flowcontrol mode
+	cp210x_read_reg_block(port, CP210X_GET_FLOW, &flow_ctl, sizeof(flow_ctl));
+	ctl_hs = le32_to_cpu(flow_ctl.ulControlHandshake);
+	flow_repl = le32_to_cpu(flow_ctl.ulFlowReplace);
+
+	if (CP210X_SERIAL_DTR_SHIFT(CP210X_SERIAL_DTR_FLOW_CTL) == (ctl_hs & CP210X_SERIAL_DTR_MASK))
+		auto_dtr = true;
+	else
+		auto_dtr = false;
+
+	if (CP210X_SERIAL_RTS_SHIFT(CP210X_SERIAL_RTS_FLOW_CTL) == (flow_repl & CP210X_SERIAL_RTS_MASK))
+		auto_rts = true;
+	else
+		auto_rts = false;
+
+	if (auto_dtr || auto_rts) {
+		dev_dbg(&port->dev, "Don't set MHS when while device in flow control mode\n");
+		return 0;
+	}
 
 	return cp210x_write_u16_reg(port, CP210X_SET_MHS, control);
 }
-- 
2.17.1