linux-kernel - Re: RFC: Serial port DTR/RTS

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ADB50E23-DC8B-43D0-A345-E10396A3DFD4@zytor.com>
Date: Sun, 09 Nov 2025 21:00:18 -0800
From: "H. Peter Anvin" <hpa@...or.com>
To: "Theodore Ts'o" <tytso@....edu>
CC: linux-serial@...r.kernel.org, linux-api@...r.kernel.org,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: RFC: Serial port DTR/RTS - O_NRESETDEV

On November 9, 2025 7:35:56 PM PST, Theodore Ts'o <tytso@....edu> wrote:
>On Sat, Nov 08, 2025 at 06:25:20PM -0800, H. Peter Anvin wrote:
>> 
>> The standard ESP32 configuration for its serial port is that asserting RTS#
>> even for a moment will cause a device reset, and asserting DTR# during reset
>> forces the device into boot mode. So even if you execute TIOCMSET immediately
>> after opening the device, you will have glitched the output, and only the
>> capacitance of the output will save you, in the best case.
>
>IMHO, these more esoteric use cases should involve a custom kernel
>driver which replaces the generic serial driver.  In practice, these
>things aren't really a tty, but somethiung else weird, and trying to
>do this in userspace seems really awkward.
>
>> setserial (TIOCSSERIAL) and termios (TCSETS*) both require file descriptors,
>> so that is not suitable. The 8250 driver, but *not* other serial drivers,
>> allows the setserial information to be accessed via sysfs; however, this
>> functionality is local to the 8250 driver.
>
>My suggestion of using setserial to turn on some "not really a tty;
>but some weird networking / cheap debugging hack" flag should work,
>because you would do this at boot up.  Note that the 8250
>autoconfiguration code (see drivers/tty/serial/8250/8250_port.c) is
>going to mess with DTR / RTS.  This is why I asserted that trying to
>claim that you can preserve "state" across reboots is Just Not
>Possible.
>
>If you have some weird setup where DTR or RTS is wierd to the
>"detonate the TNT" line, might I suggest that maybe we shouldn't be
>using the tty / 8250 serial driver, but it should ***really*** be a
>dedicated kernel driver?
>
>					- Ted

That is a completely unrealistic idea. And you are hardly the first one to have it. Microsoft has been trying to get rid of serial and parallel ports since the 1990s for reasons like this. 

Microsoft even have had to back off the requirement of having .ini text file "drivers" for ACM serial ports 

Yet they probably will still be with us when the 22nd century dawns, exactly *because* they are ubiquitous, supported by everything, and require no separate kernel drivers.

And these days these aren't the "esoteric" use cases at all. They are the norm.