netdev - Re: [Intel-wired-lan] [REGRESSION] e1000e heavy packet loss on Meteor Lake

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <c918d4f5-ee53-4f64-b152-cea0f6d99c4f@molgen.mpg.de>
Date: Fri, 9 May 2025 01:13:28 +0200
From: Paul Menzel <pmenzel@...gen.mpg.de>
To: Marek Marczykowski-Górecki
 <marmarek@...isiblethingslab.com>,
 Vitaly Lifshits <vitaly.lifshits@...el.com>
Cc: Tony Nguyen <anthony.l.nguyen@...el.com>,
 Przemek Kitszel <przemyslaw.kitszel@...el.com>, netdev@...r.kernel.org,
 intel-wired-lan@...ts.osuosl.org, regressions@...ts.linux.dev,
 stable@...r.kernel.org, Sasha Levin <sashal@...nel.org>
Subject: Re: [Intel-wired-lan] [REGRESSION] e1000e heavy packet loss on Meteor
 Lake - 6.14.2

Dear Marek, dear Vitaly,


Am 09.05.25 um 00:41 schrieb Marek Marczykowski-Górecki:
> On Thu, May 08, 2025 at 09:26:18AM +0300, Lifshits, Vitaly 
>> On 4/21/2025 4:28 PM, Marek Marczykowski-Górecki wrote:
>>> On Mon, Apr 21, 2025 at 03:19:12PM +0200, Marek Marczykowski-Górecki wrote:
>>>> On Mon, Apr 21, 2025 at 03:44:02PM +0300, Lifshits, Vitaly wrote:
>>>>>
>>>>>
>>>>> On 4/16/2025 3:43 PM, Marek Marczykowski-Górecki wrote:
>>>>>> On Wed, Apr 16, 2025 at 03:09:39PM +0300, Lifshits, Vitaly wrote:
>>>>>>> Can you please also share the output of ethtool -i? I would like to know the
>>>>>>> NVM version that you have on your device.
>>>>>>
>>>>>> driver: e1000e
>>>>>> version: 6.14.1+
>>>>>> firmware-version: 1.1-4
>>>>>> expansion-rom-version:
>>>>>> bus-info: 0000:00:1f.6
>>>>>> supports-statistics: yes
>>>>>> supports-test: yes
>>>>>> supports-eeprom-access: yes
>>>>>> supports-register-dump: yes
>>>>>> supports-priv-flags: yes
>>>>>>
>>>>>
>>>>> Your firmware version is not the latest, can you check with the board
>>>>> manufacturer if there is a BIOS update to your system?
>>>>
>>>> I can check, but still, it's a regression in the Linux driver - old
>>>> kernel did work perfectly well on this hw. Maybe new driver tries to use
>>>> some feature that is missing (or broken) in the old firmware?
>>>
>>> A little bit of context: I'm maintaining the kernel package for a Qubes
>>> OS distribution. While I can try to update firmware on my test system, I
>>> have no influence on what hardware users will use this kernel, and
>>> which firmware version they will use (and whether all the vendors
>>> provide newer firmware at all). I cannot ship a kernel that is known
>>> to break network on some devices.
>>>
>>>>> Also, you mentioned that on another system this issue doesn't reproduce, do
>>>>> they have the same firmware version?
>>>>
>>>> The other one has also 1.1-4 firmware. And I re-checked, e1000e from
>>>> 6.14.2 works fine there.

>> Thank you for your detailed feedback and for providing the requested
>> information.
>>
>> We have conducted extensive testing of this patch across multiple systems
>> and have not observed any packet loss issues. Upon comparing the mentioned
>> setups, we noted that while the LAN controller is similar, the CPU differs.
>> We believe that the issue may be related to transitions in the CPU's low
>> power states.
>>
>> Consequently, we kindly request that you disable the CPU low power state
>> transitions in the S0 system state and verify if the issue persists. You can
>> disable this in the kernel parameters on the command line with idle=poll.
>> Please note that this command is intended for debugging purposes only, as it
>> may result in higher power consumption.
> 
> I tried with idle=poll, and it didn't help, I still see a lot of packet
> losses. But I can also confirm that idle=poll makes the system use
> significantly more power (previously at 25-30W, with this option stays
> at about 42W).
> 
> Is there any other info I can provide, enable some debug features or
> something?
> 
> I see the problem is with receiving packets - in my simple ping test,
> the ping target sees all the echo requests (and respond to them), but
> the responses aren't reaching ping back (and are not visible on tcpdump
> on the problematic system either).

As the cause is still unclear, can the commit please be reverted in the 
master branch due adhere to Linux’ no-regression policy, so that it can 
be reverted from the stable series?

Marek, did you also test 6.15 release candidates?


Kind regards,

Paul