Message-ID: <aB0zLQawNrImVqPE@mail-itl>
Date: Fri, 9 May 2025 00:41:46 +0200
From: Marek Marczykowski-Górecki <marmarek@...isiblethingslab.com>
To: "Lifshits, Vitaly" <vitaly.lifshits@...el.com>
Cc: Jesse Brandeburg <jesse.brandeburg@...el.com>,
	Tony Nguyen <anthony.l.nguyen@...el.com>, netdev@...r.kernel.org,
	intel-wired-lan@...ts.osuosl.org, regressions@...ts.linux.dev,
	stable@...r.kernel.org, Sasha Levin <sashal@...nel.org>
Subject: Re: [Intel-wired-lan] [REGRESSION] e1000e heavy packet loss on
 Meteor Lake - 6.14.2

On Thu, May 08, 2025 at 09:26:18AM +0300, Lifshits, Vitaly wrote:
> 
> 
> On 4/21/2025 4:28 PM, Marek Marczykowski-Górecki wrote:
> > On Mon, Apr 21, 2025 at 03:19:12PM +0200, Marek Marczykowski-Górecki wrote:
> > > On Mon, Apr 21, 2025 at 03:44:02PM +0300, Lifshits, Vitaly wrote:
> > > > 
> > > > 
> > > > On 4/16/2025 3:43 PM, Marek Marczykowski-Górecki wrote:
> > > > > On Wed, Apr 16, 2025 at 03:09:39PM +0300, Lifshits, Vitaly wrote:
> > > > > > Can you please also share the output of ethtool -i? I would like to know the
> > > > > > NVM version that you have on your device.
> > > > > 
> > > > > driver: e1000e
> > > > > version: 6.14.1+
> > > > > firmware-version: 1.1-4
> > > > > expansion-rom-version:
> > > > > bus-info: 0000:00:1f.6
> > > > > supports-statistics: yes
> > > > > supports-test: yes
> > > > > supports-eeprom-access: yes
> > > > > supports-register-dump: yes
> > > > > supports-priv-flags: yes
> > > > > 
> > > > 
> > > > Your firmware version is not the latest, can you check with the board
> > > > manufacturer if there is a BIOS update to your system?
> > > 
> > > I can check, but still, it's a regression in the Linux driver - old
> > > kernel did work perfectly well on this hw. Maybe new driver tries to use
> > > some feature that is missing (or broken) in the old firmware?
> > 
> > A little bit of context: I'm maintaining the kernel package for a Qubes
> > OS distribution. While I can try to update firmware on my test system, I
> > have no influence on what hardware users will use this kernel, and
> > which firmware version they will use (and whether all the vendors
> > provide newer firmware at all). I cannot ship a kernel that is known
> > to break network on some devices.
> > 
> > > > Also, you mentioned that on another system this issue doesn't reproduce, do
> > > > they have the same firmware version?
> > > 
> > > The other one has also 1.1-4 firmware. And I re-checked, e1000e from
> > > 6.14.2 works fine there.
> > 
> 
> Dear Marek,
> 
> Thank you for your detailed feedback and for providing the requested
> information.
> 
> We have conducted extensive testing of this patch across multiple systems
> and have not observed any packet loss issues. Upon comparing the mentioned
> setups, we noted that while the LAN controller is similar, the CPU differs.
> We believe that the issue may be related to transitions in the CPU's low
> power states.
> 
> Consequently, we kindly request that you disable the CPU low power state
> transitions in the S0 system state and verify if the issue persists. You can
> disable this in the kernel parameters on the command line with idle=poll.
> Please note that this command is intended for debugging purposes only, as it
> may result in higher power consumption.

I tried with idle=poll, and it didn't help, I still see a lot of packet
losses. But I can also confirm that idle=poll makes the system use
significantly more power (previously at 25-30W, with this option stays
at about 42W).

Is there any other info I can provide, enable some debug features or
something?

I see the problem is with receiving packets - in my simple ping test,
the ping target sees all the echo requests (and respond to them), but
the responses aren't reaching ping back (and are not visible on tcpdump
on the problematic system either).

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

Download attachment "signature.asc" of type "application/pgp-signature" (489 bytes)

Powered by blists - more mailing lists