Message-ID: <89d33974-81fa-4926-9796-31bcd6d6cdc4@arm.com>
Date: Mon, 8 Jan 2024 13:36:21 +0000
From: Aishwarya TCV <aishwarya.tcv@....com>
To: Saeed Mahameed <saeed@...nel.org>, Tariq Toukan <tariqt@...dia.com>
Cc: Saeed Mahameed <saeedm@...dia.com>, netdev@...r.kernel.org,
 Gal Pressman <gal@...dia.com>, "David S. Miller" <davem@...emloft.net>,
 Jakub Kicinski <kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>,
 Eric Dumazet <edumazet@...gle.com>, Mark Brown <broonie@...nel.org>,
 Suzuki K Poulose <suzuki.poulose@....com>
Subject: Re: [net-next 08/15] net/mlx5e: Create single netdev per SD group



On 21/12/2023 00:57, Saeed Mahameed wrote:
> From: Tariq Toukan <tariqt@...dia.com>
> 
> Integrate the SD library calls into the auxiliary_driver ops in
> preparation for creating a single netdev for the multiple devices
> belonging to the same SD group.
> 
> SD is still disabled at this stage. It is enabled by a downstream patch
> when all needed parts are implemented.
> 
> The netdev is created only when the SD group, with all its participants,
> are ready. It is later destroyed if any of the participating devices
> drops.
> 
> Signed-off-by: Tariq Toukan <tariqt@...dia.com>
> Reviewed-by: Gal Pressman <gal@...dia.com>
> Signed-off-by: Saeed Mahameed <saeedm@...dia.com>
> ---

Hi Tariq,


Currently when booting the kernel against next-master(next-20240108)
with Arm64 on Marvell Thunder X2 (TX2), the kernel is failing to probe
the network card which is resulting in boot failures for our CI (with
rootfs over NFS). I can send the full logs if required. Most other
boards seem fine.

A bisect (full log below) identified this patch as introducing the
failure. Bisected it on the tag "mlx5-updates-2023-12-20" at repo
"https://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git/".

This works fine on Linux 6.7-rc5


Sample back trace from failure:
------
<3>[   67.915121] mlx5_core 0000:0b:00.1: mlx5_cmd_out_err:808:(pid
1585): ACCESS_REG(0x805) op_mod(0x1) failed, status bad parameter(0x3),
syndrome (0x6c4d48), err(-22)
<3>[   67.915121] mlx5_core 0000:0b:00.1: mlx5_cmd_out_err:808:(pid
1585): ACCESS_REG(0x805) op_mod(0x1) failed, status bad parameter(0x3),
syndrome (0x6c4d48), err(-22)
<4>[   67.945022] mlx5_core.eth: probe of mlx5_core.eth.1 failed with
error -22
<4>[   67.945022] mlx5_core.eth: probe of mlx5_core.eth.1 failed with
error -22
------


Here is the lspci o/p for the card:
------
0b:00.0 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx]
    Subsystem: Hewlett Packard Enterprise MT27710 Family [ConnectX-4 Lx]
    Flags: bus master, fast devsel, latency 0, IRQ 30, NUMA node 0,
IOMMU group 0
    Memory at 10000000000 (64-bit, prefetchable) [size=32M]
    Expansion ROM at 43000000 [disabled] [size=1M]
    Capabilities: [60] Express Endpoint, MSI 00
    Capabilities: [48] Vital Product Data
    Capabilities: [9c] MSI-X: Enable+ Count=64 Masked-
    Capabilities: [c0] Vendor Specific Information: Len=18 <?>
    Capabilities: [40] Power Management version 3
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
    Capabilities: [180] Single Root I/O Virtualization (SR-IOV)
    Capabilities: [1c0] Secondary PCI Express
    Capabilities: [230] Access Control Services
------


Bisect log:
------
git bisect start
# good: [a39b6ac3781d46ba18193c9dbb2110f31e9bffe9] Linux 6.7-rc5
git bisect good a39b6ac3781d46ba18193c9dbb2110f31e9bffe9
# bad: [22c4640698a1d47606b5a4264a584e8046641784] net/mlx5: Implement
management PF Ethernet profile
git bisect bad 22c4640698a1d47606b5a4264a584e8046641784
# good: [f12f551b5b966ec58bfba9daa15f3cb99a92c1f9] bnxt_en: Prevent TX
timeout with a very small TX ring
git bisect good f12f551b5b966ec58bfba9daa15f3cb99a92c1f9
# good: [509afc7452707e62fb7c4bb257f111617332ffad] Merge branch
'tools-net-ynl-add-sub-message-support-to-ynl'
git bisect good 509afc7452707e62fb7c4bb257f111617332ffad
# good: [0ee28c9ae042e77100fae2cd82a54750668aafce] Merge tag
'wireless-next-2023-12-18' of
git://git.kernel.org/pub/scm/linux/kernel/git/wireless/wireless-next
git bisect good 0ee28c9ae042e77100fae2cd82a54750668aafce
# good: [29c302a2e265a356434b005155990a9e766db75d] libbpf: further
decouple feature checking logic from bpf_object
git bisect good 29c302a2e265a356434b005155990a9e766db75d
# good: [852486b35f344887786d63250946dd921a05d7e8] x86/cfi,bpf: Fix
bpf_exception_cb() signature
git bisect good 852486b35f344887786d63250946dd921a05d7e8
# good: [e37a11fca41864c9f652ff81296b82e6f65a4242] bridge: add MDB state
mask uAPI attribute
git bisect good e37a11fca41864c9f652ff81296b82e6f65a4242
# good: [bee9705c679d0df8ee099e3c5312ac76f447848a] Merge branch
'net-sched-tc-drop-reason'
git bisect good bee9705c679d0df8ee099e3c5312ac76f447848a
# good: [c82d360325112ccc512fc11a3b68cdcdf04a1478] net/mlx5: SD, Add
informative prints in kernel log
git bisect good c82d360325112ccc512fc11a3b68cdcdf04a1478
# bad: [c73a3ab8fa6e93a783bd563938d7cf00d62d5d34] net/mlx5e: Support
cross-vhca RSS
git bisect bad c73a3ab8fa6e93a783bd563938d7cf00d62d5d34
# bad: [c4fb94aa822d6c9d05fc3c5aee35c7e339061dc1] net/mlx5e: Create EN
core HW resources for all secondary devices
git bisect bad c4fb94aa822d6c9d05fc3c5aee35c7e339061dc1
# bad: [e2578b4f983cfcd47837bbe3bcdbf5920e50b2ad] net/mlx5e: Create
single netdev per SD group
git bisect bad e2578b4f983cfcd47837bbe3bcdbf5920e50b2ad
# first bad commit: [e2578b4f983cfcd47837bbe3bcdbf5920e50b2ad]
net/mlx5e: Create single netdev per SD group
------

Thanks,
Aishwarya

Powered by blists - more mailing lists