lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <aee69c28-2cd1-1b0c-05db-88697db2dbd2@lucas.inf.br>
Date:   Tue, 15 Jun 2021 09:49:56 -0300
From:   Lucas Bocchi <lucas@...as.inf.br>
To:     netdev@...r.kernel.org
Subject: Strange dropped packets results in TC

Hello Folks

We have a little doubt about dropped packets in tc util. As you see, my 
interface doesn't show any dropped packets in ethtool or in kernel 
statistics, but in tc is showed dropped packets. And that ocurrs in two 
different servers, with two different network interface / queue disciplines.

The kernel

Server 1 - Intel IXGBE Driver with Intel Corporation 82599ES 10-Gigabit 
SFI/SFP+ Network Card and TP-LINK TL-SM321B Transceiver

Transceiver Statistics:

  ethtool -m enp1s0f0
         Identifier                                : 0x03 (SFP)
         Extended identifier                       : 0x04 (GBIC/SFP 
defined by 2-wire interface ID)
         Connector                                 : 0x07 (LC)
         Transceiver codes                         : 0x00 0x00 0x00 0x02 
0x82 0x00 0x01 0x01 0x00
         Transceiver type                          : Ethernet: 1000BASE-LX
         Transceiver type                          : FC: very long 
distance (V)
         Transceiver type                          : FC: Longwave laser (LC)
         Transceiver type                          : FC: Single Mode (SM)
         Transceiver type                          : FC: 100 MBytes/sec
         Encoding                                  : 0x01 (8B/10B)
         BR, Nominal                               : 1300MBd
         Rate identifier                           : 0x00 (unspecified)
         Length (SMF,km)                           : 20km
         Length (SMF)                              : 20000m
         Length (50um)                             : 0m
         Length (62.5um)                           : 0m
         Length (Copper)                           : 0m
         Length (OM3)                              : 0m
         Laser wavelength                          : 1310nm
         Vendor name                               : TP-LINK
         Vendor OUI                                : 00:00:00
         Vendor PN                                 : TL-SM321B
         Vendor rev                                : 1.1
         Option values                             : 0x00 0x1a
         Option                                    : RX_LOS implemented
         Option                                    : TX_FAULT implemented
         Option                                    : TX_DISABLE implemented
         BR margin, max                            : 0%
         BR margin, min                            : 0%
         Vendor SN                                 : 35704205710058
         Date code                                 : 170517


Ethtool Statistic

NIC statistics:
      rx_packets: 1830508376
      tx_packets: 2095191306
      rx_bytes: 1559661300658
      tx_bytes: 1796376456250
      rx_pkts_nic: 1830508375
      tx_pkts_nic: 2095191306
      rx_bytes_nic: 1566983334162
      tx_bytes_nic: 1806513286636
      lsc_int: 1
      tx_busy: 0
      non_eop_descs: 0
      rx_errors: 0
      tx_errors: 0 // No TX Error
      rx_dropped: 55706
      tx_dropped: 0 // No dropped Packets
      multicast: 55706
      broadcast: 45
      rx_no_buffer_count: 0
      collisions: 0
      rx_over_errors: 0
      rx_crc_errors: 0
      rx_frame_errors: 0
      hw_rsc_aggregated: 0
      hw_rsc_flushed: 0
      fdir_match: 1335933692
      fdir_miss: 548533583
      fdir_overflow: 888
      rx_fifo_errors: 0
      rx_missed_errors: 0
      tx_aborted_errors: 0
      tx_carrier_errors: 0
      tx_fifo_errors: 0
      tx_heartbeat_errors: 0
      tx_timeout_count: 0
      tx_restart_queue: 0
      rx_length_errors: 0
      rx_long_length_errors: 0
      rx_short_length_errors: 0
      tx_flow_control_xon: 0
      rx_flow_control_xon: 0
      tx_flow_control_xoff: 0
      rx_flow_control_xoff: 0
      rx_csum_offload_errors: 87755
      alloc_rx_page: 184014
      alloc_rx_page_failed: 0
      alloc_rx_buff_failed: 0
      rx_no_dma_resources: 0
      os2bmc_rx_by_bmc: 0
      os2bmc_tx_by_bmc: 0
      os2bmc_tx_by_host: 0
      os2bmc_rx_by_host: 0
      tx_hwtstamp_timeouts: 0
      tx_hwtstamp_skipped: 0
      rx_hwtstamp_cleared: 0
      tx_ipsec: 0
      rx_ipsec: 0
      fcoe_bad_fccrc: 0
      rx_fcoe_dropped: 0
      rx_fcoe_packets: 0
      rx_fcoe_dwords: 0
      fcoe_noddp: 0
      fcoe_noddp_ext_buff: 0
      tx_fcoe_packets: 0
      tx_fcoe_dwords: 0
      tx_queue_0_packets: 293242265
      tx_queue_0_bytes: 247153468767
      tx_queue_1_packets: 288898419
      tx_queue_1_bytes: 245035641991
      tx_queue_2_packets: 291236572
      tx_queue_2_bytes: 249561212685
      tx_queue_3_packets: 39837597
      tx_queue_3_bytes: 2358074882
      tx_queue_4_packets: 307730121
      tx_queue_4_bytes: 264761824523
      tx_queue_5_packets: 317352090
      tx_queue_5_bytes: 277380973829
      tx_queue_6_packets: 300122596
      tx_queue_6_bytes: 258114228358
      tx_queue_7_packets: 256771646
      tx_queue_7_bytes: 252011031215
...
      rx_queue_0_packets: 251550997
      rx_queue_0_bytes: 212000117060
      rx_queue_1_packets: 253297759
      rx_queue_1_bytes: 210961040647
      rx_queue_2_packets: 254114246
      rx_queue_2_bytes: 218970216150
      rx_queue_3_packets: 71013743
      rx_queue_3_bytes: 23381820239
      rx_queue_4_packets: 267448221
      rx_queue_4_bytes: 236201977962
      rx_queue_5_packets: 281248119
      rx_queue_5_bytes: 232497403664
      rx_queue_6_packets: 262045046
      rx_queue_6_bytes: 223232203577
      rx_queue_7_packets: 189790245
      rx_queue_7_bytes: 202416521359
...

ip statistics

  ip -s -s addr show dev enp1s0f0
4: enp1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc multiq 
state UP group default qlen 1000
     link/ether 90:e2:ba:89:11:70 brd ff:ff:ff:ff:ff:ff
     inet 187.60.215.230/30 brd 187.60.215.231 scope global enp1s0f0
        valid_lft forever preferred_lft forever
     inet6 2804:6fc:1:10::66/126 scope global
        valid_lft forever preferred_lft forever
     inet6 fe80::92e2:baff:fe89:1170/64 scope link
        valid_lft forever preferred_lft forever
     RX: bytes  packets  errors  dropped missed  mcast
     1563726136109 1835102960 0       55846   0       55845
     RX errors: length   crc     frame   fifo    overrun
                0        0       0       0       0
     TX: bytes  packets  errors  dropped carrier collsns
     1799499580192 2099722447 0       0       0       0  // No packet 
dropped
     TX errors: aborted  fifo   window heartbeat transns
                0        0       0       0       2


tc qdisc

tc -s qdisc ls dev enp1s0f0
qdisc multiq 8003: root refcnt 65 bands 8/64
  Sent 1760376270449 bytes 2048050689 pkt (dropped 69, overlimits 0 
requeues 477)
  backlog 0b 0p requeues 477

tc -s class ls dev enp1s0f0
class multiq 8003:1 parent 8003:
  Sent 242641753495 bytes 286672005 pkt (dropped 15, overlimits 0 
requeues 0)
  backlog 0b 0p requeues 0
class multiq 8003:2 parent 8003:
  Sent 241001820689 bytes 282960042 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
class multiq 8003:3 parent 8003:
  Sent 244006535406 bytes 284363659 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
class multiq 8003:4 parent 8003:
  Sent 2290044667 bytes 39029076 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
class multiq 8003:5 parent 8003:
  Sent 260775212492 bytes 301997242 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
class multiq 8003:6 parent 8003:
  Sent 271403512839 bytes 309429699 pkt (dropped 54, overlimits 0 
requeues 0)
  backlog 0b 0p requeues 0
class multiq 8003:7 parent 8003:
  Sent 252190559164 bytes 292911989 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
class multiq 8003:8 parent 8003:
  Sent 245887513841 bytes 250404773 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0


Server 2 - Intel Corporation Ethernet Connection (12) I219-V

ethtool -S eno1
NIC statistics:
      rx_packets: 16327063
      tx_packets: 8024473
      rx_bytes: 22725903298
      tx_bytes: 3014818845
      rx_broadcast: 2598
      tx_broadcast: 1
      rx_multicast: 0
      tx_multicast: 7
      rx_errors: 0
      tx_errors: 0
      tx_dropped: 0 // No Dropped or error in TX
      multicast: 0
      collisions: 0
      rx_length_errors: 0
      rx_over_errors: 0
      rx_crc_errors: 0
      rx_frame_errors: 0
      rx_no_buffer_count: 0
      rx_missed_errors: 0
      tx_aborted_errors: 0
      tx_carrier_errors: 0
      tx_fifo_errors: 0
      tx_heartbeat_errors: 0
      tx_window_errors: 0
      tx_abort_late_coll: 0
      tx_deferred_ok: 0
      tx_single_coll_ok: 0
      tx_multi_coll_ok: 0
      tx_timeout_count: 0
      tx_restart_queue: 0
      rx_long_length_errors: 0
      rx_short_length_errors: 0
      rx_align_errors: 0
      tx_tcp_seg_good: 0
      tx_tcp_seg_failed: 0
      rx_flow_control_xon: 0
      rx_flow_control_xoff: 0
      tx_flow_control_xon: 0
      tx_flow_control_xoff: 0
      rx_csum_offload_good: 0
      rx_csum_offload_errors: 0
      rx_header_split: 0
      alloc_rx_buff_failed: 0
      tx_smbus: 0
      rx_smbus: 0
      dropped_smbus: 0
      rx_dma_failed: 0
      tx_dma_failed: 0
      rx_hwtstamp_cleared: 0
      uncorr_ecc_errors: 0
      corr_ecc_errors: 0
      tx_hwtstamp_timeouts: 0
      tx_hwtstamp_skipped: 0

ip statistics

2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo state UP 
mode DEFAULT group default qlen 1000
     link/ether a8:a1:59:5a:56:e7 brd ff:ff:ff:ff:ff:ff
     RX: bytes  packets  errors  dropped missed  mcast
     22738286184 16335559 0       2600    0       0
     RX errors: length   crc     frame   fifo    overrun
                0        0       0       0       0
     TX: bytes  packets  errors  dropped carrier collsns
     3015100201 8026007  0       0       0       0
     TX errors: aborted  fifo   window heartbeat transns
                0        0       0       0       2
     altname enp0s31f6

tc statistic

  tc -s qdisc ls dev eno1
qdisc pfifo 8002: root refcnt 2 limit 1000p
  Sent 2982635243 bytes 8027086 pkt (dropped 247, overlimits 0 requeues 
4254)
  backlog 0b 0p requeues 4254

Any tip to try to identify that problem?

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ