lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <69c11e38-da50-15a3-2dfc-bc47ccc134b9@itcare.pl>
Date:   Sat, 10 Nov 2018 20:56:02 +0100
From:   Paweł Staszewski <pstaszewski@...are.pl>
To:     Jesper Dangaard Brouer <brouer@...hat.com>
Cc:     Saeed Mahameed <saeedm@...lanox.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: Re: Kernel 4.19 network performance - forwarding/routing normal users
 traffic



W dniu 10.11.2018 o 20:49, Paweł Staszewski pisze:
>
>
> W dniu 10.11.2018 o 20:34, Jesper Dangaard Brouer pisze:
>> On Fri, 9 Nov 2018 23:20:38 +0100 Paweł Staszewski 
>> <pstaszewski@...are.pl> wrote:
>>
>>> W dniu 08.11.2018 o 20:12, Paweł Staszewski pisze:
>>>> CPU load is lower than for connectx4 - but it looks like bandwidth
>>>> limit is the same :)
>>>> But also after reaching 60Gbit/60Gbit
>>>>
>>>>   bwm-ng v0.6.1 (probing every 1.000s), press 'h' for help
>>>>    input: /proc/net/dev type: rate
>>>>    -         iface                   Rx Tx Total
>>>> ========================================================================== 
>>>>
>>>>
>>>>           enp175s0:          45.09 Gb/s           15.09 Gb/s     
>>>> 60.18 Gb/s
>>>>           enp216s0:          15.14 Gb/s           45.19 Gb/s     
>>>> 60.33 Gb/s
>>>> -------------------------------------------------------------------------- 
>>>>
>>>>
>>>>              total:          60.45 Gb/s           60.48 Gb/s 120.93 
>>>> Gb/s
>>> Today reached 65/65Gbit/s
>>>
>>> But starting from 60Gbit/s RX / 60Gbit TX nics start to drop packets
>>> (with 50%CPU on all 28cores) - so still there is cpu power to use :).
>> This is weird!
>>
>> How do you see / measure these drops?
> Simple icmp test like ping -i 0.1
> And im testing by icmp management ip address on vlan that is attacked 
> to one NIC (the side that is more stressed with RX)
> And another icmp test is forward thru this router - host behind it
>
> Both measurements shows same loss ratio from 0.1 to 0.5% after 
> reaching ~45Gbit/s RX side - depends how much RX side is pushed drops 
> vary between 0.1 to 0.5 - even 0.6%:)
>
>
>>
>>
>>> So checked other stats.
>>> softnet_stats shows average 1k squeezed per sec:
>> Is below output the raw counters? not per sec?
>>
>> It would be valuable to see the per sec stats instead...
>> I use this tool:
>> https://github.com/netoptimizer/network-testing/blob/master/bin/softnet_stat.pl
CPU          total/sec     dropped/sec    squeezed/sec 
collision/sec      rx_rps/sec  flow_limit/sec
CPU:00               0               0               0 0               
0               0
CPU:01               0               0               0 0               
0               0
CPU:02               0               0               0 0               
0               0
CPU:03               0               0               0 0               
0               0
CPU:04               0               0               0 0               
0               0
CPU:05               0               0               0 0               
0               0
CPU:06               0               0               0 0               
0               0
CPU:07               0               0               0 0               
0               0
CPU:08               0               0               0 0               
0               0
CPU:09               0               0               0 0               
0               0
CPU:10               0               0               0 0               
0               0
CPU:11               0               0               0 0               
0               0
CPU:12               0               0               0 0               
0               0
CPU:13               0               0               0 0               
0               0
CPU:14          485538               0              43 0               
0               0
CPU:15          474794               0              51 0               
0               0
CPU:16          449322               0              41 0               
0               0
CPU:17          476420               0              46 0               
0               0
CPU:18          440436               0              38 0               
0               0
CPU:19          501499               0              49 0               
0               0
CPU:20          459468               0              49 0               
0               0
CPU:21          438928               0              47 0               
0               0
CPU:22          468983               0              40 0               
0               0
CPU:23          446253               0              47 0               
0               0
CPU:24          451909               0              46 0               
0               0
CPU:25          479373               0              55 0               
0               0
CPU:26          467848               0              49 0               
0               0
CPU:27          453153               0              51 0               
0               0
CPU:28               0               0               0 0               
0               0
CPU:29               0               0               0 0               
0               0
CPU:30               0               0               0 0               
0               0
CPU:31               0               0               0 0               
0               0
CPU:32               0               0               0 0               
0               0
CPU:33               0               0               0 0               
0               0
CPU:34               0               0               0 0               
0               0
CPU:35               0               0               0 0               
0               0
CPU:36               0               0               0 0               
0               0
CPU:37               0               0               0 0               
0               0
CPU:38               0               0               0 0               
0               0
CPU:39               0               0               0 0               
0               0
CPU:40               0               0               0 0               
0               0
CPU:41               0               0               0 0               
0               0
CPU:42          466853               0              43 0               
0               0
CPU:43          453059               0              54 0               
0               0
CPU:44          363219               0              34 0               
0               0
CPU:45          353632               0              38 0               
0               0
CPU:46          371618               0              40 0               
0               0
CPU:47          350518               0              46 0               
0               0
CPU:48          397544               0              40 0               
0               0
CPU:49          364873               0              38 0               
0               0
CPU:50          383630               0              38 0               
0               0
CPU:51          358771               0              39 0               
0               0
CPU:52          372547               0              38 0               
0               0
CPU:53          372882               0              36 0               
0               0
CPU:54          366244               0              43 0               
0               0
CPU:55          365886               0              39 0               
0               0

Summed:       11835201               0            1217 0               
0               0





>>
>>> cpu      total    dropped   squeezed collision        rps flow_limit
>>>     0      18554          0          1          0          0 0
>>>     1      16728          0          1          0          0 0
>>>     2      18033          0          1          0          0 0
>>>     3      17757          0          1          0          0 0
>>>     4      18861          0          0          0          0 0
>>>     5          0          0          1          0          0 0
>>>     6          2          0          1          0          0 0
>>>     7          0          0          1          0          0 0
>>>     8          0          0          0          0          0 0
>>>     9          0          0          1          0          0 0
>>>    10          0          0          0          0          0 0
>>>    11          0          0          1          0          0 0
>>>    12         50          0          1          0          0 0
>>>    13        257          0          0          0          0 0
>>>    14 3629115363          0    3353259          0          0 0
>>>    15  255167835          0    3138271          0          0 0
>>>    16 4240101961          0    3036130          0          0 0
>>>    17  599810018          0    3072169          0          0 0
>>>    18  432796524          0    3034191          0          0 0
>>>    19   41803906          0    3037405          0          0 0
>>>    20  900382666          0    3112294          0          0 0
>>>    21  620926085          0    3086009          0          0 0
>>>    22   41861198          0    3023142          0          0 0
>>>    23 4090425574          0    2990412          0          0 0
>>>    24 4264870218          0    3010272          0          0 0
>>>    25  141401811          0    3027153          0          0 0
>>>    26  104155188          0    3051251          0          0 0
>>>    27 4261258691          0    3039765          0          0 0
>>>    28          4          0          1          0          0 0
>>>    29          4          0          0          0          0 0
>>>    30          0          0          1          0          0 0
>>>    31          0          0          0          0          0 0
>>>    32          3          0          1          0          0 0
>>>    33          1          0          1          0          0 0
>>>    34          0          0          1          0          0 0
>>>    35          0          0          0          0          0 0
>>>    36          0          0          1          0          0 0
>>>    37          0          0          1          0          0 0
>>>    38          0          0          1          0          0 0
>>>    39          0          0          1          0          0 0
>>>    40          0          0          0          0          0 0
>>>    41          0          0          1          0          0 0
>>>    42  299758202          0    3139693          0          0 0
>>>    43 4254727979          0    3103577          0          0 0
>>>    44 1959555543          0    2554885          0          0 0
>>>    45 1675702723          0    2513481          0          0 0
>>>    46 1908435503          0    2519698          0          0 0
>>>    47 1877799710          0    2537768          0          0 0
>>>    48 2384274076          0    2584673          0          0 0
>>>    49 2598104878          0    2593616          0          0 0
>>>    50 1897566829          0    2530857          0          0 0
>>>    51 1712741629          0    2489089          0          0 0
>>>    52 1704033648          0    2495892          0          0 0
>>>    53 1636781820          0    2499783          0          0 0
>>>    54 1861997734          0    2541060          0          0 0
>>>    55 2113521616          0    2555673          0          0 0
>>>
>>>
>>> So i rised netdev backlog and budged to rly high values
>>> 524288 for netdev_budget and same for backlog
>> Does it affect the squeezed counters?
> a little - but not much
> After change budget from 65536 to to 524k - number of squeezed 
> counters for all cpus changed from 1.5k per second to 0.9-1k per 
> second - but increasing it more like above 524k change nothing - same 
> 0.9 to 1k/s squeezed
>>
>> Notice, this (crazy) huge netdev_budget limit will also be limited
>> by /proc/sys/net/core/netdev_budget_usecs.
> Yes changed that also to 1000 / 2000 / 3000 / 4000  not much 
> difference on squeezed - even cant see the difference
>
>>
>>> This rised sortirqs from about 600k/sec to 800k/sec for NET_TX/NET_RX
>> Hmmm, this could indicated not enough NAPI bulking is occurring.
>>
>> I have a BPF tool, that can give you some insight into NAPI bulking and
>> softirq idle/kthread starting. Called 'napi_monitor', could you try to
>> run this, so can try to understand this? You find the tool here:
>>
>> https://github.com/netoptimizer/prototype-kernel/blob/master/kernel/samples/bpf/
>> https://github.com/netoptimizer/prototype-kernel/blob/master/kernel/samples/bpf/napi_monitor_user.c
>> https://github.com/netoptimizer/prototype-kernel/blob/master/kernel/samples/bpf/napi_monitor_kern.c
> yes will try it
>
>>> But after this changes i have less packets drops.
>>>
>>>
>>> Below perf top from max traffic reached:
>>>      PerfTop:   72230 irqs/sec  kernel:99.4%  exact:  0.0% [4000Hz
>>> cycles],  (all, 56 CPUs)
>>> ------------------------------------------------------------------------------------------ 
>>>
>>>
>>>       12.62%  [kernel]       [k] mlx5e_skb_from_cqe_mpwrq_linear
>>>        8.44%  [kernel]       [k] mlx5e_sq_xmit
>>>        6.69%  [kernel]       [k] build_skb
>>>        5.21%  [kernel]       [k] fib_table_lookup
>>>        3.54%  [kernel]       [k] memcpy_erms
>>>        3.20%  [kernel]       [k] mlx5e_poll_rx_cq
>>>        2.25%  [kernel]       [k] vlan_do_receive
>>>        2.20%  [kernel]       [k] mlx5e_post_rx_mpwqes
>>>        2.02%  [kernel]       [k] mlx5e_handle_rx_cqe_mpwrq
>>>        1.95%  [kernel]       [k] __dev_queue_xmit
>>>        1.83%  [kernel]       [k] dev_gro_receive
>>>        1.79%  [kernel]       [k] tcp_gro_receive
>>>        1.73%  [kernel]       [k] ip_finish_output2
>>>        1.63%  [kernel]       [k] mlx5e_poll_tx_cq
>>>        1.49%  [kernel]       [k] ipt_do_table
>>>        1.38%  [kernel]       [k] inet_gro_receive
>>>        1.31%  [kernel]       [k] __netif_receive_skb_core
>>>        1.30%  [kernel]       [k] _raw_spin_lock
>>>        1.28%  [kernel]       [k] mlx5_eq_int
>>>        1.24%  [kernel]       [k] irq_entries_start
>>>        1.19%  [kernel]       [k] __build_skb
>>>        1.15%  [kernel]       [k] swiotlb_map_page
>>>        1.02%  [kernel]       [k] vlan_dev_hard_start_xmit
>>>        0.94%  [kernel]       [k] pfifo_fast_dequeue
>>>        0.92%  [kernel]       [k] ip_route_input_rcu
>>>        0.86%  [kernel]       [k] kmem_cache_alloc
>>>        0.80%  [kernel]       [k] mlx5e_xmit
>>>        0.79%  [kernel]       [k] dev_hard_start_xmit
>>>        0.78%  [kernel]       [k] _raw_spin_lock_irqsave
>>>        0.74%  [kernel]       [k] ip_forward
>>>        0.72%  [kernel]       [k] tasklet_action_common.isra.21
>>>        0.68%  [kernel]       [k] pfifo_fast_enqueue
>>>        0.67%  [kernel]       [k] netif_skb_features
>>>        0.66%  [kernel]       [k] skb_segment
>>>        0.60%  [kernel]       [k] skb_gro_receive
>>>        0.56%  [kernel]       [k] validate_xmit_skb.isra.142
>>>        0.53%  [kernel]       [k] skb_release_data
>>>        0.51%  [kernel]       [k] mlx5e_page_release
>>>        0.51%  [kernel]       [k] ip_rcv_core.isra.20.constprop.25
>>>        0.51%  [kernel]       [k] __qdisc_run
>>>        0.50%  [kernel]       [k] tcp4_gro_receive
>>>        0.49%  [kernel]       [k] page_frag_free
>>>        0.46%  [kernel]       [k] kmem_cache_free_bulk
>>>        0.43%  [kernel]       [k] kmem_cache_free
>>>        0.42%  [kernel]       [k] try_to_wake_up
>>>        0.39%  [kernel]       [k] _raw_spin_lock_irq
>>>        0.39%  [kernel]       [k] find_busiest_group
>>>        0.37%  [kernel]       [k] __memcpy
>>>
>>>
>>>
>>> Remember those tests are now on two separate connectx5 connected to
>>> two separate pcie x16  gen 3.0
>>   That is strange... I still suspect some HW NIC issue, can you provide
>> ethtool stats info via tool:
>>
>> https://github.com/netoptimizer/network-testing/blob/master/bin/ethtool_stats.pl
>>
>> $ ethtool_stats.pl --dev enp175s0 --dev enp216s0
>>
>> The tool remove zero-stats counters and report per sec stats. It makes
>> it easier to spot that is relevant for the given workload.
> yes mlnx have just too many counters that are always 0 for my case :)
> Will try this also
>
But still alot of non 0 counters
Show adapter(s) (enp175s0 enp216s0) statistics (ONLY that changed!)
Ethtool(enp175s0) stat:         8891 (          8,891) <= ch0_arm /sec
Ethtool(enp175s0) stat:        10265 (         10,265) <= ch0_events /sec
Ethtool(enp175s0) stat:        11072 (         11,072) <= ch0_poll /sec
Ethtool(enp175s0) stat:         9003 (          9,003) <= ch10_arm /sec
Ethtool(enp175s0) stat:        10476 (         10,476) <= ch10_events /sec
Ethtool(enp175s0) stat:        11284 (         11,284) <= ch10_poll /sec
Ethtool(enp175s0) stat:        11211 (         11,211) <= ch11_arm /sec
Ethtool(enp175s0) stat:        12645 (         12,645) <= ch11_events /sec
Ethtool(enp175s0) stat:        13450 (         13,450) <= ch11_poll /sec
Ethtool(enp175s0) stat:         9012 (          9,012) <= ch12_arm /sec
Ethtool(enp175s0) stat:        10366 (         10,366) <= ch12_events /sec
Ethtool(enp175s0) stat:        11074 (         11,074) <= ch12_poll /sec
Ethtool(enp175s0) stat:         8810 (          8,810) <= ch13_arm /sec
Ethtool(enp175s0) stat:        10177 (         10,177) <= ch13_events /sec
Ethtool(enp175s0) stat:        10886 (         10,886) <= ch13_poll /sec
Ethtool(enp175s0) stat:         9794 (          9,794) <= ch14_arm /sec
Ethtool(enp175s0) stat:        11159 (         11,159) <= ch14_events /sec
Ethtool(enp175s0) stat:        11932 (         11,932) <= ch14_poll /sec
Ethtool(enp175s0) stat:         8703 (          8,703) <= ch15_arm /sec
Ethtool(enp175s0) stat:        10052 (         10,052) <= ch15_events /sec
Ethtool(enp175s0) stat:        10774 (         10,774) <= ch15_poll /sec
Ethtool(enp175s0) stat:         6429 (          6,429) <= ch16_arm /sec
Ethtool(enp175s0) stat:         7591 (          7,591) <= ch16_events /sec
Ethtool(enp175s0) stat:         8223 (          8,223) <= ch16_poll /sec
Ethtool(enp175s0) stat:         8981 (          8,981) <= ch17_arm /sec
Ethtool(enp175s0) stat:        10229 (         10,229) <= ch17_events /sec
Ethtool(enp175s0) stat:        10887 (         10,887) <= ch17_poll /sec
Ethtool(enp175s0) stat:         6786 (          6,786) <= ch18_arm /sec
Ethtool(enp175s0) stat:         7887 (          7,887) <= ch18_events /sec
Ethtool(enp175s0) stat:         8484 (          8,484) <= ch18_poll /sec
Ethtool(enp175s0) stat:         6080 (          6,080) <= ch19_arm /sec
Ethtool(enp175s0) stat:         7377 (          7,377) <= ch19_events /sec
Ethtool(enp175s0) stat:         8124 (          8,124) <= ch19_poll /sec
Ethtool(enp175s0) stat:         7715 (          7,715) <= ch1_arm /sec
Ethtool(enp175s0) stat:         9109 (          9,109) <= ch1_events /sec
Ethtool(enp175s0) stat:         9923 (          9,923) <= ch1_poll /sec
Ethtool(enp175s0) stat:         7303 (          7,303) <= ch20_arm /sec
Ethtool(enp175s0) stat:         8514 (          8,514) <= ch20_events /sec
Ethtool(enp175s0) stat:         9169 (          9,169) <= ch20_poll /sec
Ethtool(enp175s0) stat:         8972 (          8,972) <= ch21_arm /sec
Ethtool(enp175s0) stat:        10060 (         10,060) <= ch21_events /sec
Ethtool(enp175s0) stat:        10647 (         10,647) <= ch21_poll /sec
Ethtool(enp175s0) stat:         7729 (          7,729) <= ch22_arm /sec
Ethtool(enp175s0) stat:         8932 (          8,932) <= ch22_events /sec
Ethtool(enp175s0) stat:         9585 (          9,585) <= ch22_poll /sec
Ethtool(enp175s0) stat:         8125 (          8,125) <= ch23_arm /sec
Ethtool(enp175s0) stat:         9218 (          9,218) <= ch23_events /sec
Ethtool(enp175s0) stat:         9805 (          9,805) <= ch23_poll /sec
Ethtool(enp175s0) stat:         7212 (          7,212) <= ch24_arm /sec
Ethtool(enp175s0) stat:         8369 (          8,369) <= ch24_events /sec
Ethtool(enp175s0) stat:         8993 (          8,993) <= ch24_poll /sec
Ethtool(enp175s0) stat:         6328 (          6,328) <= ch25_arm /sec
Ethtool(enp175s0) stat:         7567 (          7,567) <= ch25_events /sec
Ethtool(enp175s0) stat:         8274 (          8,274) <= ch25_poll /sec
Ethtool(enp175s0) stat:         6210 (          6,210) <= ch26_arm /sec
Ethtool(enp175s0) stat:         7409 (          7,409) <= ch26_events /sec
Ethtool(enp175s0) stat:         8062 (          8,062) <= ch26_poll /sec
Ethtool(enp175s0) stat:         7035 (          7,035) <= ch27_arm /sec
Ethtool(enp175s0) stat:         8203 (          8,203) <= ch27_events /sec
Ethtool(enp175s0) stat:         8840 (          8,840) <= ch27_poll /sec
Ethtool(enp175s0) stat:        11278 (         11,278) <= ch2_arm /sec
Ethtool(enp175s0) stat:        12632 (         12,632) <= ch2_events /sec
Ethtool(enp175s0) stat:        13348 (         13,348) <= ch2_poll /sec
Ethtool(enp175s0) stat:        10612 (         10,612) <= ch3_arm /sec
Ethtool(enp175s0) stat:        11900 (         11,900) <= ch3_events /sec
Ethtool(enp175s0) stat:        12567 (         12,567) <= ch3_poll /sec
Ethtool(enp175s0) stat:         8936 (          8,936) <= ch4_arm /sec
Ethtool(enp175s0) stat:        10248 (         10,248) <= ch4_events /sec
Ethtool(enp175s0) stat:        10962 (         10,962) <= ch4_poll /sec
Ethtool(enp175s0) stat:        11631 (         11,631) <= ch5_arm /sec
Ethtool(enp175s0) stat:        12953 (         12,953) <= ch5_events /sec
Ethtool(enp175s0) stat:        13629 (         13,629) <= ch5_poll /sec
Ethtool(enp175s0) stat:         9877 (          9,877) <= ch6_arm /sec
Ethtool(enp175s0) stat:        11114 (         11,114) <= ch6_events /sec
Ethtool(enp175s0) stat:        11800 (         11,800) <= ch6_poll /sec
Ethtool(enp175s0) stat:         8228 (          8,228) <= ch7_arm /sec
Ethtool(enp175s0) stat:         9577 (          9,577) <= ch7_events /sec
Ethtool(enp175s0) stat:        10320 (         10,320) <= ch7_poll /sec
Ethtool(enp175s0) stat:        11808 (         11,808) <= ch8_arm /sec
Ethtool(enp175s0) stat:        13135 (         13,135) <= ch8_events /sec
Ethtool(enp175s0) stat:        13828 (         13,828) <= ch8_poll /sec
Ethtool(enp175s0) stat:        10566 (         10,566) <= ch9_arm /sec
Ethtool(enp175s0) stat:        11904 (         11,904) <= ch9_events /sec
Ethtool(enp175s0) stat:        12634 (         12,634) <= ch9_poll /sec
Ethtool(enp175s0) stat:       243256 (        243,256) <= ch_arm /sec
Ethtool(enp175s0) stat:       279057 (        279,057) <= ch_events /sec
Ethtool(enp175s0) stat:       298563 (        298,563) <= ch_poll /sec
Ethtool(enp175s0) stat:    186677525 (    186,677,525) <= rx0_bytes /sec
Ethtool(enp175s0) stat:        72870 (         72,870) <= 
rx0_cache_reuse /sec
Ethtool(enp175s0) stat:       145627 (        145,627) <= 
rx0_csum_complete /sec
Ethtool(enp175s0) stat:           88 (             88) <= rx0_csum_none /sec
Ethtool(enp175s0) stat:       145715 (        145,715) <= rx0_packets /sec
Ethtool(enp175s0) stat:       145715 (        145,715) <= 
rx0_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    198552827 (    198,552,827) <= rx10_bytes /sec
Ethtool(enp175s0) stat:        75553 (         75,553) <= 
rx10_cache_reuse /sec
Ethtool(enp175s0) stat:       151021 (        151,021) <= 
rx10_csum_complete /sec
Ethtool(enp175s0) stat:       151021 (        151,021) <= rx10_packets /sec
Ethtool(enp175s0) stat:       151021 (        151,021) <= 
rx10_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    200924148 (    200,924,148) <= rx11_bytes /sec
Ethtool(enp175s0) stat:        76589 (         76,589) <= 
rx11_cache_reuse /sec
Ethtool(enp175s0) stat:       153221 (        153,221) <= 
rx11_csum_complete /sec
Ethtool(enp175s0) stat:       153221 (        153,221) <= rx11_packets /sec
Ethtool(enp175s0) stat:       153221 (        153,221) <= 
rx11_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    186259790 (    186,259,790) <= rx12_bytes /sec
Ethtool(enp175s0) stat:        70675 (         70,675) <= 
rx12_cache_reuse /sec
Ethtool(enp175s0) stat:       141440 (        141,440) <= 
rx12_csum_complete /sec
Ethtool(enp175s0) stat:       141440 (        141,440) <= rx12_packets /sec
Ethtool(enp175s0) stat:       141440 (        141,440) <= 
rx12_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    189627451 (    189,627,451) <= rx13_bytes /sec
Ethtool(enp175s0) stat:        72626 (         72,626) <= 
rx13_cache_reuse /sec
Ethtool(enp175s0) stat:       145327 (        145,327) <= 
rx13_csum_complete /sec
Ethtool(enp175s0) stat:       145327 (        145,327) <= rx13_packets /sec
Ethtool(enp175s0) stat:       145327 (        145,327) <= 
rx13_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    199246096 (    199,246,096) <= rx14_bytes /sec
Ethtool(enp175s0) stat:        77992 (         77,992) <= 
rx14_cache_reuse /sec
Ethtool(enp175s0) stat:       156043 (        156,043) <= 
rx14_csum_complete /sec
Ethtool(enp175s0) stat:       156043 (        156,043) <= rx14_packets /sec
Ethtool(enp175s0) stat:       156043 (        156,043) <= 
rx14_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    189698176 (    189,698,176) <= rx15_bytes /sec
Ethtool(enp175s0) stat:        72382 (         72,382) <= 
rx15_cache_reuse /sec
Ethtool(enp175s0) stat:       144658 (        144,658) <= 
rx15_csum_complete /sec
Ethtool(enp175s0) stat:       144658 (        144,658) <= rx15_packets /sec
Ethtool(enp175s0) stat:       144658 (        144,658) <= 
rx15_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    143896232 (    143,896,232) <= rx16_bytes /sec
Ethtool(enp175s0) stat:        55369 (         55,369) <= 
rx16_cache_reuse /sec
Ethtool(enp175s0) stat:       110745 (        110,745) <= 
rx16_csum_complete /sec
Ethtool(enp175s0) stat:       110745 (        110,745) <= rx16_packets /sec
Ethtool(enp175s0) stat:       110745 (        110,745) <= 
rx16_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    171449483 (    171,449,483) <= rx17_bytes /sec
Ethtool(enp175s0) stat:        65308 (         65,308) <= 
rx17_cache_reuse /sec
Ethtool(enp175s0) stat:       130563 (        130,563) <= 
rx17_csum_complete /sec
Ethtool(enp175s0) stat:       130563 (        130,563) <= rx17_packets /sec
Ethtool(enp175s0) stat:       130563 (        130,563) <= 
rx17_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    141033264 (    141,033,264) <= rx18_bytes /sec
Ethtool(enp175s0) stat:        54515 (         54,515) <= 
rx18_cache_reuse /sec
Ethtool(enp175s0) stat:       108966 (        108,966) <= 
rx18_csum_complete /sec
Ethtool(enp175s0) stat:       108966 (        108,966) <= rx18_packets /sec
Ethtool(enp175s0) stat:       108966 (        108,966) <= 
rx18_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    163097410 (    163,097,410) <= rx19_bytes /sec
Ethtool(enp175s0) stat:        61894 (         61,894) <= 
rx19_cache_reuse /sec
Ethtool(enp175s0) stat:       123773 (        123,773) <= 
rx19_csum_complete /sec
Ethtool(enp175s0) stat:       123773 (        123,773) <= rx19_packets /sec
Ethtool(enp175s0) stat:       123773 (        123,773) <= 
rx19_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    181707418 (    181,707,418) <= rx1_bytes /sec
Ethtool(enp175s0) stat:        71223 (         71,223) <= 
rx1_cache_reuse /sec
Ethtool(enp175s0) stat:       142445 (        142,445) <= 
rx1_csum_complete /sec
Ethtool(enp175s0) stat:       142445 (        142,445) <= rx1_packets /sec
Ethtool(enp175s0) stat:       142445 (        142,445) <= 
rx1_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    161626368 (    161,626,368) <= rx20_bytes /sec
Ethtool(enp175s0) stat:        61345 (         61,345) <= 
rx20_cache_reuse /sec
Ethtool(enp175s0) stat:       122724 (        122,724) <= 
rx20_csum_complete /sec
Ethtool(enp175s0) stat:       122724 (        122,724) <= rx20_packets /sec
Ethtool(enp175s0) stat:       122724 (        122,724) <= 
rx20_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    138593554 (    138,593,554) <= rx21_bytes /sec
Ethtool(enp175s0) stat:        53478 (         53,478) <= 
rx21_cache_reuse /sec
Ethtool(enp175s0) stat:       106949 (        106,949) <= 
rx21_csum_complete /sec
Ethtool(enp175s0) stat:       106949 (        106,949) <= rx21_packets /sec
Ethtool(enp175s0) stat:       106949 (        106,949) <= 
rx21_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    149217722 (    149,217,722) <= rx22_bytes /sec
Ethtool(enp175s0) stat:        58174 (         58,174) <= 
rx22_cache_reuse /sec
Ethtool(enp175s0) stat:       116342 (        116,342) <= 
rx22_csum_complete /sec
Ethtool(enp175s0) stat:       116342 (        116,342) <= rx22_packets /sec
Ethtool(enp175s0) stat:       116342 (        116,342) <= 
rx22_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    147968086 (    147,968,086) <= rx23_bytes /sec
Ethtool(enp175s0) stat:        55979 (         55,979) <= 
rx23_cache_reuse /sec
Ethtool(enp175s0) stat:       111901 (        111,901) <= 
rx23_csum_complete /sec
Ethtool(enp175s0) stat:       111901 (        111,901) <= rx23_packets /sec
Ethtool(enp175s0) stat:       111901 (        111,901) <= 
rx23_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    145955524 (    145,955,524) <= rx24_bytes /sec
Ethtool(enp175s0) stat:        55491 (         55,491) <= 
rx24_cache_reuse /sec
Ethtool(enp175s0) stat:       110980 (        110,980) <= 
rx24_csum_complete /sec
Ethtool(enp175s0) stat:       110980 (        110,980) <= rx24_packets /sec
Ethtool(enp175s0) stat:       110980 (        110,980) <= 
rx24_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    155552699 (    155,552,699) <= rx25_bytes /sec
Ethtool(enp175s0) stat:        59028 (         59,028) <= 
rx25_cache_reuse /sec
Ethtool(enp175s0) stat:       118074 (        118,074) <= 
rx25_csum_complete /sec
Ethtool(enp175s0) stat:       118074 (        118,074) <= rx25_packets /sec
Ethtool(enp175s0) stat:       118074 (        118,074) <= 
rx25_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    144880442 (    144,880,442) <= rx26_bytes /sec
Ethtool(enp175s0) stat:        56223 (         56,223) <= 
rx26_cache_reuse /sec
Ethtool(enp175s0) stat:       112334 (        112,334) <= 
rx26_csum_complete /sec
Ethtool(enp175s0) stat:       112334 (        112,334) <= rx26_packets /sec
Ethtool(enp175s0) stat:       112334 (        112,334) <= 
rx26_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    154545288 (    154,545,288) <= rx27_bytes /sec
Ethtool(enp175s0) stat:        58784 (         58,784) <= 
rx27_cache_reuse /sec
Ethtool(enp175s0) stat:       117627 (        117,627) <= 
rx27_csum_complete /sec
Ethtool(enp175s0) stat:       117627 (        117,627) <= rx27_packets /sec
Ethtool(enp175s0) stat:       117627 (        117,627) <= 
rx27_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    182425129 (    182,425,129) <= rx2_bytes /sec
Ethtool(enp175s0) stat:        71406 (         71,406) <= 
rx2_cache_reuse /sec
Ethtool(enp175s0) stat:       142872 (        142,872) <= 
rx2_csum_complete /sec
Ethtool(enp175s0) stat:       142872 (        142,872) <= rx2_packets /sec
Ethtool(enp175s0) stat:       142872 (        142,872) <= 
rx2_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    188368405 (    188,368,405) <= rx3_bytes /sec
Ethtool(enp175s0) stat:        72138 (         72,138) <= 
rx3_cache_reuse /sec
Ethtool(enp175s0) stat:       144259 (        144,259) <= 
rx3_csum_complete /sec
Ethtool(enp175s0) stat:       144259 (        144,259) <= rx3_packets /sec
Ethtool(enp175s0) stat:       144259 (        144,259) <= 
rx3_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    186009984 (    186,009,984) <= rx4_bytes /sec
Ethtool(enp175s0) stat:        70004 (         70,004) <= 
rx4_cache_reuse /sec
Ethtool(enp175s0) stat:       139939 (        139,939) <= 
rx4_csum_complete /sec
Ethtool(enp175s0) stat:       139939 (        139,939) <= rx4_packets /sec
Ethtool(enp175s0) stat:       139939 (        139,939) <= 
rx4_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    198040550 (    198,040,550) <= rx5_bytes /sec
Ethtool(enp175s0) stat:        75492 (         75,492) <= 
rx5_cache_reuse /sec
Ethtool(enp175s0) stat:       150950 (        150,950) <= 
rx5_csum_complete /sec
Ethtool(enp175s0) stat:       150950 (        150,950) <= rx5_packets /sec
Ethtool(enp175s0) stat:       150950 (        150,950) <= 
rx5_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    182607101 (    182,607,101) <= rx6_bytes /sec
Ethtool(enp175s0) stat:        69699 (         69,699) <= 
rx6_cache_reuse /sec
Ethtool(enp175s0) stat:       139335 (        139,335) <= 
rx6_csum_complete /sec
Ethtool(enp175s0) stat:       139335 (        139,335) <= rx6_packets /sec
Ethtool(enp175s0) stat:       139335 (        139,335) <= 
rx6_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    174999243 (    174,999,243) <= rx7_bytes /sec
Ethtool(enp175s0) stat:        66650 (         66,650) <= 
rx7_cache_reuse /sec
Ethtool(enp175s0) stat:       133323 (        133,323) <= 
rx7_csum_complete /sec
Ethtool(enp175s0) stat:       133323 (        133,323) <= rx7_packets /sec
Ethtool(enp175s0) stat:       133323 (        133,323) <= 
rx7_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    204109286 (    204,109,286) <= rx8_bytes /sec
Ethtool(enp175s0) stat:        76711 (         76,711) <= 
rx8_cache_reuse /sec
Ethtool(enp175s0) stat:       153481 (        153,481) <= 
rx8_csum_complete /sec
Ethtool(enp175s0) stat:       153481 (        153,481) <= rx8_packets /sec
Ethtool(enp175s0) stat:       153481 (        153,481) <= 
rx8_removed_vlan_packets /sec
Ethtool(enp175s0) stat:    183703752 (    183,703,752) <= rx9_bytes /sec
Ethtool(enp175s0) stat:        71101 (         71,101) <= 
rx9_cache_reuse /sec
Ethtool(enp175s0) stat:       142172 (        142,172) <= 
rx9_csum_complete /sec
Ethtool(enp175s0) stat:       142172 (        142,172) <= rx9_packets /sec
Ethtool(enp175s0) stat:       142172 (        142,172) <= 
rx9_removed_vlan_packets /sec
Ethtool(enp175s0) stat:      2024072 (      2,024,072) <= 
rx_1024_to_1518_bytes_phy /sec
Ethtool(enp175s0) stat:       106582 (        106,582) <= 
rx_128_to_255_bytes_phy /sec
Ethtool(enp175s0) stat:      1296735 (      1,296,735) <= 
rx_1519_to_2047_bytes_phy /sec
Ethtool(enp175s0) stat:        59460 (         59,460) <= 
rx_256_to_511_bytes_phy /sec
Ethtool(enp175s0) stat:        57326 (         57,326) <= 
rx_512_to_1023_bytes_phy /sec
Ethtool(enp175s0) stat:         7159 (          7,159) <= 
rx_64_bytes_phy /sec
Ethtool(enp175s0) stat:       310730 (        310,730) <= 
rx_65_to_127_bytes_phy /sec
Ethtool(enp175s0) stat:          232 (            232) <= 
rx_broadcast_phy /sec
Ethtool(enp175s0) stat:   4850734036 (  4,850,734,036) <= rx_bytes /sec
Ethtool(enp175s0) stat:   5069043007 (  5,069,043,007) <= rx_bytes_phy /sec
Ethtool(enp175s0) stat:      1858636 (      1,858,636) <= rx_cache_reuse 
/sec
Ethtool(enp175s0) stat:      3717060 (      3,717,060) <= 
rx_csum_complete /sec
Ethtool(enp175s0) stat:           88 (             88) <= rx_csum_none /sec
Ethtool(enp175s0) stat:       139602 (        139,602) <= 
rx_discards_phy /sec
Ethtool(enp175s0) stat:          354 (            354) <= 
rx_multicast_phy /sec
Ethtool(enp175s0) stat:      3717148 (      3,717,148) <= rx_packets /sec
Ethtool(enp175s0) stat:      3862420 (      3,862,420) <= rx_packets_phy 
/sec
Ethtool(enp175s0) stat:   5063355121 (  5,063,355,121) <= rx_prio0_bytes 
/sec
Ethtool(enp175s0) stat:      3718759 (      3,718,759) <= 
rx_prio0_packets /sec
Ethtool(enp175s0) stat:      7193190 (      7,193,190) <= rx_prio1_bytes 
/sec
Ethtool(enp175s0) stat:         5031 (          5,031) <= 
rx_prio1_packets /sec
Ethtool(enp175s0) stat:          557 (            557) <= rx_prio2_bytes 
/sec
Ethtool(enp175s0) stat:            5 (              5) <= 
rx_prio2_packets /sec
Ethtool(enp175s0) stat:           61 (             61) <= rx_prio3_bytes 
/sec
Ethtool(enp175s0) stat:            1 (              1) <= 
rx_prio3_packets /sec
Ethtool(enp175s0) stat:        21010 (         21,010) <= rx_prio4_bytes 
/sec
Ethtool(enp175s0) stat:           39 (             39) <= 
rx_prio4_packets /sec
Ethtool(enp175s0) stat:          187 (            187) <= rx_prio5_bytes 
/sec
Ethtool(enp175s0) stat:            2 (              2) <= 
rx_prio5_packets /sec
Ethtool(enp175s0) stat:         1711 (          1,711) <= rx_prio6_bytes 
/sec
Ethtool(enp175s0) stat:           15 (             15) <= 
rx_prio6_packets /sec
Ethtool(enp175s0) stat:        19498 (         19,498) <= rx_prio7_bytes 
/sec
Ethtool(enp175s0) stat:          273 (            273) <= 
rx_prio7_packets /sec
Ethtool(enp175s0) stat:      3717148 (      3,717,148) <= 
rx_removed_vlan_packets /sec
Ethtool(enp175s0) stat:         5737 (          5,737) <= 
rx_steer_missed_packets /sec
Ethtool(enp175s0) stat:        14573 (         14,573) <= 
rx_vport_broadcast_bytes /sec
Ethtool(enp175s0) stat:          232 (            232) <= 
rx_vport_broadcast_packets /sec
Ethtool(enp175s0) stat:        25491 (         25,491) <= 
rx_vport_multicast_bytes /sec
Ethtool(enp175s0) stat:          354 (            354) <= 
rx_vport_multicast_packets /sec
Ethtool(enp175s0) stat:   4872354516 (  4,872,354,516) <= 
rx_vport_unicast_bytes /sec
Ethtool(enp175s0) stat:      3721920 (      3,721,920) <= 
rx_vport_unicast_packets /sec
Ethtool(enp175s0) stat:       158883 (        158,883) <= 
tx0_added_vlan_packets /sec
Ethtool(enp175s0) stat:     93790423 (     93,790,423) <= tx0_bytes /sec
Ethtool(enp175s0) stat:       158854 (        158,854) <= tx0_cqes /sec
Ethtool(enp175s0) stat:       146499 (        146,499) <= tx0_csum_none /sec
Ethtool(enp175s0) stat:        12384 (         12,384) <= 
tx0_csum_partial /sec
Ethtool(enp175s0) stat:         2144 (          2,144) <= tx0_nop /sec
Ethtool(enp175s0) stat:       188173 (        188,173) <= tx0_packets /sec
Ethtool(enp175s0) stat:     53068613 (     53,068,613) <= tx0_tso_bytes /sec
Ethtool(enp175s0) stat:         8839 (          8,839) <= 
tx0_tso_packets /sec
Ethtool(enp175s0) stat:           30 (             30) <= tx0_xmit_more /sec
Ethtool(enp175s0) stat:       165538 (        165,538) <= 
tx10_added_vlan_packets /sec
Ethtool(enp175s0) stat:    102395057 (    102,395,057) <= tx10_bytes /sec
Ethtool(enp175s0) stat:       165303 (        165,303) <= tx10_cqes /sec
Ethtool(enp175s0) stat:       151089 (        151,089) <= tx10_csum_none 
/sec
Ethtool(enp175s0) stat:        14448 (         14,448) <= 
tx10_csum_partial /sec
Ethtool(enp175s0) stat:         2391 (          2,391) <= tx10_nop /sec
Ethtool(enp175s0) stat:       198798 (        198,798) <= tx10_packets /sec
Ethtool(enp175s0) stat:     58951449 (     58,951,449) <= tx10_tso_bytes 
/sec
Ethtool(enp175s0) stat:         8987 (          8,987) <= 
tx10_tso_packets /sec
Ethtool(enp175s0) stat:          234 (            234) <= tx10_xmit_more 
/sec
Ethtool(enp175s0) stat:       166402 (        166,402) <= 
tx11_added_vlan_packets /sec
Ethtool(enp175s0) stat:     98591304 (     98,591,304) <= tx11_bytes /sec
Ethtool(enp175s0) stat:       166384 (        166,384) <= tx11_cqes /sec
Ethtool(enp175s0) stat:       152456 (        152,456) <= tx11_csum_none 
/sec
Ethtool(enp175s0) stat:        13946 (         13,946) <= 
tx11_csum_partial /sec
Ethtool(enp175s0) stat:         2386 (          2,386) <= tx11_nop /sec
Ethtool(enp175s0) stat:       201615 (        201,615) <= tx11_packets /sec
Ethtool(enp175s0) stat:     63844660 (     63,844,660) <= tx11_tso_bytes 
/sec
Ethtool(enp175s0) stat:        10515 (         10,515) <= 
tx11_tso_packets /sec
Ethtool(enp175s0) stat:           18 (             18) <= tx11_xmit_more 
/sec
Ethtool(enp175s0) stat:       156312 (        156,312) <= 
tx12_added_vlan_packets /sec
Ethtool(enp175s0) stat:     97068537 (     97,068,537) <= tx12_bytes /sec
Ethtool(enp175s0) stat:       156302 (        156,302) <= tx12_cqes /sec
Ethtool(enp175s0) stat:       142468 (        142,468) <= tx12_csum_none 
/sec
Ethtool(enp175s0) stat:        13844 (         13,844) <= 
tx12_csum_partial /sec
Ethtool(enp175s0) stat:         2278 (          2,278) <= tx12_nop /sec
Ethtool(enp175s0) stat:       187535 (        187,535) <= tx12_packets /sec
Ethtool(enp175s0) stat:     58368798 (     58,368,798) <= tx12_tso_bytes 
/sec
Ethtool(enp175s0) stat:        10398 (         10,398) <= 
tx12_tso_packets /sec
Ethtool(enp175s0) stat:           10 (             10) <= tx12_xmit_more 
/sec
Ethtool(enp175s0) stat:       161768 (        161,768) <= 
tx13_added_vlan_packets /sec
Ethtool(enp175s0) stat:    120232518 (    120,232,518) <= tx13_bytes /sec
Ethtool(enp175s0) stat:       161584 (        161,584) <= tx13_cqes /sec
Ethtool(enp175s0) stat:       144388 (        144,388) <= tx13_csum_none 
/sec
Ethtool(enp175s0) stat:        17380 (         17,380) <= 
tx13_csum_partial /sec
Ethtool(enp175s0) stat:         2425 (          2,425) <= tx13_nop /sec
Ethtool(enp175s0) stat:       202823 (        202,823) <= tx13_packets /sec
Ethtool(enp175s0) stat:     72804507 (     72,804,507) <= tx13_tso_bytes 
/sec
Ethtool(enp175s0) stat:        10865 (         10,865) <= 
tx13_tso_packets /sec
Ethtool(enp175s0) stat:          185 (            185) <= tx13_xmit_more 
/sec
Ethtool(enp175s0) stat:       165762 (        165,762) <= 
tx14_added_vlan_packets /sec
Ethtool(enp175s0) stat:     99271622 (     99,271,622) <= tx14_bytes /sec
Ethtool(enp175s0) stat:       165688 (        165,688) <= tx14_cqes /sec
Ethtool(enp175s0) stat:       153195 (        153,195) <= tx14_csum_none 
/sec
Ethtool(enp175s0) stat:        12566 (         12,566) <= 
tx14_csum_partial /sec
Ethtool(enp175s0) stat:         2195 (          2,195) <= tx14_nop /sec
Ethtool(enp175s0) stat:       195504 (        195,504) <= tx14_packets /sec
Ethtool(enp175s0) stat:     53717277 (     53,717,277) <= tx14_tso_bytes 
/sec
Ethtool(enp175s0) stat:         8743 (          8,743) <= 
tx14_tso_packets /sec
Ethtool(enp175s0) stat:           32 (             32) <= tx14_xmit_more 
/sec
Ethtool(enp175s0) stat:       162803 (        162,803) <= 
tx15_added_vlan_packets /sec
Ethtool(enp175s0) stat:    105591893 (    105,591,893) <= tx15_bytes /sec
Ethtool(enp175s0) stat:       162673 (        162,673) <= tx15_cqes /sec
Ethtool(enp175s0) stat:       147080 (        147,080) <= tx15_csum_none 
/sec
Ethtool(enp175s0) stat:        15723 (         15,723) <= 
tx15_csum_partial /sec
Ethtool(enp175s0) stat:         2355 (          2,355) <= tx15_nop /sec
Ethtool(enp175s0) stat:       198282 (        198,282) <= tx15_packets /sec
Ethtool(enp175s0) stat:     64278573 (     64,278,573) <= tx15_tso_bytes 
/sec
Ethtool(enp175s0) stat:        10704 (         10,704) <= 
tx15_tso_packets /sec
Ethtool(enp175s0) stat:          183 (            183) <= tx15_xmit_more 
/sec
Ethtool(enp175s0) stat:       125282 (        125,282) <= 
tx16_added_vlan_packets /sec
Ethtool(enp175s0) stat:     81835815 (     81,835,815) <= tx16_bytes /sec
Ethtool(enp175s0) stat:       125264 (        125,264) <= tx16_cqes /sec
Ethtool(enp175s0) stat:       113284 (        113,284) <= tx16_csum_none 
/sec
Ethtool(enp175s0) stat:        11998 (         11,998) <= 
tx16_csum_partial /sec
Ethtool(enp175s0) stat:         1773 (          1,773) <= tx16_nop /sec
Ethtool(enp175s0) stat:       150812 (        150,812) <= tx16_packets /sec
Ethtool(enp175s0) stat:     46027767 (     46,027,767) <= tx16_tso_bytes 
/sec
Ethtool(enp175s0) stat:         7361 (          7,361) <= 
tx16_tso_packets /sec
Ethtool(enp175s0) stat:           18 (             18) <= tx16_xmit_more 
/sec
Ethtool(enp175s0) stat:       131865 (        131,865) <= 
tx17_added_vlan_packets /sec
Ethtool(enp175s0) stat:     89213555 (     89,213,555) <= tx17_bytes /sec
Ethtool(enp175s0) stat:       131409 (        131,409) <= tx17_cqes /sec
Ethtool(enp175s0) stat:       117199 (        117,199) <= tx17_csum_none 
/sec
Ethtool(enp175s0) stat:        14665 (         14,665) <= 
tx17_csum_partial /sec
Ethtool(enp175s0) stat:         1933 (          1,933) <= tx17_nop /sec
Ethtool(enp175s0) stat:       161576 (        161,576) <= tx17_packets /sec
Ethtool(enp175s0) stat:     53682297 (     53,682,297) <= tx17_tso_bytes 
/sec
Ethtool(enp175s0) stat:         8825 (          8,825) <= 
tx17_tso_packets /sec
Ethtool(enp175s0) stat:          459 (            459) <= tx17_xmit_more 
/sec
Ethtool(enp175s0) stat:       122492 (        122,492) <= 
tx18_added_vlan_packets /sec
Ethtool(enp175s0) stat:     87373476 (     87,373,476) <= tx18_bytes /sec
Ethtool(enp175s0) stat:       122138 (        122,138) <= tx18_cqes /sec
Ethtool(enp175s0) stat:       109037 (        109,037) <= tx18_csum_none 
/sec
Ethtool(enp175s0) stat:        13455 (         13,455) <= 
tx18_csum_partial /sec
Ethtool(enp175s0) stat:         1933 (          1,933) <= tx18_nop /sec
Ethtool(enp175s0) stat:       152821 (        152,821) <= tx18_packets /sec
Ethtool(enp175s0) stat:     54163102 (     54,163,102) <= tx18_tso_bytes 
/sec
Ethtool(enp175s0) stat:         8674 (          8,674) <= 
tx18_tso_packets /sec
Ethtool(enp175s0) stat:          354 (            354) <= tx18_xmit_more 
/sec
Ethtool(enp175s0) stat:       119294 (        119,294) <= 
tx19_added_vlan_packets /sec
Ethtool(enp175s0) stat:     67719609 (     67,719,609) <= tx19_bytes /sec
Ethtool(enp175s0) stat:       119262 (        119,262) <= tx19_cqes /sec
Ethtool(enp175s0) stat:       108591 (        108,591) <= tx19_csum_none 
/sec
Ethtool(enp175s0) stat:        10703 (         10,703) <= 
tx19_csum_partial /sec
Ethtool(enp175s0) stat:         1551 (          1,551) <= tx19_nop /sec
Ethtool(enp175s0) stat:       141344 (        141,344) <= tx19_packets /sec
Ethtool(enp175s0) stat:     39539778 (     39,539,778) <= tx19_tso_bytes 
/sec
Ethtool(enp175s0) stat:         6310 (          6,310) <= 
tx19_tso_packets /sec
Ethtool(enp175s0) stat:           31 (             31) <= tx19_xmit_more 
/sec
Ethtool(enp175s0) stat:       157094 (        157,094) <= 
tx1_added_vlan_packets /sec
Ethtool(enp175s0) stat:     93505806 (     93,505,806) <= tx1_bytes /sec
Ethtool(enp175s0) stat:       156935 (        156,935) <= tx1_cqes /sec
Ethtool(enp175s0) stat:       144805 (        144,805) <= tx1_csum_none /sec
Ethtool(enp175s0) stat:        12289 (         12,289) <= 
tx1_csum_partial /sec
Ethtool(enp175s0) stat:         2201 (          2,201) <= tx1_nop /sec
Ethtool(enp175s0) stat:       184561 (        184,561) <= tx1_packets /sec
Ethtool(enp175s0) stat:     50513729 (     50,513,729) <= tx1_tso_bytes /sec
Ethtool(enp175s0) stat:         8699 (          8,699) <= 
tx1_tso_packets /sec
Ethtool(enp175s0) stat:          159 (            159) <= tx1_xmit_more /sec
Ethtool(enp175s0) stat:       134411 (        134,411) <= 
tx20_added_vlan_packets /sec
Ethtool(enp175s0) stat:     88898658 (     88,898,658) <= tx20_bytes /sec
Ethtool(enp175s0) stat:       134221 (        134,221) <= tx20_cqes /sec
Ethtool(enp175s0) stat:       120787 (        120,787) <= tx20_csum_none 
/sec
Ethtool(enp175s0) stat:        13624 (         13,624) <= 
tx20_csum_partial /sec
Ethtool(enp175s0) stat:         2064 (          2,064) <= tx20_nop /sec
Ethtool(enp175s0) stat:       165064 (        165,064) <= tx20_packets /sec
Ethtool(enp175s0) stat:     54707259 (     54,707,259) <= tx20_tso_bytes 
/sec
Ethtool(enp175s0) stat:         8769 (          8,769) <= 
tx20_tso_packets /sec
Ethtool(enp175s0) stat:          188 (            188) <= tx20_xmit_more 
/sec
Ethtool(enp175s0) stat:       127029 (        127,029) <= 
tx21_added_vlan_packets /sec
Ethtool(enp175s0) stat:     98012630 (     98,012,630) <= tx21_bytes /sec
Ethtool(enp175s0) stat:       126990 (        126,990) <= tx21_cqes /sec
Ethtool(enp175s0) stat:       111553 (        111,553) <= tx21_csum_none 
/sec
Ethtool(enp175s0) stat:        15476 (         15,476) <= 
tx21_csum_partial /sec
Ethtool(enp175s0) stat:         2002 (          2,002) <= tx21_nop /sec
Ethtool(enp175s0) stat:       159688 (        159,688) <= tx21_packets /sec
Ethtool(enp175s0) stat:     59637304 (     59,637,304) <= tx21_tso_bytes 
/sec
Ethtool(enp175s0) stat:         9988 (          9,988) <= 
tx21_tso_packets /sec
Ethtool(enp175s0) stat:           39 (             39) <= tx21_xmit_more 
/sec
Ethtool(enp175s0) stat:       122610 (        122,610) <= 
tx22_added_vlan_packets /sec
Ethtool(enp175s0) stat:     70972052 (     70,972,052) <= tx22_bytes /sec
Ethtool(enp175s0) stat:       122600 (        122,600) <= tx22_cqes /sec
Ethtool(enp175s0) stat:       111526 (        111,526) <= tx22_csum_none 
/sec
Ethtool(enp175s0) stat:        11085 (         11,085) <= 
tx22_csum_partial /sec
Ethtool(enp175s0) stat:         1544 (          1,544) <= tx22_nop /sec
Ethtool(enp175s0) stat:       144874 (        144,874) <= tx22_packets /sec
Ethtool(enp175s0) stat:     40175057 (     40,175,057) <= tx22_tso_bytes 
/sec
Ethtool(enp175s0) stat:         6517 (          6,517) <= 
tx22_tso_packets /sec
Ethtool(enp175s0) stat:           30 (             30) <= tx22_xmit_more 
/sec
Ethtool(enp175s0) stat:       126809 (        126,809) <= 
tx23_added_vlan_packets /sec
Ethtool(enp175s0) stat:     76906314 (     76,906,314) <= tx23_bytes /sec
Ethtool(enp175s0) stat:       126791 (        126,791) <= tx23_cqes /sec
Ethtool(enp175s0) stat:       116656 (        116,656) <= tx23_csum_none 
/sec
Ethtool(enp175s0) stat:        10153 (         10,153) <= 
tx23_csum_partial /sec
Ethtool(enp175s0) stat:         1686 (          1,686) <= tx23_nop /sec
Ethtool(enp175s0) stat:       148551 (        148,551) <= tx23_packets /sec
Ethtool(enp175s0) stat:     39913764 (     39,913,764) <= tx23_tso_bytes 
/sec
Ethtool(enp175s0) stat:         6978 (          6,978) <= 
tx23_tso_packets /sec
Ethtool(enp175s0) stat:           18 (             18) <= tx23_xmit_more 
/sec
Ethtool(enp175s0) stat:       133910 (        133,910) <= 
tx24_added_vlan_packets /sec
Ethtool(enp175s0) stat:     69609913 (     69,609,913) <= tx24_bytes /sec
Ethtool(enp175s0) stat:       133890 (        133,890) <= tx24_cqes /sec
Ethtool(enp175s0) stat:       124462 (        124,462) <= tx24_csum_none 
/sec
Ethtool(enp175s0) stat:         9448 (          9,448) <= 
tx24_csum_partial /sec
Ethtool(enp175s0) stat:         1639 (          1,639) <= tx24_nop /sec
Ethtool(enp175s0) stat:       154475 (        154,475) <= tx24_packets /sec
Ethtool(enp175s0) stat:     37456148 (     37,456,148) <= tx24_tso_bytes 
/sec
Ethtool(enp175s0) stat:         6247 (          6,247) <= 
tx24_tso_packets /sec
Ethtool(enp175s0) stat:           20 (             20) <= tx24_xmit_more 
/sec
Ethtool(enp175s0) stat:       118528 (        118,528) <= 
tx25_added_vlan_packets /sec
Ethtool(enp175s0) stat:     62435525 (     62,435,525) <= tx25_bytes /sec
Ethtool(enp175s0) stat:       118471 (        118,471) <= tx25_cqes /sec
Ethtool(enp175s0) stat:       108887 (        108,887) <= tx25_csum_none 
/sec
Ethtool(enp175s0) stat:         9640 (          9,640) <= 
tx25_csum_partial /sec
Ethtool(enp175s0) stat:         1528 (          1,528) <= tx25_nop /sec
Ethtool(enp175s0) stat:       138748 (        138,748) <= tx25_packets /sec
Ethtool(enp175s0) stat:     36592045 (     36,592,045) <= tx25_tso_bytes 
/sec
Ethtool(enp175s0) stat:         5993 (          5,993) <= 
tx25_tso_packets /sec
Ethtool(enp175s0) stat:           44 (             44) <= tx25_xmit_more 
/sec
Ethtool(enp175s0) stat:       119890 (        119,890) <= 
tx26_added_vlan_packets /sec
Ethtool(enp175s0) stat:     76743929 (     76,743,929) <= tx26_bytes /sec
Ethtool(enp175s0) stat:       119873 (        119,873) <= tx26_cqes /sec
Ethtool(enp175s0) stat:       108181 (        108,181) <= tx26_csum_none 
/sec
Ethtool(enp175s0) stat:        11709 (         11,709) <= 
tx26_csum_partial /sec
Ethtool(enp175s0) stat:         1690 (          1,690) <= tx26_nop /sec
Ethtool(enp175s0) stat:       144495 (        144,495) <= tx26_packets /sec
Ethtool(enp175s0) stat:     43922304 (     43,922,304) <= tx26_tso_bytes 
/sec
Ethtool(enp175s0) stat:         7043 (          7,043) <= 
tx26_tso_packets /sec
Ethtool(enp175s0) stat:           17 (             17) <= tx26_xmit_more 
/sec
Ethtool(enp175s0) stat:       130825 (        130,825) <= 
tx27_added_vlan_packets /sec
Ethtool(enp175s0) stat:     88723162 (     88,723,162) <= tx27_bytes /sec
Ethtool(enp175s0) stat:       130769 (        130,769) <= tx27_cqes /sec
Ethtool(enp175s0) stat:       116555 (        116,555) <= tx27_csum_none 
/sec
Ethtool(enp175s0) stat:        14270 (         14,270) <= 
tx27_csum_partial /sec
Ethtool(enp175s0) stat:         2042 (          2,042) <= tx27_nop /sec
Ethtool(enp175s0) stat:       161228 (        161,228) <= tx27_packets /sec
Ethtool(enp175s0) stat:     55637023 (     55,637,023) <= tx27_tso_bytes 
/sec
Ethtool(enp175s0) stat:         9707 (          9,707) <= 
tx27_tso_packets /sec
Ethtool(enp175s0) stat:           71 (             71) <= tx27_xmit_more 
/sec
Ethtool(enp175s0) stat:       166973 (        166,973) <= 
tx2_added_vlan_packets /sec
Ethtool(enp175s0) stat:    103659503 (    103,659,503) <= tx2_bytes /sec
Ethtool(enp175s0) stat:       166941 (        166,941) <= tx2_cqes /sec
Ethtool(enp175s0) stat:       151389 (        151,389) <= tx2_csum_none /sec
Ethtool(enp175s0) stat:        15585 (         15,585) <= 
tx2_csum_partial /sec
Ethtool(enp175s0) stat:         2455 (          2,455) <= tx2_nop /sec
Ethtool(enp175s0) stat:       201854 (        201,854) <= tx2_packets /sec
Ethtool(enp175s0) stat:     65384298 (     65,384,298) <= tx2_tso_bytes /sec
Ethtool(enp175s0) stat:        11809 (         11,809) <= 
tx2_tso_packets /sec
Ethtool(enp175s0) stat:           30 (             30) <= tx2_xmit_more /sec
Ethtool(enp175s0) stat:       172353 (        172,353) <= 
tx3_added_vlan_packets /sec
Ethtool(enp175s0) stat:     88248541 (     88,248,541) <= tx3_bytes /sec
Ethtool(enp175s0) stat:       172277 (        172,277) <= tx3_cqes /sec
Ethtool(enp175s0) stat:       160714 (        160,714) <= tx3_csum_none /sec
Ethtool(enp175s0) stat:        11639 (         11,639) <= 
tx3_csum_partial /sec
Ethtool(enp175s0) stat:         2258 (          2,258) <= tx3_nop /sec
Ethtool(enp175s0) stat:       199853 (        199,853) <= tx3_packets /sec
Ethtool(enp175s0) stat:     49776463 (     49,776,463) <= tx3_tso_bytes /sec
Ethtool(enp175s0) stat:         8353 (          8,353) <= 
tx3_tso_packets /sec
Ethtool(enp175s0) stat:           76 (             76) <= tx3_xmit_more /sec
Ethtool(enp175s0) stat:       157527 (        157,527) <= 
tx4_added_vlan_packets /sec
Ethtool(enp175s0) stat:    110770979 (    110,770,979) <= tx4_bytes /sec
Ethtool(enp175s0) stat:       157492 (        157,492) <= tx4_cqes /sec
Ethtool(enp175s0) stat:       141858 (        141,858) <= tx4_csum_none /sec
Ethtool(enp175s0) stat:        15670 (         15,670) <= 
tx4_csum_partial /sec
Ethtool(enp175s0) stat:         2320 (          2,320) <= tx4_nop /sec
Ethtool(enp175s0) stat:       192429 (        192,429) <= tx4_packets /sec
Ethtool(enp175s0) stat:     64689503 (     64,689,503) <= tx4_tso_bytes /sec
Ethtool(enp175s0) stat:        11367 (         11,367) <= 
tx4_tso_packets /sec
Ethtool(enp175s0) stat:           35 (             35) <= tx4_xmit_more /sec
Ethtool(enp175s0) stat:       169077 (        169,077) <= 
tx5_added_vlan_packets /sec
Ethtool(enp175s0) stat:    121536690 (    121,536,690) <= tx5_bytes /sec
Ethtool(enp175s0) stat:       168906 (        168,906) <= tx5_cqes /sec
Ethtool(enp175s0) stat:       150099 (        150,099) <= tx5_csum_none /sec
Ethtool(enp175s0) stat:        18978 (         18,978) <= 
tx5_csum_partial /sec
Ethtool(enp175s0) stat:         2678 (          2,678) <= tx5_nop /sec
Ethtool(enp175s0) stat:       210733 (        210,733) <= tx5_packets /sec
Ethtool(enp175s0) stat:     76448346 (     76,448,346) <= tx5_tso_bytes /sec
Ethtool(enp175s0) stat:        13238 (         13,238) <= 
tx5_tso_packets /sec
Ethtool(enp175s0) stat:          171 (            171) <= tx5_xmit_more /sec
Ethtool(enp175s0) stat:       156881 (        156,881) <= 
tx6_added_vlan_packets /sec
Ethtool(enp175s0) stat:     85752393 (     85,752,393) <= tx6_bytes /sec
Ethtool(enp175s0) stat:       156859 (        156,859) <= tx6_cqes /sec
Ethtool(enp175s0) stat:       144843 (        144,843) <= tx6_csum_none /sec
Ethtool(enp175s0) stat:        12038 (         12,038) <= 
tx6_csum_partial /sec
Ethtool(enp175s0) stat:         2034 (          2,034) <= tx6_nop /sec
Ethtool(enp175s0) stat:       181315 (        181,315) <= tx6_packets /sec
Ethtool(enp175s0) stat:     44625307 (     44,625,307) <= tx6_tso_bytes /sec
Ethtool(enp175s0) stat:         7752 (          7,752) <= 
tx6_tso_packets /sec
Ethtool(enp175s0) stat:           24 (             24) <= tx6_xmit_more /sec
Ethtool(enp175s0) stat:       157744 (        157,744) <= 
tx7_added_vlan_packets /sec
Ethtool(enp175s0) stat:     88795890 (     88,795,890) <= tx7_bytes /sec
Ethtool(enp175s0) stat:       157666 (        157,666) <= tx7_cqes /sec
Ethtool(enp175s0) stat:       146023 (        146,023) <= tx7_csum_none /sec
Ethtool(enp175s0) stat:        11720 (         11,720) <= 
tx7_csum_partial /sec
Ethtool(enp175s0) stat:         2190 (          2,190) <= tx7_nop /sec
Ethtool(enp175s0) stat:       184845 (        184,845) <= tx7_packets /sec
Ethtool(enp175s0) stat:     48986735 (     48,986,735) <= tx7_tso_bytes /sec
Ethtool(enp175s0) stat:         8401 (          8,401) <= 
tx7_tso_packets /sec
Ethtool(enp175s0) stat:           78 (             78) <= tx7_xmit_more /sec
Ethtool(enp175s0) stat:       165190 (        165,190) <= 
tx8_added_vlan_packets /sec
Ethtool(enp175s0) stat:    108303283 (    108,303,283) <= tx8_bytes /sec
Ethtool(enp175s0) stat:       165145 (        165,145) <= tx8_cqes /sec
Ethtool(enp175s0) stat:       150047 (        150,047) <= tx8_csum_none /sec
Ethtool(enp175s0) stat:        15143 (         15,143) <= 
tx8_csum_partial /sec
Ethtool(enp175s0) stat:         2497 (          2,497) <= tx8_nop /sec
Ethtool(enp175s0) stat:       201615 (        201,615) <= tx8_packets /sec
Ethtool(enp175s0) stat:     67575199 (     67,575,199) <= tx8_tso_bytes /sec
Ethtool(enp175s0) stat:        11828 (         11,828) <= 
tx8_tso_packets /sec
Ethtool(enp175s0) stat:           45 (             45) <= tx8_xmit_more /sec
Ethtool(enp175s0) stat:       168256 (        168,256) <= 
tx9_added_vlan_packets /sec
Ethtool(enp175s0) stat:    115981227 (    115,981,227) <= tx9_bytes /sec
Ethtool(enp175s0) stat:       168225 (        168,225) <= tx9_cqes /sec
Ethtool(enp175s0) stat:       147980 (        147,980) <= tx9_csum_none /sec
Ethtool(enp175s0) stat:        20275 (         20,275) <= 
tx9_csum_partial /sec
Ethtool(enp175s0) stat:         2522 (          2,522) <= tx9_nop /sec
Ethtool(enp175s0) stat:       203398 (        203,398) <= tx9_packets /sec
Ethtool(enp175s0) stat:     63891360 (     63,891,360) <= tx9_tso_bytes /sec
Ethtool(enp175s0) stat:        10610 (         10,610) <= 
tx9_tso_packets /sec
Ethtool(enp175s0) stat:           30 (             30) <= tx9_xmit_more /sec
Ethtool(enp175s0) stat:      4121491 (      4,121,491) <= 
tx_added_vlan_packets /sec
Ethtool(enp175s0) stat:            1 (              1) <= 
tx_broadcast_phy /sec
Ethtool(enp175s0) stat:   2591851137 (  2,591,851,137) <= tx_bytes /sec
Ethtool(enp175s0) stat:   2626884154 (  2,626,884,154) <= tx_bytes_phy /sec
Ethtool(enp175s0) stat:      4118829 (      4,118,829) <= tx_cqes /sec
Ethtool(enp175s0) stat:      3741682 (      3,741,682) <= tx_csum_none /sec
Ethtool(enp175s0) stat:       379810 (        379,810) <= 
tx_csum_partial /sec
Ethtool(enp175s0) stat:        58718 (         58,718) <= tx_nop /sec
Ethtool(enp175s0) stat:      4956995 (      4,956,995) <= tx_packets /sec
Ethtool(enp175s0) stat:      4956579 (      4,956,579) <= tx_packets_phy 
/sec
Ethtool(enp175s0) stat:   2627152852 (  2,627,152,852) <= tx_prio0_bytes 
/sec
Ethtool(enp175s0) stat:      4957649 (      4,957,649) <= 
tx_prio0_packets /sec
Ethtool(enp175s0) stat:   1518383905 (  1,518,383,905) <= tx_tso_bytes /sec
Ethtool(enp175s0) stat:       253526 (        253,526) <= tx_tso_packets 
/sec
Ethtool(enp175s0) stat:           57 (             57) <= 
tx_vport_broadcast_bytes /sec
Ethtool(enp175s0) stat:            1 (              1) <= 
tx_vport_broadcast_packets /sec
Ethtool(enp175s0) stat:   2607061269 (  2,607,061,269) <= 
tx_vport_unicast_bytes /sec
Ethtool(enp175s0) stat:      4956492 (      4,956,492) <= 
tx_vport_unicast_packets /sec
Ethtool(enp175s0) stat:         2630 (          2,630) <= tx_xmit_more /sec
Ethtool(enp216s0) stat:         6923 (          6,923) <= ch0_arm /sec
Ethtool(enp216s0) stat:         8367 (          8,367) <= ch0_events /sec
Ethtool(enp216s0) stat:         9797 (          9,797) <= ch0_poll /sec
Ethtool(enp216s0) stat:         7703 (          7,703) <= ch10_arm /sec
Ethtool(enp216s0) stat:         9253 (          9,253) <= ch10_events /sec
Ethtool(enp216s0) stat:        10807 (         10,807) <= ch10_poll /sec
Ethtool(enp216s0) stat:         7886 (          7,886) <= ch11_arm /sec
Ethtool(enp216s0) stat:         9391 (          9,391) <= ch11_events /sec
Ethtool(enp216s0) stat:        10896 (         10,896) <= ch11_poll /sec
Ethtool(enp216s0) stat:         9793 (          9,793) <= ch12_arm /sec
Ethtool(enp216s0) stat:        11299 (         11,299) <= ch12_events /sec
Ethtool(enp216s0) stat:        12637 (         12,637) <= ch12_poll /sec
Ethtool(enp216s0) stat:         9119 (          9,119) <= ch13_arm /sec
Ethtool(enp216s0) stat:        10671 (         10,671) <= ch13_events /sec
Ethtool(enp216s0) stat:        12205 (         12,205) <= ch13_poll /sec
Ethtool(enp216s0) stat:         8784 (          8,784) <= ch14_arm /sec
Ethtool(enp216s0) stat:        10189 (         10,189) <= ch14_events /sec
Ethtool(enp216s0) stat:        11565 (         11,565) <= ch14_poll /sec
Ethtool(enp216s0) stat:         9248 (          9,248) <= ch15_arm /sec
Ethtool(enp216s0) stat:        10739 (         10,739) <= ch15_events /sec
Ethtool(enp216s0) stat:        12238 (         12,238) <= ch15_poll /sec
Ethtool(enp216s0) stat:         5201 (          5,201) <= ch16_arm /sec
Ethtool(enp216s0) stat:         6554 (          6,554) <= ch16_events /sec
Ethtool(enp216s0) stat:         7667 (          7,667) <= ch16_poll /sec
Ethtool(enp216s0) stat:         7799 (          7,799) <= ch17_arm /sec
Ethtool(enp216s0) stat:         9120 (          9,120) <= ch17_events /sec
Ethtool(enp216s0) stat:        10194 (         10,194) <= ch17_poll /sec
Ethtool(enp216s0) stat:         6889 (          6,889) <= ch18_arm /sec
Ethtool(enp216s0) stat:         8266 (          8,266) <= ch18_events /sec
Ethtool(enp216s0) stat:         9332 (          9,332) <= ch18_poll /sec
Ethtool(enp216s0) stat:         5494 (          5,494) <= ch19_arm /sec
Ethtool(enp216s0) stat:         6813 (          6,813) <= ch19_events /sec
Ethtool(enp216s0) stat:         7837 (          7,837) <= ch19_poll /sec
Ethtool(enp216s0) stat:         6672 (          6,672) <= ch1_arm /sec
Ethtool(enp216s0) stat:         8124 (          8,124) <= ch1_events /sec
Ethtool(enp216s0) stat:         9604 (          9,604) <= ch1_poll /sec
Ethtool(enp216s0) stat:         7705 (          7,705) <= ch20_arm /sec
Ethtool(enp216s0) stat:         9102 (          9,102) <= ch20_events /sec
Ethtool(enp216s0) stat:        10287 (         10,287) <= ch20_poll /sec
Ethtool(enp216s0) stat:         5929 (          5,929) <= ch21_arm /sec
Ethtool(enp216s0) stat:         7333 (          7,333) <= ch21_events /sec
Ethtool(enp216s0) stat:         8463 (          8,463) <= ch21_poll /sec
Ethtool(enp216s0) stat:         5495 (          5,495) <= ch22_arm /sec
Ethtool(enp216s0) stat:         6813 (          6,813) <= ch22_events /sec
Ethtool(enp216s0) stat:         7843 (          7,843) <= ch22_poll /sec
Ethtool(enp216s0) stat:         7091 (          7,091) <= ch23_arm /sec
Ethtool(enp216s0) stat:         8367 (          8,367) <= ch23_events /sec
Ethtool(enp216s0) stat:         9344 (          9,344) <= ch23_poll /sec
Ethtool(enp216s0) stat:         5481 (          5,481) <= ch24_arm /sec
Ethtool(enp216s0) stat:         6879 (          6,879) <= ch24_events /sec
Ethtool(enp216s0) stat:         7995 (          7,995) <= ch24_poll /sec
Ethtool(enp216s0) stat:         5642 (          5,642) <= ch25_arm /sec
Ethtool(enp216s0) stat:         6959 (          6,959) <= ch25_events /sec
Ethtool(enp216s0) stat:         7927 (          7,927) <= ch25_poll /sec
Ethtool(enp216s0) stat:         5289 (          5,289) <= ch26_arm /sec
Ethtool(enp216s0) stat:         6643 (          6,643) <= ch26_events /sec
Ethtool(enp216s0) stat:         7691 (          7,691) <= ch26_poll /sec
Ethtool(enp216s0) stat:         7313 (          7,313) <= ch27_arm /sec
Ethtool(enp216s0) stat:         8719 (          8,719) <= ch27_events /sec
Ethtool(enp216s0) stat:         9876 (          9,876) <= ch27_poll /sec
Ethtool(enp216s0) stat:         7791 (          7,791) <= ch2_arm /sec
Ethtool(enp216s0) stat:         9328 (          9,328) <= ch2_events /sec
Ethtool(enp216s0) stat:        10838 (         10,838) <= ch2_poll /sec
Ethtool(enp216s0) stat:         9595 (          9,595) <= ch3_arm /sec
Ethtool(enp216s0) stat:        11015 (         11,015) <= ch3_events /sec
Ethtool(enp216s0) stat:        12402 (         12,402) <= ch3_poll /sec
Ethtool(enp216s0) stat:         9653 (          9,653) <= ch4_arm /sec
Ethtool(enp216s0) stat:        11163 (         11,163) <= ch4_events /sec
Ethtool(enp216s0) stat:        12553 (         12,553) <= ch4_poll /sec
Ethtool(enp216s0) stat:        10269 (         10,269) <= ch5_arm /sec
Ethtool(enp216s0) stat:        11727 (         11,727) <= ch5_events /sec
Ethtool(enp216s0) stat:        13160 (         13,160) <= ch5_poll /sec
Ethtool(enp216s0) stat:         8806 (          8,806) <= ch6_arm /sec
Ethtool(enp216s0) stat:        10191 (         10,191) <= ch6_events /sec
Ethtool(enp216s0) stat:        11431 (         11,431) <= ch6_poll /sec
Ethtool(enp216s0) stat:         6866 (          6,866) <= ch7_arm /sec
Ethtool(enp216s0) stat:         8412 (          8,412) <= ch7_events /sec
Ethtool(enp216s0) stat:         9815 (          9,815) <= ch7_poll /sec
Ethtool(enp216s0) stat:        10533 (         10,533) <= ch8_arm /sec
Ethtool(enp216s0) stat:        11945 (         11,945) <= ch8_events /sec
Ethtool(enp216s0) stat:        13286 (         13,286) <= ch8_poll /sec
Ethtool(enp216s0) stat:         7126 (          7,126) <= ch9_arm /sec
Ethtool(enp216s0) stat:         8643 (          8,643) <= ch9_events /sec
Ethtool(enp216s0) stat:        10210 (         10,210) <= ch9_poll /sec
Ethtool(enp216s0) stat:       212206 (        212,206) <= ch_arm /sec
Ethtool(enp216s0) stat:       252118 (        252,118) <= ch_events /sec
Ethtool(enp216s0) stat:       288009 (        288,009) <= ch_poll /sec
Ethtool(enp216s0) stat:     93281134 (     93,281,134) <= rx0_bytes /sec
Ethtool(enp216s0) stat:        94200 (         94,200) <= 
rx0_cache_reuse /sec
Ethtool(enp216s0) stat:       188327 (        188,327) <= 
rx0_csum_complete /sec
Ethtool(enp216s0) stat:            7 (              7) <= rx0_csum_none /sec
Ethtool(enp216s0) stat:       188334 (        188,334) <= rx0_packets /sec
Ethtool(enp216s0) stat:       188334 (        188,334) <= 
rx0_removed_vlan_packets /sec
Ethtool(enp216s0) stat:    102443052 (    102,443,052) <= rx10_bytes /sec
Ethtool(enp216s0) stat:        99816 (         99,816) <= 
rx10_cache_reuse /sec
Ethtool(enp216s0) stat:       199616 (        199,616) <= 
rx10_csum_complete /sec
Ethtool(enp216s0) stat:       199616 (        199,616) <= rx10_packets /sec
Ethtool(enp216s0) stat:       199616 (        199,616) <= 
rx10_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     97655592 (     97,655,592) <= rx11_bytes /sec
Ethtool(enp216s0) stat:       100854 (        100,854) <= 
rx11_cache_reuse /sec
Ethtool(enp216s0) stat:       201655 (        201,655) <= 
rx11_csum_complete /sec
Ethtool(enp216s0) stat:       201655 (        201,655) <= rx11_packets /sec
Ethtool(enp216s0) stat:       201655 (        201,655) <= 
rx11_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     97076715 (     97,076,715) <= rx12_bytes /sec
Ethtool(enp216s0) stat:        94078 (         94,078) <= 
rx12_cache_reuse /sec
Ethtool(enp216s0) stat:       188037 (        188,037) <= 
rx12_csum_complete /sec
Ethtool(enp216s0) stat:       188037 (        188,037) <= rx12_packets /sec
Ethtool(enp216s0) stat:       188037 (        188,037) <= 
rx12_removed_vlan_packets /sec
Ethtool(enp216s0) stat:    120547921 (    120,547,921) <= rx13_bytes /sec
Ethtool(enp216s0) stat:       101709 (        101,709) <= 
rx13_cache_reuse /sec
Ethtool(enp216s0) stat:       203473 (        203,473) <= 
rx13_csum_complete /sec
Ethtool(enp216s0) stat:       203473 (        203,473) <= rx13_packets /sec
Ethtool(enp216s0) stat:       203473 (        203,473) <= 
rx13_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     98367936 (     98,367,936) <= rx14_bytes /sec
Ethtool(enp216s0) stat:        97741 (         97,741) <= 
rx14_cache_reuse /sec
Ethtool(enp216s0) stat:       195506 (        195,506) <= 
rx14_csum_complete /sec
Ethtool(enp216s0) stat:       195506 (        195,506) <= rx14_packets /sec
Ethtool(enp216s0) stat:       195506 (        195,506) <= 
rx14_removed_vlan_packets /sec
Ethtool(enp216s0) stat:    106726542 (    106,726,542) <= rx15_bytes /sec
Ethtool(enp216s0) stat:        99694 (         99,694) <= 
rx15_cache_reuse /sec
Ethtool(enp216s0) stat:       199395 (        199,395) <= 
rx15_csum_complete /sec
Ethtool(enp216s0) stat:       199395 (        199,395) <= rx15_packets /sec
Ethtool(enp216s0) stat:       199395 (        199,395) <= 
rx15_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     81928969 (     81,928,969) <= rx16_bytes /sec
Ethtool(enp216s0) stat:        75580 (         75,580) <= 
rx16_cache_reuse /sec
Ethtool(enp216s0) stat:       151139 (        151,139) <= 
rx16_csum_complete /sec
Ethtool(enp216s0) stat:       151139 (        151,139) <= rx16_packets /sec
Ethtool(enp216s0) stat:       151139 (        151,139) <= 
rx16_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     90509227 (     90,509,227) <= rx17_bytes /sec
Ethtool(enp216s0) stat:        81196 (         81,196) <= 
rx17_cache_reuse /sec
Ethtool(enp216s0) stat:       162403 (        162,403) <= 
rx17_csum_complete /sec
Ethtool(enp216s0) stat:       162403 (        162,403) <= rx17_packets /sec
Ethtool(enp216s0) stat:       162403 (        162,403) <= 
rx17_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     87854385 (     87,854,385) <= rx18_bytes /sec
Ethtool(enp216s0) stat:        76923 (         76,923) <= 
rx18_cache_reuse /sec
Ethtool(enp216s0) stat:       153866 (        153,866) <= 
rx18_csum_complete /sec
Ethtool(enp216s0) stat:       153866 (        153,866) <= rx18_packets /sec
Ethtool(enp216s0) stat:       153866 (        153,866) <= 
rx18_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     67849725 (     67,849,725) <= rx19_bytes /sec
Ethtool(enp216s0) stat:        71001 (         71,001) <= 
rx19_cache_reuse /sec
Ethtool(enp216s0) stat:       142064 (        142,064) <= 
rx19_csum_complete /sec
Ethtool(enp216s0) stat:       142064 (        142,064) <= rx19_packets /sec
Ethtool(enp216s0) stat:       142064 (        142,064) <= 
rx19_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     92611021 (     92,611,021) <= rx1_bytes /sec
Ethtool(enp216s0) stat:        92307 (         92,307) <= 
rx1_cache_reuse /sec
Ethtool(enp216s0) stat:       184639 (        184,639) <= 
rx1_csum_complete /sec
Ethtool(enp216s0) stat:       184639 (        184,639) <= rx1_packets /sec
Ethtool(enp216s0) stat:       184639 (        184,639) <= 
rx1_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     88902617 (     88,902,617) <= rx20_bytes /sec
Ethtool(enp216s0) stat:        82844 (         82,844) <= 
rx20_cache_reuse /sec
Ethtool(enp216s0) stat:       165764 (        165,764) <= 
rx20_csum_complete /sec
Ethtool(enp216s0) stat:       165764 (        165,764) <= rx20_packets /sec
Ethtool(enp216s0) stat:       165764 (        165,764) <= 
rx20_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     98096942 (     98,096,942) <= rx21_bytes /sec
Ethtool(enp216s0) stat:        79975 (         79,975) <= 
rx21_cache_reuse /sec
Ethtool(enp216s0) stat:       159908 (        159,908) <= 
rx21_csum_complete /sec
Ethtool(enp216s0) stat:       159908 (        159,908) <= rx21_packets /sec
Ethtool(enp216s0) stat:       159908 (        159,908) <= 
rx21_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     71885445 (     71,885,445) <= rx22_bytes /sec
Ethtool(enp216s0) stat:        73565 (         73,565) <= 
rx22_cache_reuse /sec
Ethtool(enp216s0) stat:       147136 (        147,136) <= 
rx22_csum_complete /sec
Ethtool(enp216s0) stat:       147136 (        147,136) <= rx22_packets /sec
Ethtool(enp216s0) stat:       147136 (        147,136) <= 
rx22_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     77030721 (     77,030,721) <= rx23_bytes /sec
Ethtool(enp216s0) stat:        74481 (         74,481) <= 
rx23_cache_reuse /sec
Ethtool(enp216s0) stat:       148989 (        148,989) <= 
rx23_csum_complete /sec
Ethtool(enp216s0) stat:       148989 (        148,989) <= rx23_packets /sec
Ethtool(enp216s0) stat:       148989 (        148,989) <= 
rx23_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     69603951 (     69,603,951) <= rx24_bytes /sec
Ethtool(enp216s0) stat:        77472 (         77,472) <= 
rx24_cache_reuse /sec
Ethtool(enp216s0) stat:       154916 (        154,916) <= 
rx24_csum_complete /sec
Ethtool(enp216s0) stat:       154916 (        154,916) <= rx24_packets /sec
Ethtool(enp216s0) stat:       154916 (        154,916) <= 
rx24_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     62277522 (     62,277,522) <= rx25_bytes /sec
Ethtool(enp216s0) stat:        69414 (         69,414) <= 
rx25_cache_reuse /sec
Ethtool(enp216s0) stat:       138835 (        138,835) <= 
rx25_csum_complete /sec
Ethtool(enp216s0) stat:       138835 (        138,835) <= rx25_packets /sec
Ethtool(enp216s0) stat:       138835 (        138,835) <= 
rx25_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     77498258 (     77,498,258) <= rx26_bytes /sec
Ethtool(enp216s0) stat:        72466 (         72,466) <= 
rx26_cache_reuse /sec
Ethtool(enp216s0) stat:       144925 (        144,925) <= 
rx26_csum_complete /sec
Ethtool(enp216s0) stat:       144925 (        144,925) <= rx26_packets /sec
Ethtool(enp216s0) stat:       144925 (        144,925) <= 
rx26_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     88962856 (     88,962,856) <= rx27_bytes /sec
Ethtool(enp216s0) stat:        80952 (         80,952) <= 
rx27_cache_reuse /sec
Ethtool(enp216s0) stat:       161879 (        161,879) <= 
rx27_csum_complete /sec
Ethtool(enp216s0) stat:       161879 (        161,879) <= rx27_packets /sec
Ethtool(enp216s0) stat:       161879 (        161,879) <= 
rx27_removed_vlan_packets /sec
Ethtool(enp216s0) stat:    103624977 (    103,624,977) <= rx2_bytes /sec
Ethtool(enp216s0) stat:       101098 (        101,098) <= 
rx2_cache_reuse /sec
Ethtool(enp216s0) stat:       202130 (        202,130) <= 
rx2_csum_complete /sec
Ethtool(enp216s0) stat:       202130 (        202,130) <= rx2_packets /sec
Ethtool(enp216s0) stat:       202130 (        202,130) <= 
rx2_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     87213368 (     87,213,368) <= rx3_bytes /sec
Ethtool(enp216s0) stat:        99877 (         99,877) <= 
rx3_cache_reuse /sec
Ethtool(enp216s0) stat:       199778 (        199,778) <= 
rx3_csum_complete /sec
Ethtool(enp216s0) stat:       199778 (        199,778) <= rx3_packets /sec
Ethtool(enp216s0) stat:       199778 (        199,778) <= 
rx3_removed_vlan_packets /sec
Ethtool(enp216s0) stat:    110458845 (    110,458,845) <= rx4_bytes /sec
Ethtool(enp216s0) stat:          692 (            692) <= rx4_cache_busy 
/sec
Ethtool(enp216s0) stat:          692 (            692) <= rx4_cache_full 
/sec
Ethtool(enp216s0) stat:        95523 (         95,523) <= 
rx4_cache_reuse /sec
Ethtool(enp216s0) stat:       192402 (        192,402) <= 
rx4_csum_complete /sec
Ethtool(enp216s0) stat:       192402 (        192,402) <= rx4_packets /sec
Ethtool(enp216s0) stat:       192402 (        192,402) <= 
rx4_removed_vlan_packets /sec
Ethtool(enp216s0) stat:    121222062 (    121,222,062) <= rx5_bytes /sec
Ethtool(enp216s0) stat:       105616 (        105,616) <= 
rx5_cache_reuse /sec
Ethtool(enp216s0) stat:       211273 (        211,273) <= 
rx5_csum_complete /sec
Ethtool(enp216s0) stat:       211273 (        211,273) <= rx5_packets /sec
Ethtool(enp216s0) stat:       211273 (        211,273) <= 
rx5_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     85541470 (     85,541,470) <= rx6_bytes /sec
Ethtool(enp216s0) stat:        91147 (         91,147) <= 
rx6_cache_reuse /sec
Ethtool(enp216s0) stat:       182257 (        182,257) <= 
rx6_csum_complete /sec
Ethtool(enp216s0) stat:       182257 (        182,257) <= rx6_packets /sec
Ethtool(enp216s0) stat:       182257 (        182,257) <= 
rx6_removed_vlan_packets /sec
Ethtool(enp216s0) stat:     88443649 (     88,443,649) <= rx7_bytes /sec
Ethtool(enp216s0) stat:        92368 (         92,368) <= 
rx7_cache_reuse /sec
Ethtool(enp216s0) stat:       184828 (        184,828) <= 
rx7_csum_complete /sec
Ethtool(enp216s0) stat:       184828 (        184,828) <= rx7_packets /sec
Ethtool(enp216s0) stat:       184828 (        184,828) <= 
rx7_removed_vlan_packets /sec
Ethtool(enp216s0) stat:    108419072 (    108,419,072) <= rx8_bytes /sec
Ethtool(enp216s0) stat:       101098 (        101,098) <= 
rx8_cache_reuse /sec
Ethtool(enp216s0) stat:       202241 (        202,241) <= 
rx8_csum_complete /sec
Ethtool(enp216s0) stat:       202241 (        202,241) <= rx8_packets /sec
Ethtool(enp216s0) stat:       202241 (        202,241) <= 
rx8_removed_vlan_packets /sec
Ethtool(enp216s0) stat:    116210958 (    116,210,958) <= rx9_bytes /sec
Ethtool(enp216s0) stat:       102014 (        102,014) <= 
rx9_cache_reuse /sec
Ethtool(enp216s0) stat:       204092 (        204,092) <= 
rx9_csum_complete /sec
Ethtool(enp216s0) stat:       204092 (        204,092) <= rx9_packets /sec
Ethtool(enp216s0) stat:       204092 (        204,092) <= 
rx9_removed_vlan_packets /sec
Ethtool(enp216s0) stat:      1065697 (      1,065,697) <= 
rx_1024_to_1518_bytes_phy /sec
Ethtool(enp216s0) stat:       141705 (        141,705) <= 
rx_128_to_255_bytes_phy /sec
Ethtool(enp216s0) stat:       512465 (        512,465) <= 
rx_1519_to_2047_bytes_phy /sec
Ethtool(enp216s0) stat:        49523 (         49,523) <= 
rx_256_to_511_bytes_phy /sec
Ethtool(enp216s0) stat:        56321 (         56,321) <= 
rx_512_to_1023_bytes_phy /sec
Ethtool(enp216s0) stat:       957286 (        957,286) <= 
rx_64_bytes_phy /sec
Ethtool(enp216s0) stat:      2193644 (      2,193,644) <= 
rx_65_to_127_bytes_phy /sec
Ethtool(enp216s0) stat:            1 (              1) <= 
rx_broadcast_phy /sec
Ethtool(enp216s0) stat:   2592286809 (  2,592,286,809) <= rx_bytes /sec
Ethtool(enp216s0) stat:   2633575771 (  2,633,575,771) <= rx_bytes_phy /sec
Ethtool(enp216s0) stat:          692 (            692) <= rx_cache_busy /sec
Ethtool(enp216s0) stat:          692 (            692) <= rx_cache_full /sec
Ethtool(enp216s0) stat:      2484928 (      2,484,928) <= rx_cache_reuse 
/sec
Ethtool(enp216s0) stat:      4971670 (      4,971,670) <= 
rx_csum_complete /sec
Ethtool(enp216s0) stat:            7 (              7) <= rx_csum_none /sec
Ethtool(enp216s0) stat:          464 (            464) <= 
rx_discards_phy /sec
Ethtool(enp216s0) stat:         1376 (          1,376) <= 
rx_multicast_phy /sec
Ethtool(enp216s0) stat:      4971677 (      4,971,677) <= rx_packets /sec
Ethtool(enp216s0) stat:      4975563 (      4,975,563) <= rx_packets_phy 
/sec
Ethtool(enp216s0) stat:   2634727119 (  2,634,727,119) <= rx_prio0_bytes 
/sec
Ethtool(enp216s0) stat:      4975012 (      4,975,012) <= 
rx_prio0_packets /sec
Ethtool(enp216s0) stat:           61 (             61) <= rx_prio1_bytes 
/sec
Ethtool(enp216s0) stat:            1 (              1) <= 
rx_prio1_packets /sec
Ethtool(enp216s0) stat:         2003 (          2,003) <= rx_prio3_bytes 
/sec
Ethtool(enp216s0) stat:            7 (              7) <= 
rx_prio3_packets /sec
Ethtool(enp216s0) stat:         2497 (          2,497) <= rx_prio4_bytes 
/sec
Ethtool(enp216s0) stat:            3 (              3) <= 
rx_prio4_packets /sec
Ethtool(enp216s0) stat:        10697 (         10,697) <= rx_prio5_bytes 
/sec
Ethtool(enp216s0) stat:          163 (            163) <= 
rx_prio5_packets /sec
Ethtool(enp216s0) stat:         3519 (          3,519) <= rx_prio6_bytes 
/sec
Ethtool(enp216s0) stat:           26 (             26) <= 
rx_prio6_packets /sec
Ethtool(enp216s0) stat:        95583 (         95,583) <= rx_prio7_bytes 
/sec
Ethtool(enp216s0) stat:         1365 (          1,365) <= 
rx_prio7_packets /sec
Ethtool(enp216s0) stat:      4971677 (      4,971,677) <= 
rx_removed_vlan_packets /sec
Ethtool(enp216s0) stat:         1377 (          1,377) <= 
rx_steer_missed_packets /sec
Ethtool(enp216s0) stat:          150 (            150) <= 
rx_vport_broadcast_bytes /sec
Ethtool(enp216s0) stat:            1 (              1) <= 
rx_vport_broadcast_packets /sec
Ethtool(enp216s0) stat:        90890 (         90,890) <= 
rx_vport_multicast_bytes /sec
Ethtool(enp216s0) stat:         1376 (          1,376) <= 
rx_vport_multicast_packets /sec
Ethtool(enp216s0) stat:   2612989221 (  2,612,989,221) <= 
rx_vport_unicast_bytes /sec
Ethtool(enp216s0) stat:      4973191 (      4,973,191) <= 
rx_vport_unicast_packets /sec
Ethtool(enp216s0) stat:        82580 (         82,580) <= 
tx0_added_vlan_packets /sec
Ethtool(enp216s0) stat:    186927644 (    186,927,644) <= tx0_bytes /sec
Ethtool(enp216s0) stat:        81347 (         81,347) <= tx0_cqes /sec
Ethtool(enp216s0) stat:        49836 (         49,836) <= tx0_csum_none /sec
Ethtool(enp216s0) stat:        32745 (         32,745) <= 
tx0_csum_partial /sec
Ethtool(enp216s0) stat:         2093 (          2,093) <= tx0_nop /sec
Ethtool(enp216s0) stat:       146450 (        146,450) <= tx0_packets /sec
Ethtool(enp216s0) stat:    131547357 (    131,547,357) <= tx0_tso_bytes /sec
Ethtool(enp216s0) stat:        28323 (         28,323) <= 
tx0_tso_packets /sec
Ethtool(enp216s0) stat:         1226 (          1,226) <= tx0_xmit_more /sec
Ethtool(enp216s0) stat:        85851 (         85,851) <= 
tx10_added_vlan_packets /sec
Ethtool(enp216s0) stat:    198440153 (    198,440,153) <= tx10_bytes /sec
Ethtool(enp216s0) stat:        84107 (         84,107) <= tx10_cqes /sec
Ethtool(enp216s0) stat:        51644 (         51,644) <= tx10_csum_none 
/sec
Ethtool(enp216s0) stat:        34207 (         34,207) <= 
tx10_csum_partial /sec
Ethtool(enp216s0) stat:         2256 (          2,256) <= tx10_nop /sec
Ethtool(enp216s0) stat:       151642 (        151,642) <= tx10_packets /sec
Ethtool(enp216s0) stat:    135145744 (    135,145,744) <= tx10_tso_bytes 
/sec
Ethtool(enp216s0) stat:        29140 (         29,140) <= 
tx10_tso_packets /sec
Ethtool(enp216s0) stat:         1739 (          1,739) <= tx10_xmit_more 
/sec
Ethtool(enp216s0) stat:        85526 (         85,526) <= 
tx11_added_vlan_packets /sec
Ethtool(enp216s0) stat:    199463985 (    199,463,985) <= tx11_bytes /sec
Ethtool(enp216s0) stat:        83793 (         83,793) <= tx11_cqes /sec
Ethtool(enp216s0) stat:        50188 (         50,188) <= tx11_csum_none 
/sec
Ethtool(enp216s0) stat:        35338 (         35,338) <= 
tx11_csum_partial /sec
Ethtool(enp216s0) stat:         2227 (          2,227) <= tx11_nop /sec
Ethtool(enp216s0) stat:       152815 (        152,815) <= tx11_packets /sec
Ethtool(enp216s0) stat:    138134263 (    138,134,263) <= tx11_tso_bytes 
/sec
Ethtool(enp216s0) stat:        29948 (         29,948) <= 
tx11_tso_packets /sec
Ethtool(enp216s0) stat:         1733 (          1,733) <= tx11_xmit_more 
/sec
Ethtool(enp216s0) stat:        77464 (         77,464) <= 
tx12_added_vlan_packets /sec
Ethtool(enp216s0) stat:    185723223 (    185,723,223) <= tx12_bytes /sec
Ethtool(enp216s0) stat:        75907 (         75,907) <= tx12_cqes /sec
Ethtool(enp216s0) stat:        43727 (         43,727) <= tx12_csum_none 
/sec
Ethtool(enp216s0) stat:        33738 (         33,738) <= 
tx12_csum_partial /sec
Ethtool(enp216s0) stat:         2060 (          2,060) <= tx12_nop /sec
Ethtool(enp216s0) stat:       141903 (        141,903) <= tx12_packets /sec
Ethtool(enp216s0) stat:    133227574 (    133,227,574) <= tx12_tso_bytes 
/sec
Ethtool(enp216s0) stat:        29139 (         29,139) <= 
tx12_tso_packets /sec
Ethtool(enp216s0) stat:         1559 (          1,559) <= tx12_xmit_more 
/sec
Ethtool(enp216s0) stat:        79682 (         79,682) <= 
tx13_added_vlan_packets /sec
Ethtool(enp216s0) stat:    189943899 (    189,943,899) <= tx13_bytes /sec
Ethtool(enp216s0) stat:        78110 (         78,110) <= tx13_cqes /sec
Ethtool(enp216s0) stat:        46294 (         46,294) <= tx13_csum_none 
/sec
Ethtool(enp216s0) stat:        33388 (         33,388) <= 
tx13_csum_partial /sec
Ethtool(enp216s0) stat:         2109 (          2,109) <= tx13_nop /sec
Ethtool(enp216s0) stat:       145877 (        145,877) <= tx13_packets /sec
Ethtool(enp216s0) stat:    136076991 (    136,076,991) <= tx13_tso_bytes 
/sec
Ethtool(enp216s0) stat:        29367 (         29,367) <= 
tx13_tso_packets /sec
Ethtool(enp216s0) stat:         1572 (          1,572) <= tx13_xmit_more 
/sec
Ethtool(enp216s0) stat:        86096 (         86,096) <= 
tx14_added_vlan_packets /sec
Ethtool(enp216s0) stat:    199318635 (    199,318,635) <= tx14_bytes /sec
Ethtool(enp216s0) stat:        84335 (         84,335) <= tx14_cqes /sec
Ethtool(enp216s0) stat:        50934 (         50,934) <= tx14_csum_none 
/sec
Ethtool(enp216s0) stat:        35163 (         35,163) <= 
tx14_csum_partial /sec
Ethtool(enp216s0) stat:         2263 (          2,263) <= tx14_nop /sec
Ethtool(enp216s0) stat:       156445 (        156,445) <= tx14_packets /sec
Ethtool(enp216s0) stat:    142293555 (    142,293,555) <= tx14_tso_bytes 
/sec
Ethtool(enp216s0) stat:        29703 (         29,703) <= 
tx14_tso_packets /sec
Ethtool(enp216s0) stat:         1761 (          1,761) <= tx14_xmit_more 
/sec
Ethtool(enp216s0) stat:        79698 (         79,698) <= 
tx15_added_vlan_packets /sec
Ethtool(enp216s0) stat:    189620424 (    189,620,424) <= tx15_bytes /sec
Ethtool(enp216s0) stat:        78356 (         78,356) <= tx15_cqes /sec
Ethtool(enp216s0) stat:        45528 (         45,528) <= tx15_csum_none 
/sec
Ethtool(enp216s0) stat:        34170 (         34,170) <= 
tx15_csum_partial /sec
Ethtool(enp216s0) stat:         2156 (          2,156) <= tx15_nop /sec
Ethtool(enp216s0) stat:       144935 (        144,935) <= tx15_packets /sec
Ethtool(enp216s0) stat:    135023821 (    135,023,821) <= tx15_tso_bytes 
/sec
Ethtool(enp216s0) stat:        29400 (         29,400) <= 
tx15_tso_packets /sec
Ethtool(enp216s0) stat:         1344 (          1,344) <= tx15_xmit_more 
/sec
Ethtool(enp216s0) stat:        59598 (         59,598) <= 
tx16_added_vlan_packets /sec
Ethtool(enp216s0) stat:    143187495 (    143,187,495) <= tx16_bytes /sec
Ethtool(enp216s0) stat:        58408 (         58,408) <= tx16_cqes /sec
Ethtool(enp216s0) stat:        35002 (         35,002) <= tx16_csum_none 
/sec
Ethtool(enp216s0) stat:        24595 (         24,595) <= 
tx16_csum_partial /sec
Ethtool(enp216s0) stat:         1585 (          1,585) <= tx16_nop /sec
Ethtool(enp216s0) stat:       110250 (        110,250) <= tx16_packets /sec
Ethtool(enp216s0) stat:    101339613 (    101,339,613) <= tx16_tso_bytes 
/sec
Ethtool(enp216s0) stat:        20698 (         20,698) <= 
tx16_tso_packets /sec
Ethtool(enp216s0) stat:         1179 (          1,179) <= tx16_xmit_more 
/sec
Ethtool(enp216s0) stat:        69504 (         69,504) <= 
tx17_added_vlan_packets /sec
Ethtool(enp216s0) stat:    171534675 (    171,534,675) <= tx17_bytes /sec
Ethtool(enp216s0) stat:        68155 (         68,155) <= tx17_cqes /sec
Ethtool(enp216s0) stat:        39445 (         39,445) <= tx17_csum_none 
/sec
Ethtool(enp216s0) stat:        30059 (         30,059) <= 
tx17_csum_partial /sec
Ethtool(enp216s0) stat:         1886 (          1,886) <= tx17_nop /sec
Ethtool(enp216s0) stat:       130910 (        130,910) <= tx17_packets /sec
Ethtool(enp216s0) stat:    123978012 (    123,978,012) <= tx17_tso_bytes 
/sec
Ethtool(enp216s0) stat:        26215 (         26,215) <= 
tx17_tso_packets /sec
Ethtool(enp216s0) stat:         1349 (          1,349) <= tx17_xmit_more 
/sec
Ethtool(enp216s0) stat:        58880 (         58,880) <= 
tx18_added_vlan_packets /sec
Ethtool(enp216s0) stat:    141863299 (    141,863,299) <= tx18_bytes /sec
Ethtool(enp216s0) stat:        57755 (         57,755) <= tx18_cqes /sec
Ethtool(enp216s0) stat:        33875 (         33,875) <= tx18_csum_none 
/sec
Ethtool(enp216s0) stat:        25005 (         25,005) <= 
tx18_csum_partial /sec
Ethtool(enp216s0) stat:         1592 (          1,592) <= tx18_nop /sec
Ethtool(enp216s0) stat:       110248 (        110,248) <= tx18_packets /sec
Ethtool(enp216s0) stat:    103105544 (    103,105,544) <= tx18_tso_bytes 
/sec
Ethtool(enp216s0) stat:        21243 (         21,243) <= 
tx18_tso_packets /sec
Ethtool(enp216s0) stat:         1108 (          1,108) <= tx18_xmit_more 
/sec
Ethtool(enp216s0) stat:        69804 (         69,804) <= 
tx19_added_vlan_packets /sec
Ethtool(enp216s0) stat:    164225730 (    164,225,730) <= tx19_bytes /sec
Ethtool(enp216s0) stat:        68756 (         68,756) <= tx19_cqes /sec
Ethtool(enp216s0) stat:        42003 (         42,003) <= tx19_csum_none 
/sec
Ethtool(enp216s0) stat:        27801 (         27,801) <= 
tx19_csum_partial /sec
Ethtool(enp216s0) stat:         1790 (          1,790) <= tx19_nop /sec
Ethtool(enp216s0) stat:       124794 (        124,794) <= tx19_packets /sec
Ethtool(enp216s0) stat:    110814620 (    110,814,620) <= tx19_tso_bytes 
/sec
Ethtool(enp216s0) stat:        23278 (         23,278) <= 
tx19_tso_packets /sec
Ethtool(enp216s0) stat:         1045 (          1,045) <= tx19_xmit_more 
/sec
Ethtool(enp216s0) stat:        79346 (         79,346) <= 
tx1_added_vlan_packets /sec
Ethtool(enp216s0) stat:    181179251 (    181,179,251) <= tx1_bytes /sec
Ethtool(enp216s0) stat:        78062 (         78,062) <= tx1_cqes /sec
Ethtool(enp216s0) stat:        46525 (         46,525) <= tx1_csum_none /sec
Ethtool(enp216s0) stat:        32821 (         32,821) <= 
tx1_csum_partial /sec
Ethtool(enp216s0) stat:         1996 (          1,996) <= tx1_nop /sec
Ethtool(enp216s0) stat:       142716 (        142,716) <= tx1_packets /sec
Ethtool(enp216s0) stat:    129507562 (    129,507,562) <= tx1_tso_bytes /sec
Ethtool(enp216s0) stat:        27579 (         27,579) <= 
tx1_tso_packets /sec
Ethtool(enp216s0) stat:         1281 (          1,281) <= tx1_xmit_more /sec
Ethtool(enp216s0) stat:        66641 (         66,641) <= 
tx20_added_vlan_packets /sec
Ethtool(enp216s0) stat:    161452661 (    161,452,661) <= tx20_bytes /sec
Ethtool(enp216s0) stat:        65374 (         65,374) <= tx20_cqes /sec
Ethtool(enp216s0) stat:        37657 (         37,657) <= tx20_csum_none 
/sec
Ethtool(enp216s0) stat:        28983 (         28,983) <= 
tx20_csum_partial /sec
Ethtool(enp216s0) stat:         1739 (          1,739) <= tx20_nop /sec
Ethtool(enp216s0) stat:       122824 (        122,824) <= tx20_packets /sec
Ethtool(enp216s0) stat:    115707977 (    115,707,977) <= tx20_tso_bytes 
/sec
Ethtool(enp216s0) stat:        24823 (         24,823) <= 
tx20_tso_packets /sec
Ethtool(enp216s0) stat:         1263 (          1,263) <= tx20_xmit_more 
/sec
Ethtool(enp216s0) stat:        60564 (         60,564) <= 
tx21_added_vlan_packets /sec
Ethtool(enp216s0) stat:    138260273 (    138,260,273) <= tx21_bytes /sec
Ethtool(enp216s0) stat:        59611 (         59,611) <= tx21_cqes /sec
Ethtool(enp216s0) stat:        36105 (         36,105) <= tx21_csum_none 
/sec
Ethtool(enp216s0) stat:        24459 (         24,459) <= 
tx21_csum_partial /sec
Ethtool(enp216s0) stat:         1490 (          1,490) <= tx21_nop /sec
Ethtool(enp216s0) stat:       106913 (        106,913) <= tx21_packets /sec
Ethtool(enp216s0) stat:     97216016 (     97,216,016) <= tx21_tso_bytes 
/sec
Ethtool(enp216s0) stat:        21944 (         21,944) <= 
tx21_tso_packets /sec
Ethtool(enp216s0) stat:          930 (            930) <= tx21_xmit_more 
/sec
Ethtool(enp216s0) stat:        66527 (         66,527) <= 
tx22_added_vlan_packets /sec
Ethtool(enp216s0) stat:    149686537 (    149,686,537) <= tx22_bytes /sec
Ethtool(enp216s0) stat:        65262 (         65,262) <= tx22_cqes /sec
Ethtool(enp216s0) stat:        40609 (         40,609) <= tx22_csum_none 
/sec
Ethtool(enp216s0) stat:        25918 (         25,918) <= 
tx22_csum_partial /sec
Ethtool(enp216s0) stat:         1676 (          1,676) <= tx22_nop /sec
Ethtool(enp216s0) stat:       118071 (        118,071) <= tx22_packets /sec
Ethtool(enp216s0) stat:    104822520 (    104,822,520) <= tx22_tso_bytes 
/sec
Ethtool(enp216s0) stat:        22046 (         22,046) <= 
tx22_tso_packets /sec
Ethtool(enp216s0) stat:         1265 (          1,265) <= tx22_xmit_more 
/sec
Ethtool(enp216s0) stat:        59973 (         59,973) <= 
tx23_added_vlan_packets /sec
Ethtool(enp216s0) stat:    147788578 (    147,788,578) <= tx23_bytes /sec
Ethtool(enp216s0) stat:        58827 (         58,827) <= tx23_cqes /sec
Ethtool(enp216s0) stat:        33197 (         33,197) <= tx23_csum_none 
/sec
Ethtool(enp216s0) stat:        26776 (         26,776) <= 
tx23_csum_partial /sec
Ethtool(enp216s0) stat:         1669 (          1,669) <= tx23_nop /sec
Ethtool(enp216s0) stat:       112053 (        112,053) <= tx23_packets /sec
Ethtool(enp216s0) stat:    106098597 (    106,098,597) <= tx23_tso_bytes 
/sec
Ethtool(enp216s0) stat:        22433 (         22,433) <= 
tx23_tso_packets /sec
Ethtool(enp216s0) stat:         1126 (          1,126) <= tx23_xmit_more 
/sec
Ethtool(enp216s0) stat:        58819 (         58,819) <= 
tx24_added_vlan_packets /sec
Ethtool(enp216s0) stat:    146231570 (    146,231,570) <= tx24_bytes /sec
Ethtool(enp216s0) stat:        57661 (         57,661) <= tx24_cqes /sec
Ethtool(enp216s0) stat:        32150 (         32,150) <= tx24_csum_none 
/sec
Ethtool(enp216s0) stat:        26669 (         26,669) <= 
tx24_csum_partial /sec
Ethtool(enp216s0) stat:         1578 (          1,578) <= tx24_nop /sec
Ethtool(enp216s0) stat:       111230 (        111,230) <= tx24_packets /sec
Ethtool(enp216s0) stat:    106359530 (    106,359,530) <= tx24_tso_bytes 
/sec
Ethtool(enp216s0) stat:        22402 (         22,402) <= 
tx24_tso_packets /sec
Ethtool(enp216s0) stat:         1158 (          1,158) <= tx24_xmit_more 
/sec
Ethtool(enp216s0) stat:        64116 (         64,116) <= 
tx25_added_vlan_packets /sec
Ethtool(enp216s0) stat:    156132090 (    156,132,090) <= tx25_bytes /sec
Ethtool(enp216s0) stat:        62901 (         62,901) <= tx25_cqes /sec
Ethtool(enp216s0) stat:        36357 (         36,357) <= tx25_csum_none 
/sec
Ethtool(enp216s0) stat:        27759 (         27,759) <= 
tx25_csum_partial /sec
Ethtool(enp216s0) stat:         1717 (          1,717) <= tx25_nop /sec
Ethtool(enp216s0) stat:       118618 (        118,618) <= tx25_packets /sec
Ethtool(enp216s0) stat:    110752893 (    110,752,893) <= tx25_tso_bytes 
/sec
Ethtool(enp216s0) stat:        23282 (         23,282) <= 
tx25_tso_packets /sec
Ethtool(enp216s0) stat:         1212 (          1,212) <= tx25_xmit_more 
/sec
Ethtool(enp216s0) stat:        62028 (         62,028) <= 
tx26_added_vlan_packets /sec
Ethtool(enp216s0) stat:    144966495 (    144,966,495) <= tx26_bytes /sec
Ethtool(enp216s0) stat:        60906 (         60,906) <= tx26_cqes /sec
Ethtool(enp216s0) stat:        36345 (         36,345) <= tx26_csum_none 
/sec
Ethtool(enp216s0) stat:        25684 (         25,684) <= 
tx26_csum_partial /sec
Ethtool(enp216s0) stat:         1615 (          1,615) <= tx26_nop /sec
Ethtool(enp216s0) stat:       112487 (        112,487) <= tx26_packets /sec
Ethtool(enp216s0) stat:    102083200 (    102,083,200) <= tx26_tso_bytes 
/sec
Ethtool(enp216s0) stat:        21598 (         21,598) <= 
tx26_tso_packets /sec
Ethtool(enp216s0) stat:         1123 (          1,123) <= tx26_xmit_more 
/sec
Ethtool(enp216s0) stat:        64029 (         64,029) <= 
tx27_added_vlan_packets /sec
Ethtool(enp216s0) stat:    154747343 (    154,747,343) <= tx27_bytes /sec
Ethtool(enp216s0) stat:        62881 (         62,881) <= tx27_cqes /sec
Ethtool(enp216s0) stat:        35343 (         35,343) <= tx27_csum_none 
/sec
Ethtool(enp216s0) stat:        28686 (         28,686) <= 
tx27_csum_partial /sec
Ethtool(enp216s0) stat:         1707 (          1,707) <= tx27_nop /sec
Ethtool(enp216s0) stat:       118074 (        118,074) <= tx27_packets /sec
Ethtool(enp216s0) stat:    110960676 (    110,960,676) <= tx27_tso_bytes 
/sec
Ethtool(enp216s0) stat:        24139 (         24,139) <= 
tx27_tso_packets /sec
Ethtool(enp216s0) stat:         1148 (          1,148) <= tx27_xmit_more 
/sec
Ethtool(enp216s0) stat:        82542 (         82,542) <= 
tx2_added_vlan_packets /sec
Ethtool(enp216s0) stat:    183007051 (    183,007,051) <= tx2_bytes /sec
Ethtool(enp216s0) stat:        81320 (         81,320) <= tx2_cqes /sec
Ethtool(enp216s0) stat:        49989 (         49,989) <= tx2_csum_none /sec
Ethtool(enp216s0) stat:        32553 (         32,553) <= 
tx2_csum_partial /sec
Ethtool(enp216s0) stat:         2142 (          2,142) <= tx2_nop /sec
Ethtool(enp216s0) stat:       143613 (        143,613) <= tx2_packets /sec
Ethtool(enp216s0) stat:    126895404 (    126,895,404) <= tx2_tso_bytes /sec
Ethtool(enp216s0) stat:        28173 (         28,173) <= 
tx2_tso_packets /sec
Ethtool(enp216s0) stat:         1220 (          1,220) <= tx2_xmit_more /sec
Ethtool(enp216s0) stat:        78737 (         78,737) <= 
tx3_added_vlan_packets /sec
Ethtool(enp216s0) stat:    188546556 (    188,546,556) <= tx3_bytes /sec
Ethtool(enp216s0) stat:        77389 (         77,389) <= tx3_cqes /sec
Ethtool(enp216s0) stat:        46147 (         46,147) <= tx3_csum_none /sec
Ethtool(enp216s0) stat:        32590 (         32,590) <= 
tx3_csum_partial /sec
Ethtool(enp216s0) stat:         2171 (          2,171) <= tx3_nop /sec
Ethtool(enp216s0) stat:       144570 (        144,570) <= tx3_packets /sec
Ethtool(enp216s0) stat:    134231500 (    134,231,500) <= tx3_tso_bytes /sec
Ethtool(enp216s0) stat:        28857 (         28,857) <= 
tx3_tso_packets /sec
Ethtool(enp216s0) stat:         1348 (          1,348) <= tx3_xmit_more /sec
Ethtool(enp216s0) stat:        76284 (         76,284) <= 
tx4_added_vlan_packets /sec
Ethtool(enp216s0) stat:    185633403 (    185,633,403) <= tx4_bytes /sec
Ethtool(enp216s0) stat:        74710 (         74,710) <= tx4_cqes /sec
Ethtool(enp216s0) stat:        42803 (         42,803) <= tx4_csum_none /sec
Ethtool(enp216s0) stat:        33482 (         33,482) <= 
tx4_csum_partial /sec
Ethtool(enp216s0) stat:         2005 (          2,005) <= tx4_nop /sec
Ethtool(enp216s0) stat:       139978 (        139,978) <= tx4_packets /sec
Ethtool(enp216s0) stat:    131762891 (    131,762,891) <= tx4_tso_bytes /sec
Ethtool(enp216s0) stat:        28843 (         28,843) <= 
tx4_tso_packets /sec
Ethtool(enp216s0) stat:         1563 (          1,563) <= tx4_xmit_more /sec
Ethtool(enp216s0) stat:        83878 (         83,878) <= 
tx5_added_vlan_packets /sec
Ethtool(enp216s0) stat:    197920426 (    197,920,426) <= tx5_bytes /sec
Ethtool(enp216s0) stat:        82146 (         82,146) <= tx5_cqes /sec
Ethtool(enp216s0) stat:        48122 (         48,122) <= tx5_csum_none /sec
Ethtool(enp216s0) stat:        35755 (         35,755) <= 
tx5_csum_partial /sec
Ethtool(enp216s0) stat:         2227 (          2,227) <= tx5_nop /sec
Ethtool(enp216s0) stat:       151214 (        151,214) <= tx5_packets /sec
Ethtool(enp216s0) stat:    139113072 (    139,113,072) <= tx5_tso_bytes /sec
Ethtool(enp216s0) stat:        30390 (         30,390) <= 
tx5_tso_packets /sec
Ethtool(enp216s0) stat:         1730 (          1,730) <= tx5_xmit_more /sec
Ethtool(enp216s0) stat:        76307 (         76,307) <= 
tx6_added_vlan_packets /sec
Ethtool(enp216s0) stat:    182153538 (    182,153,538) <= tx6_bytes /sec
Ethtool(enp216s0) stat:        74744 (         74,744) <= tx6_cqes /sec
Ethtool(enp216s0) stat:        44381 (         44,381) <= tx6_csum_none /sec
Ethtool(enp216s0) stat:        31926 (         31,926) <= 
tx6_csum_partial /sec
Ethtool(enp216s0) stat:         2068 (          2,068) <= tx6_nop /sec
Ethtool(enp216s0) stat:       139745 (        139,745) <= tx6_packets /sec
Ethtool(enp216s0) stat:    129175625 (    129,175,625) <= tx6_tso_bytes /sec
Ethtool(enp216s0) stat:        27349 (         27,349) <= 
tx6_tso_packets /sec
Ethtool(enp216s0) stat:         1562 (          1,562) <= tx6_xmit_more /sec
Ethtool(enp216s0) stat:        72151 (         72,151) <= 
tx7_added_vlan_packets /sec
Ethtool(enp216s0) stat:    175340247 (    175,340,247) <= tx7_bytes /sec
Ethtool(enp216s0) stat:        70565 (         70,565) <= tx7_cqes /sec
Ethtool(enp216s0) stat:        40824 (         40,824) <= tx7_csum_none /sec
Ethtool(enp216s0) stat:        31327 (         31,327) <= 
tx7_csum_partial /sec
Ethtool(enp216s0) stat:         1962 (          1,962) <= tx7_nop /sec
Ethtool(enp216s0) stat:       133790 (        133,790) <= tx7_packets /sec
Ethtool(enp216s0) stat:    124931972 (    124,931,972) <= tx7_tso_bytes /sec
Ethtool(enp216s0) stat:        26314 (         26,314) <= 
tx7_tso_packets /sec
Ethtool(enp216s0) stat:         1586 (          1,586) <= tx7_xmit_more /sec
Ethtool(enp216s0) stat:        83983 (         83,983) <= 
tx8_added_vlan_packets /sec
Ethtool(enp216s0) stat:    203970262 (    203,970,262) <= tx8_bytes /sec
Ethtool(enp216s0) stat:        82475 (         82,475) <= tx8_cqes /sec
Ethtool(enp216s0) stat:        47937 (         47,937) <= tx8_csum_none /sec
Ethtool(enp216s0) stat:        36046 (         36,046) <= 
tx8_csum_partial /sec
Ethtool(enp216s0) stat:         2203 (          2,203) <= tx8_nop /sec
Ethtool(enp216s0) stat:       153743 (        153,743) <= tx8_packets /sec
Ethtool(enp216s0) stat:    143296525 (    143,296,525) <= tx8_tso_bytes /sec
Ethtool(enp216s0) stat:        30936 (         30,936) <= 
tx8_tso_packets /sec
Ethtool(enp216s0) stat:         1500 (          1,500) <= tx8_xmit_more /sec
Ethtool(enp216s0) stat:        79406 (         79,406) <= 
tx9_added_vlan_packets /sec
Ethtool(enp216s0) stat:    183286836 (    183,286,836) <= tx9_bytes /sec
Ethtool(enp216s0) stat:        77769 (         77,769) <= tx9_cqes /sec
Ethtool(enp216s0) stat:        48014 (         48,014) <= tx9_csum_none /sec
Ethtool(enp216s0) stat:        31392 (         31,392) <= 
tx9_csum_partial /sec
Ethtool(enp216s0) stat:         2146 (          2,146) <= tx9_nop /sec
Ethtool(enp216s0) stat:       142229 (        142,229) <= tx9_packets /sec
Ethtool(enp216s0) stat:    127419807 (    127,419,807) <= tx9_tso_bytes /sec
Ethtool(enp216s0) stat:        26644 (         26,644) <= 
tx9_tso_packets /sec
Ethtool(enp216s0) stat:         1633 (          1,633) <= tx9_xmit_more /sec
Ethtool(enp216s0) stat:      2050335 (      2,050,335) <= 
tx_added_vlan_packets /sec
Ethtool(enp216s0) stat:   4851410368 (  4,851,410,368) <= tx_bytes /sec
Ethtool(enp216s0) stat:   4881854912 (  4,881,854,912) <= tx_bytes_phy /sec
Ethtool(enp216s0) stat:      2011989 (      2,011,989) <= tx_cqes /sec
Ethtool(enp216s0) stat:      1191137 (      1,191,137) <= tx_csum_none /sec
Ethtool(enp216s0) stat:       859197 (        859,197) <= 
tx_csum_partial /sec
Ethtool(enp216s0) stat:        54140 (         54,140) <= tx_nop /sec
Ethtool(enp216s0) stat:      3728759 (      3,728,759) <= tx_packets /sec
Ethtool(enp216s0) stat:      3729416 (      3,729,416) <= tx_packets_phy 
/sec
Ethtool(enp216s0) stat:   4882774572 (  4,882,774,572) <= tx_prio0_bytes 
/sec
Ethtool(enp216s0) stat:      3730195 (      3,730,195) <= 
tx_prio0_packets /sec
Ethtool(enp216s0) stat:   3431642829 (  3,431,642,829) <= tx_tso_bytes /sec
Ethtool(enp216s0) stat:       734343 (        734,343) <= tx_tso_packets 
/sec
Ethtool(enp216s0) stat:   4866327192 (  4,866,327,192) <= 
tx_vport_unicast_bytes /sec
Ethtool(enp216s0) stat:      3728966 (      3,728,966) <= 
tx_vport_unicast_packets /sec
Ethtool(enp216s0) stat:        38268 (         38,268) <= tx_xmit_more /sec


>>
>> Can you give output put from:
>>   $ ethtool --show-priv-flag DEVICE
>>
>> I want you to experiment with:
> ethtool --show-priv-flags enp175s0
> Private flags for enp175s0:
> rx_cqe_moder       : on
> tx_cqe_moder       : off
> rx_cqe_compress    : off
> rx_striding_rq     : on
> rx_no_csum_complete: off
>
>>
>>   ethtool --set-priv-flags DEVICE rx_striding_rq off
> ok i will first check on test server if this will reset my interface 
> and will not produce kernel panic :)
>>
>> I think you already have played with 'rx_cqe_compress', right.
> yes - and compress increasing number of irq's but doing not much for 
> bandwidth same limit 60-64Gbit/s total RX+TX on one 100G port
>
> And what is weird - that limit is in overall symetric - cause if for 
> example 100G port is receiving 42G traffic and transmitting 20G 
> traffic - when i flood rx side with pktgen or other for example icmp 
> traffic 1/2/3/4/5G - then receiving side increase with 1/2/3/4/5Gbit 
> of traffic but transmitting is going down for same lvl's
>
>
>
>

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ