lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening PHC | |
Open Source and information security mailing list archives
| ||
|
Date: Fri, 25 Mar 2022 20:44:18 +0800 From: Mingbao Sun <sunmingbao@....com> To: Sagi Grimberg <sagi@...mberg.me> Cc: Keith Busch <kbusch@...nel.org>, Jens Axboe <axboe@...com>, Christoph Hellwig <hch@....de>, Chaitanya Kulkarni <kch@...dia.com>, linux-nvme@...ts.infradead.org, linux-kernel@...r.kernel.org, Eric Dumazet <edumazet@...gle.com>, "David S . Miller" <davem@...emloft.net>, Hideaki YOSHIFUJI <yoshfuji@...ux-ipv6.org>, David Ahern <dsahern@...nel.org>, Jakub Kicinski <kuba@...nel.org>, netdev@...r.kernel.org, tyler.sun@...l.com, ping.gan@...l.com, yanxiu.cai@...l.com, libin.zhang@...l.com, ao.sun@...l.com Subject: Re: [PATCH v2 2/3] nvme-tcp: support specifying the congestion-control [Combination 2] cubic + switch ECN-marking on This is a bad scenario. Since the switch supports ECN-marking, but cubic can not take advantage of that. So we can see the bandwidth of each TX node could suddenly drop several hundreds MB/S due to packet dropping and timeout-retransmission. And the total bandwidth on the RX node can not reachto the full link bandwidth (which is about 6 GB/S). /* * before loading traffic, clear the counters on the 2 switches. */ hound-dirt# clear qos statistics type queuing interface ethernet 1/1/4 hound-dirt# show queuing statistics interface ethernet 1/1/4 Interface ethernet1/1/4 Queue Packets Bytes Dropped-Packets Dropped-Bytes 0 0 0 0 0 1 0 0 0 0 2 0 0 0 0 3 0 0 0 0 4 0 0 0 0 5 0 0 0 0 6 0 0 0 0 7 0 0 0 0 hound-dirt# fox-dirt# clear qos statistics type queuing interface ethernet 1/1/4 fox-dirt# show queuing statistics interface ethernet 1/1/4 Interface ethernet1/1/4 Queue Packets Bytes Dropped-Packets Dropped-Bytes 0 0 0 0 0 1 0 0 0 0 2 0 0 0 0 3 0 0 0 0 4 0 0 0 0 5 0 0 0 0 6 0 0 0 0 7 0 0 0 0 fox-dirt# /* * logs of RX node. */ ogden-dirt:/home/admin/tyler # echo cubic >/proc/sys/net/ipv4/tcp_congestion_control ogden-dirt:/home/admin/tyler # cat /proc/sys/net/ipv4/tcp_congestion_control cubic ogden-dirt:/home/admin/tyler # date; ./tcp_perf.exe --server-ip 169.254.85.4 --server-port 10001 --msg-len 65536 --conn-num 24 -s -r --data-random --svr-no-wait-all-conn-OK --show-bandwidth-only Fri Mar 25 08:23:13 EDT 2022 DBG:/mnt/hgfs/src/linux-dev-framework-master/libs/app_utils/src/socket.c(104)-socket_init_2: bind socket fd 3 to 169.254.85.4:10001 succeed DBG:perf_frmwk.c(1472)-create_tcp_conns_start_load: start listen on fd 3 conn [0] local 169.254.85.4:10001 peer 169.254.85.3:59676 created. rx thread of conn 0 started conn [1] local 169.254.85.4:10001 peer 169.254.85.3:59678 created. rx thread of conn 1 started conn [2] local 169.254.85.4:10001 peer 169.254.85.3:59680 created. rx thread of conn 2 started conn [3] local 169.254.85.4:10001 peer 169.254.85.3:59682 created. rx thread of conn 3 started conn [4] local 169.254.85.4:10001 peer 169.254.85.3:59684 created. conn [5] local 169.254.85.4:10001 peer 169.254.85.3:59686 created. rx thread of conn 4 started conn [6] local 169.254.85.4:10001 peer 169.254.85.3:59688 created. rx thread of conn 5 started rx thread of conn 6 started conn [7] local 169.254.85.4:10001 peer 169.254.85.3:59690 created. rx thread of conn 7 started conn [8] local 169.254.85.4:10001 peer 169.254.85.2:37974 created. rx thread of conn 8 started conn [9] local 169.254.85.4:10001 peer 169.254.85.2:37978 created. rx thread of conn 9 started conn [10] local 169.254.85.4:10001 peer 169.254.85.2:37980 created. rx thread of conn 10 started conn [11] local 169.254.85.4:10001 peer 169.254.85.2:37982 created. rx thread of conn 11 started conn [12] local 169.254.85.4:10001 peer 169.254.85.2:37984 created. rx thread of conn 12 started conn [13] local 169.254.85.4:10001 peer 169.254.85.2:37986 created. conn [14] local 169.254.85.4:10001 peer 169.254.85.2:37988 created. rx thread of conn 13 started conn [15] local 169.254.85.4:10001 peer 169.254.85.2:37990 created. rx thread of conn 14 started rx thread of conn 15 started [time lasts]: 1 rx_msg_succ_bytes 0x1625d0000 (5,945,229,312) conn [16] local 169.254.85.4:10001 peer 169.254.85.1:49484 created. rx thread of conn 16 started conn [17] local 169.254.85.4:10001 peer 169.254.85.1:49486 created. rx thread of conn 17 started conn [18] local 169.254.85.4:10001 peer 169.254.85.1:49490 created. rx thread of conn 18 started conn [19] local 169.254.85.4:10001 peer 169.254.85.1:49492 created. rx thread of conn 19 started conn [20] local 169.254.85.4:10001 peer 169.254.85.1:49494 created. rx thread of conn 20 started conn [21] local 169.254.85.4:10001 peer 169.254.85.1:49496 created. rx thread of conn 21 started conn [22] local 169.254.85.4:10001 peer 169.254.85.1:49498 created. rx thread of conn 22 started conn [23] local 169.254.85.4:10001 peer 169.254.85.1:49488 created. 24 connection(s) created in total rx thread of conn 23 started [time lasts]: 2 rx_msg_succ_bytes 0x153a90000 (5,698,551,808) [time lasts]: 3 rx_msg_succ_bytes 0x1589a0000 (5,781,454,848) [time lasts]: 4 rx_msg_succ_bytes 0x156770000 (5,745,606,656) [time lasts]: 5 rx_msg_succ_bytes 0x155e30000 (5,735,907,328) [time lasts]: 6 rx_msg_succ_bytes 0x1583f0000 (5,775,491,072) [time lasts]: 7 rx_msg_succ_bytes 0x159de0000 (5,802,688,512) [time lasts]: 8 rx_msg_succ_bytes 0x153b00000 (5,699,010,560) [time lasts]: 9 rx_msg_succ_bytes 0x153a50000 (5,698,289,664) [time lasts]: 10 rx_msg_succ_bytes 0x1568a0000 (5,746,851,840) [time lasts]: 11 rx_msg_succ_bytes 0x157bb0000 (5,766,840,320) [time lasts]: 12 rx_msg_succ_bytes 0x15a2a0000 (5,807,669,248) [time lasts]: 13 rx_msg_succ_bytes 0x155020000 (5,721,161,728) [time lasts]: 14 rx_msg_succ_bytes 0x154b40000 (5,716,049,920) [time lasts]: 15 rx_msg_succ_bytes 0x157610000 (5,760,942,080) [time lasts]: 16 rx_msg_succ_bytes 0x157cd0000 (5,768,019,968) [time lasts]: 17 rx_msg_succ_bytes 0x153cf0000 (5,701,042,176) [time lasts]: 18 rx_msg_succ_bytes 0x151820000 (5,662,441,472) [time lasts]: 19 rx_msg_succ_bytes 0x153fa0000 (5,703,860,224) [time lasts]: 20 rx_msg_succ_bytes 0x150af0000 (5,648,613,376) [time lasts]: 21 rx_msg_succ_bytes 0x152a70000 (5,681,643,520) [time lasts]: 22 rx_msg_succ_bytes 0x154e10000 (5,718,999,040) [time lasts]: 23 rx_msg_succ_bytes 0x157d60000 (5,768,609,792) [time lasts]: 24 rx_msg_succ_bytes 0x1581d0000 (5,773,262,848) [time lasts]: 25 rx_msg_succ_bytes 0x157240000 (5,756,944,384) [time lasts]: 26 rx_msg_succ_bytes 0x159e50000 (5,803,147,264) [time lasts]: 27 rx_msg_succ_bytes 0x1535b0000 (5,693,440,000) [time lasts]: 28 rx_msg_succ_bytes 0x157390000 (5,758,320,640) [time lasts]: 29 rx_msg_succ_bytes 0x155620000 (5,727,453,184) [time lasts]: 30 rx_msg_succ_bytes 0x153c80000 (5,700,583,424) [time lasts]: 31 rx_msg_succ_bytes 0x154cb0000 (5,717,557,248) [time lasts]: 32 rx_msg_succ_bytes 0x153ea0000 (5,702,811,648) [time lasts]: 33 rx_msg_succ_bytes 0x1577f0000 (5,762,908,160) [time lasts]: 34 rx_msg_succ_bytes 0x158090000 (5,771,952,128) [time lasts]: 35 rx_msg_succ_bytes 0x154040000 (5,704,515,584) [time lasts]: 36 rx_msg_succ_bytes 0x1577f0000 (5,762,908,160) [time lasts]: 37 rx_msg_succ_bytes 0x154900000 (5,713,690,624) [time lasts]: 38 rx_msg_succ_bytes 0x155750000 (5,728,698,368) [time lasts]: 39 rx_msg_succ_bytes 0x154ca0000 (5,717,491,712) [time lasts]: 40 rx_msg_succ_bytes 0x158110000 (5,772,476,416) [time lasts]: 41 rx_msg_succ_bytes 0x155c90000 (5,734,203,392) [time lasts]: 42 rx_msg_succ_bytes 0x1563a0000 (5,741,608,960) [time lasts]: 43 rx_msg_succ_bytes 0x1566e0000 (5,745,016,832) [time lasts]: 44 rx_msg_succ_bytes 0x158d20000 (5,785,124,864) [time lasts]: 45 rx_msg_succ_bytes 0x156c70000 (5,750,849,536) [time lasts]: 46 rx_msg_succ_bytes 0x1536c0000 (5,694,554,112) [time lasts]: 47 rx_msg_succ_bytes 0x15b030000 (5,821,890,560) [time lasts]: 48 rx_msg_succ_bytes 0x156ac0000 (5,749,080,064) [time lasts]: 49 rx_msg_succ_bytes 0x151950000 (5,663,686,656) [time lasts]: 50 rx_msg_succ_bytes 0x156f80000 (5,754,060,800) [time lasts]: 51 rx_msg_succ_bytes 0x158250000 (5,773,787,136) [time lasts]: 52 rx_msg_succ_bytes 0x156830000 (5,746,393,088) [time lasts]: 53 rx_msg_succ_bytes 0x157470000 (5,759,238,144) [time lasts]: 54 rx_msg_succ_bytes 0x1557a0000 (5,729,026,048) [time lasts]: 55 rx_msg_succ_bytes 0x156270000 (5,740,363,776) [time lasts]: 56 rx_msg_succ_bytes 0x158da0000 (5,785,649,152) [time lasts]: 57 rx_msg_succ_bytes 0x158980000 (5,781,323,776) [time lasts]: 58 rx_msg_succ_bytes 0x154fe0000 (5,720,899,584) [time lasts]: 59 rx_msg_succ_bytes 0x155530000 (5,726,470,144) [time lasts]: 60 rx_msg_succ_bytes 0x156da0000 (5,752,094,720) [time lasts]: 61 rx_msg_succ_bytes 0x155f70000 (5,737,218,048) [time lasts]: 62 rx_msg_succ_bytes 0x154500000 (5,709,496,320) [time lasts]: 63 rx_msg_succ_bytes 0x157c50000 (5,767,495,680) [time lasts]: 64 rx_msg_succ_bytes 0x1550b0000 (5,721,751,552) [time lasts]: 65 rx_msg_succ_bytes 0x1586c0000 (5,778,440,192) ^Ccaught signal 2 /* * logs of TX node 1. */ provo-dirt:/home/admin/tyler # echo cubic >/proc/sys/net/ipv4/tcp_congestion_control provo-dirt:/home/admin/tyler # cat /proc/sys/net/ipv4/tcp_congestion_control cubic provo-dirt:/home/admin/tyler # date; ./tcp_perf.exe --server-ip 169.254.85.4 --server-port 10001 --msg-len 65536 --conn-num 8 -c -t --data-random --show-bandwidth-only Fri Mar 25 08:23:18 EDT 2022 conn [0] local 169.254.85.1:49484 peer 169.254.85.4:10001 created. conn [1] local 169.254.85.1:49486 peer 169.254.85.4:10001 created. conn [2] local 169.254.85.1:49488 peer 169.254.85.4:10001 created. conn [3] local 169.254.85.1:49490 peer 169.254.85.4:10001 created. conn [4] local 169.254.85.1:49492 peer 169.254.85.4:10001 created. conn [5] local 169.254.85.1:49494 peer 169.254.85.4:10001 created. conn [6] local 169.254.85.1:49496 peer 169.254.85.4:10001 created. conn [7] local 169.254.85.1:49498 peer 169.254.85.4:10001 created. 8 connection(s) created in total tx thread of conn 0 started tx thread of conn 1 started tx thread of conn 2 started tx thread of conn 3 started tx thread of conn 4 started tx thread of conn 5 started tx thread of conn 6 started tx thread of conn 7 started [time lasts]: 1 tx_succ_bytes 0x76930000 (1,989,345,280) [time lasts]: 2 tx_succ_bytes 0x639f0000 (1,671,364,608) [time lasts]: 3 tx_succ_bytes 0x7e1d0000 (2,115,829,760) [time lasts]: 4 tx_succ_bytes 0x770b0000 (1,997,209,600) [time lasts]: 5 tx_succ_bytes 0x68900000 (1,754,267,648) [time lasts]: 6 tx_succ_bytes 0x53dd0000 (1,406,992,384) [time lasts]: 7 tx_succ_bytes 0x6eab0000 (1,856,700,416) [time lasts]: 8 tx_succ_bytes 0x4c200000 (1,277,165,568) [time lasts]: 9 tx_succ_bytes 0x6eab0000 (1,856,700,416) [time lasts]: 10 tx_succ_bytes 0x793e0000 (2,034,106,368) [time lasts]: 11 tx_succ_bytes 0x5b7a0000 (1,534,722,048) [time lasts]: 12 tx_succ_bytes 0x6c220000 (1,814,167,552) [time lasts]: 13 tx_succ_bytes 0x65660000 (1,701,183,488) [time lasts]: 14 tx_succ_bytes 0x77860000 (2,005,270,528) [time lasts]: 15 tx_succ_bytes 0x6eff0000 (1,862,205,440) [time lasts]: 16 tx_succ_bytes 0x65330000 (1,697,841,152) [time lasts]: 17 tx_succ_bytes 0x73600000 (1,935,671,296) [time lasts]: 18 tx_succ_bytes 0x7cf30000 (2,096,300,032) [time lasts]: 19 tx_succ_bytes 0x838d0000 (2,207,055,872) [time lasts]: 20 tx_succ_bytes 0x8ea90000 (2,393,440,256) [time lasts]: 21 tx_succ_bytes 0x5ef50000 (1,593,114,624) [time lasts]: 22 tx_succ_bytes 0x693e0000 (1,765,670,912) [time lasts]: 23 tx_succ_bytes 0x64390000 (1,681,457,152) [time lasts]: 24 tx_succ_bytes 0x69400000 (1,765,801,984) [time lasts]: 25 tx_succ_bytes 0x69fb0000 (1,778,057,216) [time lasts]: 26 tx_succ_bytes 0x710a0000 (1,896,480,768) [time lasts]: 27 tx_succ_bytes 0x70170000 (1,880,555,520) [time lasts]: 28 tx_succ_bytes 0x7ed30000 (2,127,757,312) [time lasts]: 29 tx_succ_bytes 0x70830000 (1,887,633,408) [time lasts]: 30 tx_succ_bytes 0x6fcb0000 (1,875,574,784) [time lasts]: 31 tx_succ_bytes 0x6dc30000 (1,841,496,064) [time lasts]: 32 tx_succ_bytes 0x7f060000 (2,131,099,648) [time lasts]: 33 tx_succ_bytes 0x79760000 (2,037,776,384) [time lasts]: 34 tx_succ_bytes 0x78f30000 (2,029,191,168) [time lasts]: 35 tx_succ_bytes 0x621a0000 (1,645,871,104) [time lasts]: 36 tx_succ_bytes 0x7a5d0000 (2,052,915,200) [time lasts]: 37 tx_succ_bytes 0x7fb80000 (2,142,765,056) [time lasts]: 38 tx_succ_bytes 0x73ed0000 (1,944,911,872) [time lasts]: 39 tx_succ_bytes 0x6fb20000 (1,873,936,384) [time lasts]: 40 tx_succ_bytes 0x72110000 (1,913,716,736) [time lasts]: 41 tx_succ_bytes 0x59240000 (1,495,531,520) [time lasts]: 42 tx_succ_bytes 0x55e30000 (1,440,940,032) [time lasts]: 43 tx_succ_bytes 0x69e20000 (1,776,418,816) [time lasts]: 44 tx_succ_bytes 0x849a0000 (2,224,685,056) [time lasts]: 45 tx_succ_bytes 0x7a930000 (2,056,454,144) [time lasts]: 46 tx_succ_bytes 0x79230000 (2,032,336,896) [time lasts]: 47 tx_succ_bytes 0x6d690000 (1,835,597,824) [time lasts]: 48 tx_succ_bytes 0x77c60000 (2,009,464,832) [time lasts]: 49 tx_succ_bytes 0x5c670000 (1,550,254,080) [time lasts]: 50 tx_succ_bytes 0x6a210000 (1,780,547,584) [time lasts]: 51 tx_succ_bytes 0x64ef0000 (1,693,384,704) [time lasts]: 52 tx_succ_bytes 0x810a0000 (2,164,916,224) [time lasts]: 53 tx_succ_bytes 0x7a4f0000 (2,051,997,696) [time lasts]: 54 tx_succ_bytes 0x70ac0000 (1,890,320,384) [time lasts]: 55 tx_succ_bytes 0x92260000 (2,451,963,904) [time lasts]: 56 tx_succ_bytes 0x5d500000 (1,565,523,968) [time lasts]: 57 tx_succ_bytes 0x59030000 (1,493,368,832) [time lasts]: 58 tx_succ_bytes 0x7ad60000 (2,060,845,056) [time lasts]: 59 tx_succ_bytes 0x765a0000 (1,985,609,728) [time lasts]: 60 tx_succ_bytes 0x6c1e0000 (1,813,905,408) [time lasts]: 61 tx_succ_bytes 0x6eeb0000 (1,860,894,720) [time lasts]: 62 tx_succ_bytes 0x6ab50000 (1,790,246,912) [time lasts]: 63 tx_succ_bytes 0x64cf0000 (1,691,287,552) /* * logs of TX node 2. */ sandy-dirt:/home/admin/tyler # echo cubic >/proc/sys/net/ipv4/tcp_congestion_control sandy-dirt:/home/admin/tyler # cat /proc/sys/net/ipv4/tcp_congestion_control cubic sandy-dirt:/home/admin/tyler # date; ./tcp_perf.exe --server-ip 169.254.85.4 --server-port 10001 --msg-len 65536 --conn-num 8 -c -t --data-random --show-bandwidth-only Fri Mar 25 08:23:16 EDT 2022 conn [0] local 169.254.85.2:37974 peer 169.254.85.4:10001 created. conn [1] local 169.254.85.2:37978 peer 169.254.85.4:10001 created. conn [2] local 169.254.85.2:37980 peer 169.254.85.4:10001 created. conn [3] local 169.254.85.2:37982 peer 169.254.85.4:10001 created. conn [4] local 169.254.85.2:37984 peer 169.254.85.4:10001 created. conn [5] local 169.254.85.2:37986 peer 169.254.85.4:10001 created. conn [6] local 169.254.85.2:37988 peer 169.254.85.4:10001 created. conn [7] local 169.254.85.2:37990 peer 169.254.85.4:10001 created. 8 connection(s) created in total tx thread of conn 0 started tx thread of conn 1 started tx thread of conn 2 started tx thread of conn 3 started tx thread of conn 4 started tx thread of conn 5 started tx thread of conn 6 started tx thread of conn 7 started [time lasts]: 1 tx_succ_bytes 0x87790000 (2,272,854,016) [time lasts]: 2 tx_succ_bytes 0x62f50000 (1,660,223,488) [time lasts]: 3 tx_succ_bytes 0x70030000 (1,879,244,800) [time lasts]: 4 tx_succ_bytes 0x5b990000 (1,536,753,664) [time lasts]: 5 tx_succ_bytes 0x64210000 (1,679,884,288) [time lasts]: 6 tx_succ_bytes 0x6a0a0000 (1,779,040,256) [time lasts]: 7 tx_succ_bytes 0x75720000 (1,970,405,376) [time lasts]: 8 tx_succ_bytes 0x7d980000 (2,107,113,472) [time lasts]: 9 tx_succ_bytes 0x8d070000 (2,366,046,208) [time lasts]: 10 tx_succ_bytes 0x74030000 (1,946,353,664) [time lasts]: 11 tx_succ_bytes 0x71f20000 (1,911,685,120) [time lasts]: 12 tx_succ_bytes 0x7f900000 (2,140,143,616) [time lasts]: 13 tx_succ_bytes 0x6f690000 (1,869,152,256) [time lasts]: 14 tx_succ_bytes 0x8edd0000 (2,396,848,128) [time lasts]: 15 tx_succ_bytes 0x80c30000 (2,160,263,168) [time lasts]: 16 tx_succ_bytes 0x79a40000 (2,040,791,040) [time lasts]: 17 tx_succ_bytes 0x82120000 (2,182,217,728) [time lasts]: 18 tx_succ_bytes 0x69e30000 (1,776,484,352) [time lasts]: 19 tx_succ_bytes 0x64c80000 (1,690,828,800) [time lasts]: 20 tx_succ_bytes 0x6e430000 (1,849,884,672) [time lasts]: 21 tx_succ_bytes 0x6b110000 (1,796,276,224) [time lasts]: 22 tx_succ_bytes 0x753c0000 (1,966,866,432) [time lasts]: 23 tx_succ_bytes 0x80c30000 (2,160,263,168) [time lasts]: 24 tx_succ_bytes 0x7c700000 (2,087,714,816) [time lasts]: 25 tx_succ_bytes 0x72ec0000 (1,928,069,120) [time lasts]: 26 tx_succ_bytes 0x84d20000 (2,228,355,072) [time lasts]: 27 tx_succ_bytes 0x70f60000 (1,895,170,048) [time lasts]: 28 tx_succ_bytes 0x613c0000 (1,631,322,112) [time lasts]: 29 tx_succ_bytes 0x7cf00000 (2,096,103,424) [time lasts]: 30 tx_succ_bytes 0x61270000 (1,629,945,856) [time lasts]: 31 tx_succ_bytes 0x608d0000 (1,619,853,312) [time lasts]: 32 tx_succ_bytes 0x7cdd0000 (2,094,858,240) [time lasts]: 33 tx_succ_bytes 0x6e430000 (1,849,884,672) [time lasts]: 34 tx_succ_bytes 0x5d820000 (1,568,800,768) [time lasts]: 35 tx_succ_bytes 0x78f40000 (2,029,256,704) [time lasts]: 36 tx_succ_bytes 0x66c20000 (1,723,990,016) [time lasts]: 37 tx_succ_bytes 0x7ba60000 (2,074,476,544) [time lasts]: 38 tx_succ_bytes 0x58600000 (1,482,686,464) [time lasts]: 39 tx_succ_bytes 0x64710000 (1,685,127,168) [time lasts]: 40 tx_succ_bytes 0x6e410000 (1,849,753,600) [time lasts]: 41 tx_succ_bytes 0x65d00000 (1,708,130,304) [time lasts]: 42 tx_succ_bytes 0x83490000 (2,202,599,424) [time lasts]: 43 tx_succ_bytes 0x95950000 (2,509,570,048) [time lasts]: 44 tx_succ_bytes 0x81980000 (2,174,222,336) [time lasts]: 45 tx_succ_bytes 0x6acb0000 (1,791,688,704) [time lasts]: 46 tx_succ_bytes 0x73320000 (1,932,656,640) [time lasts]: 47 tx_succ_bytes 0x70000000 (1,879,048,192) [time lasts]: 48 tx_succ_bytes 0x73b40000 (1,941,176,320) [time lasts]: 49 tx_succ_bytes 0x5bdd0000 (1,541,210,112) [time lasts]: 50 tx_succ_bytes 0x7e980000 (2,123,890,688) [time lasts]: 51 tx_succ_bytes 0x7cf00000 (2,096,103,424) [time lasts]: 52 tx_succ_bytes 0x802c0000 (2,150,367,232) [time lasts]: 53 tx_succ_bytes 0x61540000 (1,632,894,976) [time lasts]: 54 tx_succ_bytes 0x5f690000 (1,600,716,800) [time lasts]: 55 tx_succ_bytes 0x6cde0000 (1,826,488,320) [time lasts]: 56 tx_succ_bytes 0x6af90000 (1,794,703,360) [time lasts]: 57 tx_succ_bytes 0x9ba10000 (2,611,019,776) [time lasts]: 58 tx_succ_bytes 0x82ea0000 (2,196,373,504) [time lasts]: 59 tx_succ_bytes 0x80a30000 (2,158,166,016) [time lasts]: 60 tx_succ_bytes 0x5d4c0000 (1,565,261,824) [time lasts]: 61 tx_succ_bytes 0x789e0000 (2,023,620,608) [time lasts]: 62 tx_succ_bytes 0x65cf0000 (1,708,064,768) [time lasts]: 63 tx_succ_bytes 0x83050000 (2,198,142,976) /* * logs of TX node 3. */ orem-dirt:/home/admin/tyler # echo cubic >/proc/sys/net/ipv4/tcp_congestion_control orem-dirt:/home/admin/tyler # cat /proc/sys/net/ipv4/tcp_congestion_control cubic orem-dirt:/home/admin/tyler # date; ./tcp_perf.exe --server-ip 169.254.85.4 --server-port 10001 --msg-len 65536 --conn-num 8 -c -t --data-random --show-bandwidth-only Fri Mar 25 08:23:15 EDT 2022 conn [0] local 169.254.85.3:59676 peer 169.254.85.4:10001 created. conn [1] local 169.254.85.3:59678 peer 169.254.85.4:10001 created. conn [2] local 169.254.85.3:59680 peer 169.254.85.4:10001 created. conn [3] local 169.254.85.3:59682 peer 169.254.85.4:10001 created. conn [4] local 169.254.85.3:59684 peer 169.254.85.4:10001 created. conn [5] local 169.254.85.3:59686 peer 169.254.85.4:10001 created. conn [6] local 169.254.85.3:59688 peer 169.254.85.4:10001 created. conn [7] local 169.254.85.3:59690 peer 169.254.85.4:10001 created. 8 connection(s) created in total tx thread of conn 0 started tx thread of conn 1 started tx thread of conn 2 started tx thread of conn 3 started tx thread of conn 4 started tx thread of conn 5 started tx thread of conn 6 started tx thread of conn 7 started [time lasts]: 1 tx_succ_bytes 0x15b6d0000 (5,828,837,376) [time lasts]: 2 tx_succ_bytes 0xb1590000 (2,975,399,936) [time lasts]: 3 tx_succ_bytes 0x8e2e0000 (2,385,379,328) [time lasts]: 4 tx_succ_bytes 0x79cd0000 (2,043,478,016) [time lasts]: 5 tx_succ_bytes 0x7e280000 (2,116,550,656) [time lasts]: 6 tx_succ_bytes 0x80dc0000 (2,161,901,568) [time lasts]: 7 tx_succ_bytes 0x7fa90000 (2,141,782,016) [time lasts]: 8 tx_succ_bytes 0x8d530000 (2,371,026,944) [time lasts]: 9 tx_succ_bytes 0x6bf10000 (1,810,956,288) [time lasts]: 10 tx_succ_bytes 0x7d2c0000 (2,100,035,584) [time lasts]: 11 tx_succ_bytes 0x6cca0000 (1,825,177,600) [time lasts]: 12 tx_succ_bytes 0x755f0000 (1,969,160,192) [time lasts]: 13 tx_succ_bytes 0x78240000 (2,015,625,216) [time lasts]: 14 tx_succ_bytes 0x7a880000 (2,055,733,248) [time lasts]: 15 tx_succ_bytes 0x5e980000 (1,587,019,776) [time lasts]: 16 tx_succ_bytes 0x59fe0000 (1,509,818,368) [time lasts]: 17 tx_succ_bytes 0x6e370000 (1,849,098,240) [time lasts]: 18 tx_succ_bytes 0x6db80000 (1,840,775,168) [time lasts]: 19 tx_succ_bytes 0x72f60000 (1,928,724,480) [time lasts]: 20 tx_succ_bytes 0x6b7c0000 (1,803,288,576) [time lasts]: 21 tx_succ_bytes 0x64040000 (1,677,983,744) [time lasts]: 22 tx_succ_bytes 0x63c80000 (1,674,051,584) [time lasts]: 23 tx_succ_bytes 0x7abd0000 (2,059,206,656) [time lasts]: 24 tx_succ_bytes 0x78b60000 (2,025,193,472) [time lasts]: 25 tx_succ_bytes 0x73c00000 (1,941,962,752) [time lasts]: 26 tx_succ_bytes 0x779e0000 (2,006,843,392) [time lasts]: 27 tx_succ_bytes 0x6ba70000 (1,806,106,624) [time lasts]: 28 tx_succ_bytes 0x73140000 (1,930,690,560) [time lasts]: 29 tx_succ_bytes 0x79e20000 (2,044,854,272) [time lasts]: 30 tx_succ_bytes 0x64210000 (1,679,884,288) [time lasts]: 31 tx_succ_bytes 0x80b70000 (2,159,476,736) [time lasts]: 32 tx_succ_bytes 0x79450000 (2,034,565,120) [time lasts]: 33 tx_succ_bytes 0x72e30000 (1,927,479,296) [time lasts]: 34 tx_succ_bytes 0x664f0000 (1,716,453,376) [time lasts]: 35 tx_succ_bytes 0x81120000 (2,165,440,512) [time lasts]: 36 tx_succ_bytes 0x6abd0000 (1,790,771,200) [time lasts]: 37 tx_succ_bytes 0x88530000 (2,287,140,864) [time lasts]: 38 tx_succ_bytes 0x5a210000 (1,512,112,128) [time lasts]: 39 tx_succ_bytes 0x7e4a0000 (2,118,778,880) [time lasts]: 40 tx_succ_bytes 0x86fe0000 (2,264,793,088) [time lasts]: 41 tx_succ_bytes 0x73c10000 (1,942,028,288) [time lasts]: 42 tx_succ_bytes 0x83620000 (2,204,237,824) [time lasts]: 43 tx_succ_bytes 0x7f900000 (2,140,143,616) [time lasts]: 44 tx_succ_bytes 0x699c0000 (1,771,831,296) [time lasts]: 45 tx_succ_bytes 0x5e0d0000 (1,577,910,272) [time lasts]: 46 tx_succ_bytes 0x66890000 (1,720,254,464) [time lasts]: 47 tx_succ_bytes 0x75a10000 (1,973,485,568) [time lasts]: 48 tx_succ_bytes 0x6b3f0000 (1,799,290,880) [time lasts]: 49 tx_succ_bytes 0x764d0000 (1,984,757,760) [time lasts]: 50 tx_succ_bytes 0x83320000 (2,201,092,096) [time lasts]: 51 tx_succ_bytes 0x75ca0000 (1,976,172,544) [time lasts]: 52 tx_succ_bytes 0x759e0000 (1,973,288,960) [time lasts]: 53 tx_succ_bytes 0x6b160000 (1,796,603,904) [time lasts]: 54 tx_succ_bytes 0x74430000 (1,950,547,968) [time lasts]: 55 tx_succ_bytes 0x7ed90000 (2,128,150,528) [time lasts]: 56 tx_succ_bytes 0x74ff0000 (1,962,868,736) [time lasts]: 57 tx_succ_bytes 0x60dd0000 (1,625,096,192) [time lasts]: 58 tx_succ_bytes 0x5eca0000 (1,590,296,576) [time lasts]: 59 tx_succ_bytes 0x73890000 (1,938,358,272) [time lasts]: 60 tx_succ_bytes 0x5e8c0000 (1,586,233,344) [time lasts]: 61 tx_succ_bytes 0x7d010000 (2,097,217,536) [time lasts]: 62 tx_succ_bytes 0x76cd0000 (1,993,146,368) [time lasts]: 63 tx_succ_bytes 0x81620000 (2,170,683,392) [time lasts]: 64 tx_succ_bytes 0x6b980000 (1,805,123,584) /* * counters on the switch. * we can see, the rate of packet dropping is so high (~5%). */ hound-dirt# show queuing statistics interface ethernet 1/1/4 Interface ethernet1/1/4 Queue Packets Bytes Dropped-Packets Dropped-Bytes 0 0 0 0 0 1 9 882 0 0 2 0 0 0 0 3 0 0 0 0 4 0 0 0 0 5 21377065 191400217674 1263665 11321730380 6 0 0 0 0 7 0 0 0 0 hound-dirt# fox-dirt# show queuing statistics interface ethernet 1/1/4 Interface ethernet1/1/4 Queue Packets Bytes Dropped-Packets Dropped-Bytes 0 10 654 0 0 1 6 588 0 0 2 0 0 0 0 3 0 0 0 0 4 0 0 0 0 5 21228491 189957815718 1192655 10677602488 6 0 0 0 0 7 0 0 0 0 fox-dirt#
Powered by blists - more mailing lists