Message-ID: <c4534ad7-672f-9459-bb22-46bf504231ff@bobbriscoe.net>
Date:   Sun, 17 Oct 2021 01:42:05 +0100
From:   Bob Briscoe <ietf@...briscoe.net>
To:     Neal Cardwell <ncardwell@...gle.com>,
        Eric Dumazet <edumazet@...gle.com>
Cc:     Eric Dumazet <eric.dumazet@...il.com>,
        netdev <netdev@...r.kernel.org>,
        Ingemar Johansson S <ingemar.s.johansson@...csson.com>,
        Tom Henderson <tomh@...h.org>,
        "David S . Miller" <davem@...emloft.net>,
        Jakub Kicinski <kuba@...nel.org>
Subject: Re: [PATCH net-next 2/2] fq_codel: implement L4S style
 ce_threshold_ect1 marking

Eric, (thanks Neal)

On 15/10/2021 16:49, Neal Cardwell wrote:
> On Fri, Oct 15, 2021 at 10:08 AM Eric Dumazet <edumazet@...gle.com> wrote:
>> On Fri, Oct 15, 2021 at 5:59 AM Bob Briscoe <ietf@...briscoe.net> wrote:
>>> Eric,
>>>
>>> Because the threshold is in time units, I suggest the condition for
>>> exceeding it needs to be AND'd with (*backlog > mtu), otherwise you can
>>> get 100% solid marking at low link rates.
>>>
>>> When ce_threshold is for DCs, low link rates are unlikely.
>>> However, given ce_threshold_ect1 is mainly for the Internet, during
>>> testing with 1ms threshold we encountered solid marking at low link
>>> rates, so we had to add a 1 packet floor:
>>> https://bobbriscoe.net/projects/latency/dctth_journal_draft20190726.pdf
>>>
>>> Sorry to chime in after your patch went to net-next.
>>>
>> What you describe about a minimal backlog was already there with
>> ce_threshold handling ?
> For my education, do you have a pointer to where the ce_threshold
> marking logic has a minimum backlog size requirement in packets or
> bytes? AFAICT the ce_threshold marking in include/net/codel_impl.h
> happens regardless of the current size of the backlog.

[BB] When I checked before my original posting, the only check for 
single packet backlog was within should_drop() here:
https://elixir.bootlin.com/linux/latest/source/include/net/codel_impl.h#L125
However, whether or not that causes should_drop() to return false, 
codel_dequeue() still always falls through to the ce_threshold marking 
after end:
https://elixir.bootlin.com/linux/latest/source/include/net/codel_impl.h#L249


>
>> Or is it something exclusive to L4S ?
> I don't think it's exclusive to L4S. I think Bob is raising a general
> issue about improving ECN marking based on ce_threshold. My
> interpretation of Bob's point is that there is sort of a quantization
> issue at very low link speeds, where the serialization delay for a
> packet is at or above the ce_threshold delay. In such cases it seems
> there can be behavior where the bottleneck marks every packet CE all
> the time, causing any ECN-based algorithm (even DCTCP) to suffer poor
> utilization.
>
> I suppose with a fixed-speed link the operator could adjust the
> ce_threshold based on the serialization delays implied by the link
> speed, but perhaps in general this is infeasible due to variable-speed
> (e.g., radio) links.
>
> I guess perhaps this could be reproduced/tested with DCTCP (using
> ECT(0)), a ce_threshold of 1ms (for ECT(0)), and an emulated
> bottleneck link speed with a serialization delay well above 1ms (so a
> link speed well below 12Mbps).

[BB] Yes.

>
>> This deserves a separate patch, if anything :)
> Agreed, in the Linux development model this would make sense as a
> separate patch, since it is conceptually separate and there do not
> need to be any dependencies between the two changes. :-)

[BB] Sure. We'll see to it.

The (loose/indirect) dependency I saw was just that ce_threshold_ect1 
opens up the possibility of using the ce_threshold on the public 
Internet not just in DCs. So low rate links become a certainty, rather 
than a mere theoretical possibility.



Bob

>
> neal

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

Powered by blists - more mailing lists