netdev - Re: Intro into qdisc writing?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <da23ab42-6b99-4981-97f0-3cd7a76c96b8@redhat.com>
Date:   Tue, 10 Aug 2021 12:22:15 +0200
From:   Jesper Dangaard Brouer <jbrouer@...hat.com>
To:     Eric Dumazet <eric.dumazet@...il.com>,
        Thorsten Glaser <t.glaser@...ent.de>, netdev@...r.kernel.org
Cc:     brouer@...hat.com
Subject: Re: Intro into qdisc writing?


On 10/08/2021 10.34, Eric Dumazet wrote:
> On 8/10/21 5:17 AM, Thorsten Glaser wrote:
>> Hi,
>>
>> I hope this is the right place to ask this kind of questions,
>> and not just send patches ☺
>>
>> I’m currently working with a… network simulator of sorts, which
>> has so far mostly used htb, netem, dualpi2 and fq_codel to do the
>> various tricks needed for whatever they do, but now I have rather
>> specific change requests (one of which I already implemented).
>>
>> The next things on my list basically involve delaying all traffic
>> or a subset of traffic for a certain amount of time (in the one‑ to
>> two-digit millisecond ballpark, so rather long, in CPU time). I’ve
>> seen the netem source use qdisc_watchdog_schedule_ns for this, but,
>> unlike the functions I used in my earlier module changes, I cannot
>> find any documentation for this.
>>
>> Similarily, is there an intro of sorts for qdisc writing, the things
>> to know, concepts, locking, whatever is needed?
>>
>> My background is multi-decade low-level programmer, but so far only
>> userland, libc variants and bootloaders, not kernel, and what bit of
>> kernel I touched so far was in BSD land so any pointers welcome.
>>
>> If it helps: while this is for a customer project, so far everything
>> coming out of it is published under OSS licences; mostly at
>> https://github.com/tarent/sch_jens/tree/master/sch_jens as regards
>> the kernel module (and ../jens/ for the relayfs client example) but
>> https://github.com/tarent/ECN-Bits has a related userspace project.
>>
>> Thanks in advance,
>> //mirabilos
>>
> Instead of writing a new qdisc, you could simply use FQ packet scheduler,
> and a eBPF program adjusting skb->tstamp depending on your needs.
>
> https://legacy.netdevconf.info/0x14/session.html?talk-replacing-HTB-with-EDT-and-BPF


Good link and reference.

If you want to see some code doing this via BPF see: 
https://github.com/xdp-project/bpf-examples/blob/master/traffic-pacing-edt/

I've unfortunately not had time to document the 'traffic-pacing-edt' 
code, as this was done under time pressure, for solving a production 
problem at an ISP. They needed packet pacing or transmission smoothing 
due to switches with too small buffers to handle bursts, but as close to 
1Gbit/s as possible as they sold 1G to their customers.

The comments in the code and scripts should hopefully be enough for you 
to understand the concept. Eric's slides describe the overall concept 
and background.


The main code you want to look at is in 'edt_pacer_vlan.c' [1], but 
notice that is assumes it have lockless access to the datastructure. 
This assumption is only true because the XDP-prog in 'xdp_cpumap_qinq.c' 
[2] moves packets associated with the datastructure to the right CPU 
(and invokes/starts the normal networks stack on that CPU).


[1] 
https://github.com/xdp-project/bpf-examples/blob/master/traffic-pacing-edt/edt_pacer_vlan.c

[2] 
https://github.com/xdp-project/bpf-examples/blob/master/traffic-pacing-edt/xdp_cpumap_qinq.c


--Jesper