netdev - Re: consistency for statistics with XDP mode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <cafc480f-eb7a-1fd8-c404-774f1efd561d@gmail.com>
Date:   Mon, 3 Dec 2018 08:56:28 -0700
From:   David Ahern <dsahern@...il.com>
To:     Jesper Dangaard Brouer <brouer@...hat.com>,
        Jakub Kicinski <jakub.kicinski@...ronome.com>
Cc:     "Michael S. Tsirkin" <mst@...hat.com>,
        Saeed Mahameed <saeedm@...lanox.com>,
        "toke@...e.dk" <toke@...e.dk>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "pstaszewski@...are.pl" <pstaszewski@...are.pl>,
        "davem@...emloft.net" <davem@...emloft.net>,
        "jasowang@...hat.com" <jasowang@...hat.com>
Subject: Re: consistency for statistics with XDP mode

On 12/1/18 4:14 AM, Jesper Dangaard Brouer wrote:
>>> XDP dropping a packet is completely different.
>>>
>>> stats are important. packets disappearing with no counters -- standard
>>> counters visible by standard tools -- is a user nightmare. If the
>>> agreement is for XDP drops to be in driver level (e.g., xdp_drop) that
>>> is fine since it is still retrievable by ethtool -S (existing APIs and
>>> existing tools).  
>>
>> I don't think that's completely fair.  Disappearing packets are a
>> nightmare, but if the user installed a program which silently drops
>> packets without incrementing any counter it's their own fault.  If
>> cls_bpf returns STOLEN or TRAP, I don't think that's gonna get counted
>> anywhere.
> 
> Yes, exactly. I think we are all agreeing.

The person debugging a problem is most likely NOT the person who wrote
the bpf program or loaded it where ever. I have saying this for a while
now and it is happening: ebpf programs will be coming from multiple
sources and transparently loaded (transparent to the person using an OS
or a service). That is already happening today and Ubuntu 18.04 is a
good example of that. systemd installs 'firewall' programs on services,
and those programs can drop packets. 'ip vrf exec' is loading programs
on cgroups. This will spread, and it will become a nightmare to debug
why a service is not working as expected.

What am I asking for here with XDP is to give the poor TAC/customer
support person I fighting chance at isolating where a problem is
occurring which means standard counters accessible from standard tools.

> 
> RX packets in the driver NAPI-poll need to be counted as we have always
> done in ifconfig stats.  These RX counters are incremented, regardless
> of TC/netfilter drop the packet later.  Like wise XDP is a user
> installed program, and a user XDP_DROP action/choice should not "hide"
> this packet from the drivers RX-counter (as that IMHO could be more
> confusing to the user).
> 
> 
>> I don't think DPDK drivers maintain "just in case" statistics, either..
> 
> I agree. Maintaining counters for everything will cause performance
> issues.  That said if we do choose to standardize XDP stats per action,
> as most driver do maintain some ethtool counter anyhow, then we have to
> make sure this doesn't hurt performance.  One simple trick is to bulk
> update these counters at the end of drivers napi_pool function (like
> most drivers do for the RX-counter stats).
> 

Use of DPDK means everything is done with custom tools. There is no
comparison to what I am pushing here. I do not want Linux + XDP to
require custom anything to debug basic functionality - such as isolating
a packet drop to a specific point.