Message-ID: <CAKgT0UfDQhD=J7RomDZmzjRsMSm6wgtaS-sc-grE00a=8kWN8Q@mail.gmail.com>
Date:   Tue, 12 Jan 2021 18:11:15 -0800
From:   Alexander Duyck <alexander.duyck@...il.com>
To:     Xin Long <lucien.xin@...il.com>
Cc:     network dev <netdev@...r.kernel.org>,
        "linux-sctp @ vger . kernel . org" <linux-sctp@...r.kernel.org>,
        Marcelo Ricardo Leitner <marcelo.leitner@...il.com>,
        Neil Horman <nhorman@...driver.com>,
        David Miller <davem@...emloft.net>,
        Jakub Kicinski <kuba@...nel.org>,
        Lorenzo Bianconi <lorenzo@...nel.org>
Subject: Re: [PATCHv2 net-next] ip_gre: remove CRC flag from dev features in gre_gso_segment

On Mon, Jan 11, 2021 at 9:14 PM Xin Long <lucien.xin@...il.com> wrote:
>
> On Tue, Jan 12, 2021 at 12:48 AM Alexander Duyck
> <alexander.duyck@...il.com> wrote:
> >
> > On Mon, Jan 11, 2021 at 5:22 AM Xin Long <lucien.xin@...il.com> wrote:
> > >
> > > This patch is to let it always do CRC checksum in sctp_gso_segment()
> > > by removing CRC flag from the dev features in gre_gso_segment() for
> > > SCTP over GRE, just as it does in Commit 527beb8ef9c0 ("udp: support
> > > sctp over udp in skb_udp_tunnel_segment") for SCTP over UDP.
> > >
> > > It could set csum/csum_start in GSO CB properly in sctp_gso_segment()
> > > after that commit, so it would do checksum with gso_make_checksum()
> > > in gre_gso_segment(), and Commit 622e32b7d4a6 ("net: gre: recompute
> > > gre csum for sctp over gre tunnels") can be reverted now.
> > >
> > > Note that the current HWs like igb NIC can only handle the SCTP CRC
> > > when it's in the outer packet, not in the inner packet like in this
> > > case, so here it removes CRC flag from the dev features even when
> > > need_csum is false.
> >
> > So the limitation in igb is not the hardware but the driver
> > configuration. When I had coded things up I put in a limitation on the
> > igb_tx_csum code that it would have to validate that the protocol we
> > are requesting an SCTP CRC offload since it is a different calculation
> > than a 1's complement checksum. Since igb doesn't support tunnels we
> > limited that check to the outer headers.
> Ah.. I see, thanks.
> >
> > We could probably enable this for tunnels as long as the tunnel isn't
> > requesting an outer checksum offload from the driver.
> I think in igb_tx_csum(), by checking skb->csum_not_inet would be enough
> to validate that is a SCTP request:
> -               if (((first->protocol == htons(ETH_P_IP)) &&
> -                    (ip_hdr(skb)->protocol == IPPROTO_SCTP)) ||
> -                   ((first->protocol == htons(ETH_P_IPV6)) &&
> -                    igb_ipv6_csum_is_sctp(skb))) {
> +               if (skb->csum_not_inet) {
>                         type_tucmd = E1000_ADVTXD_TUCMD_L4T_SCTP;
>                         break;
>                 }
>

So if I may ask. Why go with something like csum_not_inet instead of
specifying something like crc32_csum? I'm just wondering if there are
any other non-1's complement checksums that we are dealing with?

One thing we might want to do to make eventual backporting for this
easier would be to add an accessor inline function. Maybe something
like a skb_csum_is_crc32() so that for older kernels the function
could just be defined to return false since the csum_not_inet may not
exist.

> Otherwise, we will need to parse the packet a little bit, as it does in
> hns3_get_l4_protocol().

Agreed. If the csum_not_inet means it is a crc32 checksum then we
could just look at the offsets and as long as they are correct for
sctp we could just go forward with what we have.

> >
> > > v1->v2:
> > >   - improve the changelog.
> > >   - fix "rev xmas tree" in varibles declaration.
> > >
> > > Signed-off-by: Xin Long <lucien.xin@...il.com>
> > > ---
> > >  net/ipv4/gre_offload.c | 15 ++++-----------
> > >  1 file changed, 4 insertions(+), 11 deletions(-)
> > >
> > > diff --git a/net/ipv4/gre_offload.c b/net/ipv4/gre_offload.c
> > > index e0a2465..a681306 100644
> > > --- a/net/ipv4/gre_offload.c
> > > +++ b/net/ipv4/gre_offload.c
> > > @@ -15,10 +15,10 @@ static struct sk_buff *gre_gso_segment(struct sk_buff *skb,
> > >                                        netdev_features_t features)
> > >  {
> > >         int tnl_hlen = skb_inner_mac_header(skb) - skb_transport_header(skb);
> > > -       bool need_csum, need_recompute_csum, gso_partial;
> > >         struct sk_buff *segs = ERR_PTR(-EINVAL);
> > >         u16 mac_offset = skb->mac_header;
> > >         __be16 protocol = skb->protocol;
> > > +       bool need_csum, gso_partial;
> > >         u16 mac_len = skb->mac_len;
> > >         int gre_offset, outer_hlen;
> > >
> > > @@ -41,10 +41,11 @@ static struct sk_buff *gre_gso_segment(struct sk_buff *skb,
> > >         skb->protocol = skb->inner_protocol;
> > >
> > >         need_csum = !!(skb_shinfo(skb)->gso_type & SKB_GSO_GRE_CSUM);
> > > -       need_recompute_csum = skb->csum_not_inet;
> > >         skb->encap_hdr_csum = need_csum;
> > >
> > >         features &= skb->dev->hw_enc_features;
> > > +       /* CRC checksum can't be handled by HW when SCTP is the inner proto. */
> > > +       features &= ~NETIF_F_SCTP_CRC;
> > >
> > >         /* segment inner packet. */
> > >         segs = skb_mac_gso_segment(skb, features);
> >
> > Do we have NICs that are advertising NETIF_S_SCTP_CRC as part of their
> > hw_enc_features and then not supporting it? Based on your comment
> Yes, igb/igbvf/igc/ixgbe/ixgbevf, they have a similar code of SCTP
> proto validation.

Yeah, it is old code. It was added in 4.6 before tunnels supported
SCTP_CRC I am guessing. It looks like csum_not_inet wasn't added until
4.13. So it would probably be best to fix the drivers since the driver
code is outdated.

> > above it seems like you are masking this out because hardware is
> > advertising features it doesn't actually support. I'm just wondering
> > if that is the case or if this is something where this should be
> > cleared if need_csum is set since we only support one level of
> > checksum offload.
> Since only these drivers only do SCTP proto validation, and "only
> one level checksum offload" issue only exists when inner packet
> is SCTP packet, clearing NETIF_F_SCTP_CRC should be enough.
>
> But seems to fix the drivers will be better, as hw_enc_features should
> tell the correct features for inner proto. wdyt?

Yes, it would be better to fix the drivers. However the one limitation
is that this will only work when we don't have an outer checksum in
place. If we have an outer checksum then we have to compute the crc32
checksum and then offload the outer checksum if we can.

> (Just note udp tunneling SCTP doesn't have this issue, as the outer
>  udp checksum is always required by RFC)

Thanks. I wasn't aware of that.

> >
> > > @@ -99,15 +100,7 @@ static struct sk_buff *gre_gso_segment(struct sk_buff *skb,
> > >                 }
> > >
> > >                 *(pcsum + 1) = 0;
> > > -               if (need_recompute_csum && !skb_is_gso(skb)) {
> > > -                       __wsum csum;
> > > -
> > > -                       csum = skb_checksum(skb, gre_offset,
> > > -                                           skb->len - gre_offset, 0);
> > > -                       *pcsum = csum_fold(csum);
> > > -               } else {
> > > -                       *pcsum = gso_make_checksum(skb, 0);
> > > -               }
> > > +               *pcsum = gso_make_checksum(skb, 0);
> > >         } while ((skb = skb->next));
> > >  out:
> > >         return segs;
> > > --
> > > 2.1.0
> > >

Powered by blists - more mailing lists