Message-ID: <75f9f963c1d07f5739ce76fcfc832e7e83586bda.camel@sipsolutions.net>
Date:   Mon, 08 Apr 2019 21:49:40 +0200
From:   Johannes Berg <johannes@...solutions.net>
To:     Subash Abhinov Kasiviswanathan <subashab@...eaurora.org>
Cc:     Dan Williams <dcbw@...hat.com>,
        Bjørn Mork <bjorn@...k.no>,
        netdev@...r.kernel.org, Sean Tranchetti <stranche@...eaurora.org>,
        Daniele Palmas <dnlplm@...il.com>,
        Aleksander Morgado <aleksander@...ksander.es>
Subject: Re: cellular modem driver APIs

On Thu, 2019-04-04 at 22:45 -0600, Subash Abhinov Kasiviswanathan wrote:
> On 2019-04-04 14:38, Johannes Berg wrote:
> > Hi,
> > 
> > > The normal mode of operation of rmnet is using the rmnet netdevices
> > > in an embedded device.
> > 
> > Sure. Can you say what driver this would typically live on top of? I'm
> > actually a bit surprised to find out this isn't really a driver :-)
> > 
> 
> This needs a physical net device such as IP accelerator
> https://lkml.org/lkml/2018/11/7/233 or Modem host interface
> https://lkml.org/lkml/2018/4/26/1159

OK. But it means that you have a very specific encapsulation mode on top
of the "netdev". I'm still not convinced we should actually make that a
netdev, but I'll elaborate elsewhere.

> I recall Daniele also managed to get rmnet working with qmi_wwan
> (with an additional patch in which I had made qmi_wwan a passthrough)

I guess that uses the same encapsulation then, yes, I see it now:
qmi_wwan's struct qmimux_hdr and rmnet's struct rmnet_map_header are
very similar.

Btw, I see that struct rmnet_map_header uses a bitfield - this seems to
go down to the device so probably will not work right on both big and
little endian.

> > Yeah, I get it, it's just done in a strange way. You'd think adding a
> > tcpdump or some small application that just resends the packets 
> > directly
> > received from the underlying "real_dev" using a ptype_all socket would
> > be sufficient? Though perhaps not quite the same performance, but then
> > you could easily not use an application but a dev_add_pack() thing? Or
> > probably even tc's mirred?
> > 
> > And to extend that thought, tc's ife action would let you encapsulate
> > the things you have in ethernet headers... I think.
> 
> We need raw IP frames from a embedded device transmitted to a PC
> and vice versa.

Sure. But you just need to encap them in some kind of ethernet frame to
transport them on the wire, but don't really need to care much about
how.

> Yes, the underlying netdev itself cannot do much on its own as network
> stack wont be able to decipher the muxed frames.

Right.

> The operation of rmnet was to be agnostic of the underlying driver.
> The netdev model was chosen for it since it was easy to have a 
> rx_handler attach to the netdevice exposed by any of those drivers.

I really do think it's the wrong model though:

   1. The network stack cannot do anything with the muxed data, and so
      there's no value from that perspective in exposing it that way.
   2. The rx_handler attach thing is a bit of a red herring - if you have
      some other abstraction that the driver registers with, you can just
      say "send packets here" and then demux things properly, without
      having a netdev. Actually, I'd almost argue that rmnet should've just
      been a sort of encap/decap library that the drivers like qmi_wwan,
      rmnet_ipa and mhi use to talk to the device.
   3. Having this underlying netdev is actually very limiting. We found
      this with wifi a long time ago, and I suspect with 5G coming up
      you'll be in a similar situation. You'll want to do things like
      multi-queue, different hardware queues for different channels, etc.
      and muxing it all over a single netdev (and a single queue there)
      becomes an easily avoidable point of contention.
   4. (I thought about something else but it escapes me now)

> > Now, OTOH, this loses a bunch of benefits. We may want to be able to 
> > use
> > ethtool to flash a modem, start tcpdump on the underlying netdev
> > directly to see everything, etc.?
> > 
> 
> Yes, we use that underlying netdev to view the muxed raw IP frames in 
> tcpdump.

That's the easiest of all - just make the framework able to add a 
'sniffer' netdev that can see all the TX/RX for the other channels.

Which I already said, and you asked:

> Can you tell how this sniffer netdev works? 

Well, it's just an abstraction - a netdev that reports as received all
frames that the stack or underlying driver sees.

> If rmnet needed to create a
> separate netdev for monitoring the muxed raw IP frames, wouldnt that
> just serve the same purpose as that of underlying netdev.

No, it's different in that it's only there for debugging, only reports
all the frames the stack/driver saw, and doesn't interfere with your
actual operation like the underlying netdev does like I described above.

johannes

Powered by blists - more mailing lists