linux-kernel - Re: [PATCH V2 1/1] net: cdc_ncm: Reduce memory use when kernel memory low

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 30 Jun 2017 18:53:56 +0100
From:   "Baxter, Jim" <jim_baxter@...tor.com>
To:     Bjørn Mork <bjorn@...k.no>
CC:     <linux-usb@...r.kernel.org>, <netdev@...r.kernel.org>,
        <linux-kernel@...r.kernel.org>, Oliver Neukum <oliver@...kum.org>,
        David Laight <David.Laight@...LAB.COM>
Subject: Re: [PATCH V2 1/1] net: cdc_ncm: Reduce memory use when kernel memory
 low


> Jim Baxter <jim_baxter@...tor.com> writes:
> 
>> The CDC-NCM driver can require large amounts of memory to create
>> skb's and this can be a problem when the memory becomes fragmented.
>>
>> This especially affects embedded systems that have constrained
>> resources but wish to maximise the throughput of CDC-NCM with 16KiB
>> NTB's.
>>
>> The issue is after running for a while the kernel memory can become
>> fragmented and it needs compacting.
>> If the NTB allocation is needed before the memory has been compacted
>> the atomic allocation can fail which can cause increased latency,
>> large re-transmissions or disconnections depending upon the data
>> being transmitted at the time.
>> This situation occurs for less than a second until the kernel has
>> compacted the memory but the failed devices can take a lot longer to
>> recover from the failed TX packets.
>>
>> To ease this temporary situation I modified the CDC-NCM TX path to
>> temporarily switch into a reduced memory mode which allocates an NTB
>> that will fit into a USB_CDC_NCM_NTB_MIN_OUT_SIZE (default 2048 Bytes)
>> sized memory block and only transmit NTB's with a single network frame
>> until the memory situation is resolved.
>> Each time this issue occurs we wait for an increasing number of
>> reduced size allocations before requesting a full size one to not
>> put additional pressure on a low memory system.
>>
>> Once the memory is compacted the CDC-NCM data can resume transmitting
>> at the normal tx_max rate once again.
>>
>> Signed-off-by: Jim Baxter <jim_baxter@...tor.com>
> 
> This looks good to me.
> 
> I would still be happier if we didn't need something like this, but I
> understand that we do.  And this patch looks as clean as it can get.  I
> haven't tested the patch under any sort of memory pressure, but I did a
> basic runtime test on a single MBIM device.  As expected, I did not
> notice any changes with this patch applied.
> 
> But regarding noticable effects: The patch adds no printks, counters or
> sysfs attributes which could tell the user that the initial buffer
> allocation has failed.  Maybe add some sort of debug helper(s) in a
> followup patch? How did you verify the patch operation while testing it?
> 
> Anyway, that's no show stopper of course.  So FWIW:
> 
> Reviewed-by: Bjørn Mork <bjorn@...k.no>
> 

Hello Bjørn,

I tested this with printk's to show when the low memory code was triggered
and the value of ctx->tx_low_mem_val and ctx->tx_low_mem_max_cnt.
I created a workqueue that slowly used up the atomic memory until the
code is triggered.

I could add debug prints, though I have noticed that cdc_ncm_fill_tx_frame()
does not currently have any debug prints do you think this is because it can be
called in an atomic context and I think debug messages if enabled could cause
too great a delay?

Regards,
Jim