linux-kernel - Re: PROBLEM: Long Workqueue delays.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 17 Aug 2020 13:57:44 +0200
From:   Greg KH <gregkh@...uxfoundation.org>
To:     Jim Baxter <jim_baxter@...tor.com>
Cc:     linux-kernel@...r.kernel.org, linux-mm@...ck.org,
        linux-usb@...r.kernel.org,
        "Resch Carsten (CM/ESO6)" <Carsten.Resch@...bosch.com>,
        "Rosca, Eugeniu (ADITG/ESB)" <erosca@...adit-jv.com>
Subject: Re: PROBLEM: Long Workqueue delays.

On Mon, Aug 17, 2020 at 12:40:03PM +0100, Jim Baxter wrote:
> We have issues with the workqueue of the kernel overloading the CPU 0 
> when we we disconnect a USB stick.
> 
> This results in other items on the shared workqueue being delayed by
> around 6.5 seconds with a default kernel configuration and 2.3 seconds
> on a config tailored for our RCar embedded platform.
> 
> I am aware there will be delays on the shared workqueue, are the delays
> we are seeing considered normal?
> 
> 
> 
> We first noticed this issue on custom hardware and we have recreated it
> on an RCar Starter Kit using a test module [1] to replicate the
> behaviour, the test module outputs any delays of greater then 9ms.
> 
> To run the test we have a 4GB random file on a USB stick and perform
> the following test:
> 
> 
> - Load the Module:
> # taskset -c 0 modprobe latency-mon
> 
> - Copy large amount of data from the stick:
> # dd if=/run/media/sda1/sample.txt of=/dev/zero
> [ 1437.517603] DELAY: 10
> 8388607+1 records in
> 8388607+1 records out
> 

Is this data really flushed out to the device?

> - Disconnect the USB stick:
> [ 1551.796792] usb 2-1: USB disconnect, device number 2
> [ 1558.625517] DELAY: 6782
> 
> 
> The Delay output 6782 is in milliseconds.

What USB workqueue is taking so long?

The one trying to deal with the filesystem flushing out the data that it
can't do now that the device is removed?  :)

thanks,

greg k-h