linux-kernel - Re: multi-second application stall in open()

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20120622141211.GB18409@redhat.com>
Date:	Fri, 22 Jun 2012 10:12:11 -0400
From:	Vivek Goyal <vgoyal@...hat.com>
To:	Josh Hunt <joshhunt00@...il.com>
Cc:	Jens Axboe <axboe@...nel.dk>, linux-kernel@...r.kernel.org,
	tj@...nel.org
Subject: Re: multi-second application stall in open()

On Thu, Jun 21, 2012 at 04:11:18PM -0500, Josh Hunt wrote:

[..]
> > say put some logs in select_queue() and see where did it bail out. That
> 
> Well I did add some instrumentation in select_queue, the "keep_queue
> st->count:%d, dispatch:%u" line I mentioned above, but I will add more
> and retest.

Actually before the stall we expired the current queue. That means there
is no active queue in cfq now. So keep_queue trace will help only if there
is an active queue and we decide to keep that queue. 

  8,0    0        0  4466.139959742     0  m   N cfq20720 del_from_rr
  8,0    0        0  4466.139963653     0  m   N cfq schedule dispatch
  8,0    1  1499521  4466.791207736  7570  A   R 7258191 + 8 <- (8,1)
7258128

Thing to figure out here is that why cfq is not picking a new queue
despite the fact there are pending requests and there is no active
queue (hence cfq is not obiviously not seeing that queue).

> 
> I'm attaching a similar run with no stalls when I set slice_idle to 0.

At this point I am not sure why slice_idle=0 is not seeing this issue. It
does not see to be directly related to idling. Because when stall happens
CFQ is not idling on anything. It has expired active queue and for some
reason it thinks that I have no more requests to dispatch, so I have
nothing to do until a new request comes in.

Thanks
Vivek
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/