netdev - Re: Constantly varying download rate with a complex xen networking setup, why?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 15 Jun 2009 19:36:36 +0300 (EEST)
From:	"Ilpo Järvinen" <ilpo.jarvinen@...sinki.fi>
To:	Raphael Hertzog <raphael@...za.com>
cc:	Netdev <netdev@...r.kernel.org>
Subject: Re: Constantly varying download rate with a complex xen networking
 setup, why?

On Mon, 15 Jun 2009, Raphael Hertzog wrote:

> Le lundi 15 juin 2009, Ilpo Järvinen a écrit :
> > On Mon, 15 Jun 2009, Raphael Hertzog wrote:
> > 
> > > My problem is that when I download a big file from the internet by HTTP in
> > > the dom0, the download rate is not stable. It goes up progressively and then
> > > stalls for a few seconds, and restart again going up progressively (and
> > > all this in loop until the download is complete).
> > 
> > My guess: TCP sawtooth pattern and the associated loss recovery...? What 
> > you describe here seems to match exactly to what is expected to happen 
> > because of it (I see this happening quite often actually).
> 
> Why would this not happen for the same transfer done through a proxy (that
> runs on the same machine but different domU)?

Maybe the proxy interferes there somehow... I don't know enough about the
details to say but I suppose the proxy at least breaks your tcp connection 
to two parts.

It could well be something else. I was just saying that this 
increasing+stall+repeat pattern itself is quite typical also during normal 
operation (with short rtts one can't see that though due to humanly 
limitations in perception :-) and because of the granularity the 
reporting).

> > What is the size of the bottleneck buffer and base RTTs of the relevant
> > flows (HTTP, ssh)?
> 
> How can I collect those information?

Perhaps it's enough to just know if sshs that stall go over the adsl link 
too or are they local? If they're local it's sort of pointing to something 
else (of which I probably have no clue).

> > Please verify "blocking" using tcpdump or so. You may find out e.g. that 
> 
> I'll try to do so and follow-up.
> 
> > network is not stalled but that out-of-order arrivals prevent 
> > applications from making progress.
> > 
> > If this matches with your problem, deploying AQM at the bottleneck would 
> > help.
> 
> The bottleneck in my test setup is the ADSL connection and I don't control
> the ADSL router.

Right, those non-controllable things need a bit more sophisticated 
control. Lartc has some pre-made script which basically puts an articifial 
cap on the receiver side slightly below the downlinkg rate to create 
bottleneck there which in turn allows controlling the queue.

-- 
 i.