linux-kernel - Re: loop block-mq conversion scalability issues

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150424105908.47de489c@tom-T450>
Date:	Fri, 24 Apr 2015 10:59:08 +0800
From:	Ming Lei <ming.lei@...onical.com>
To:	"Justin M. Forbes" <jforbes@...hat.com>
Cc:	linux-kernel <linux-kernel@...r.kernel.org>, tom.leiming@...il.com
Subject: Re: loop block-mq conversion scalability issues

Hi Justin,

Thanks for the report.

On Thu, 23 Apr 2015 16:04:10 -0500
"Justin M. Forbes" <jforbes@...hat.com> wrote:

> The block-mq conversion for loop in 4.0 kernels is showing us an
> interesting scalability problem with live CDs (ro, squashfs).  It was
> noticed when testing the Fedora beta that the more CPUs a liveCD image
> was given, the slower it would boot. A 4 core qemu instance or bare
> metal instance took more than twice as long to boot compared to a single
> CPU instance.  After investigating, this came directly to the block-mq
> conversion, reverting these 4 patches will return performance. More
> details are available at
> https://bugzilla.redhat.com/show_bug.cgi?id=1210857
> I don't think that reverting the patches is the ideal solution so I am
> looking for other options.  Since you know this code a bit better than I
> do I thought I would run it by you while I am looking as well.

I can understand the issue because the default @max_active for
alloc_workqueue() is quite big(512), which may cause too much
context switchs, then loop I/O performance gets decreased.

Actually I have written the kernel dio/aio based patch for decreasing
both CPU and memory utilization without sacrificing I/O performance,
and I will try to improve and push the patch during this cycle and hope
it can be merged(kernel/aio.c change is dropped, and only fs change is
needed on fs/direct-io.c).

But the following change should help for your case, could you test it?

---
diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index c6b3726..b1cb41d 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -1831,7 +1831,7 @@ static int __init loop_init(void)
 	}

 	loop_wq = alloc_workqueue("kloopd",
-			WQ_MEM_RECLAIM | WQ_HIGHPRI | WQ_UNBOUND, 0);
+			WQ_MEM_RECLAIM | WQ_HIGHPRI | WQ_UNBOUND, 32);
 	if (!loop_wq) {
 		err = -ENOMEM;
 		goto misc_out;

Thanks,
Ming Lei
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/