linux-kernel - find_busiest_group using lots of CPU

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090930081811.GP23126@kernel.dk>
Date:	Wed, 30 Sep 2009 10:18:11 +0200
From:	Jens Axboe <jens.axboe@...cle.com>
To:	Linux Kernel <linux-kernel@...r.kernel.org>
Cc:	mingo@...e.hu, a.p.zijlstra@...llo.nl
Subject: find_busiest_group using lots of CPU

Hi,

I stuffed a few more SSDs into my text box. Running a simple workload
that just does streaming reads from 10 processes (throughput is around
2.2GB/sec), find_busiest_group() is using > 10% of the CPU time. This is
a 64 thread box.

The top two profile entries are:

    10.86%      fio  [kernel]                [k] find_busiest_group
                |          
                |--99.91%-- thread_return
                |          io_schedule
                |          sys_io_getevents
                |          system_call_fastpath
                |          0x7f4b50b61604
                |          |          
                |           --100.00%-- td_io_getevents
                |                     io_u_queued_complete
                |                     thread_main
                |                     run_threads
                |                     main
                |                     __libc_start_main
                 --0.09%-- [...]

     5.78%      fio  [kernel]                [k] cpumask_next_and
                |          
                |--67.21%-- thread_return
                |          io_schedule
                |          sys_io_getevents
                |          system_call_fastpath
                |          0x7f4b50b61604
                |          |          
                |           --100.00%-- td_io_getevents
                |                     io_u_queued_complete
                |                     thread_main
                |                     run_threads
                |                     main
                |                     __libc_start_main
                |          
                 --32.79%-- find_busiest_group
                           thread_return
                           io_schedule
                           sys_io_getevents
                           system_call_fastpath
                           0x7f4b50b61604
                           |          
                            --100.00%-- td_io_getevents
                                      io_u_queued_complete
                                      thread_main
                                      run_threads
                                      main
                                      __libc_start_main

This is with SCHED_DEBUG=y and SCHEDSTATS=y enabled, I just tried with
both disabled but that yields the same result (well actually worse, 22%
spent in there. dunno if that's normal "fluctuation"). GROUP_SCHED is
not set. This seems way excessive!

-- 
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/