| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20120720143559.GB2721@linux.vnet.ibm.com> Date: Fri, 20 Jul 2012 07:35:59 -0700 From: "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com> To: "Srivatsa S. Bhat" <srivatsa.bhat@...ux.vnet.ibm.com> Cc: Thomas Gleixner <tglx@...utronix.de>, LKML <linux-kernel@...r.kernel.org>, Ingo Molnar <mingo@...e.hu>, Peter Zijlstra <peterz@...radead.org>, Rusty Russell <rusty@...tcorp.com.au>, Namhyung Kim <namhyung@...nel.org> Subject: Re: [Patch 0/7] Per cpu thread hotplug infrastructure - V3 On Fri, Jul 20, 2012 at 06:47:30PM +0530, Srivatsa S. Bhat wrote: > On 07/19/2012 05:24 AM, Paul E. McKenney wrote: > > On Wed, Jul 18, 2012 at 11:06:52PM +0530, Srivatsa S. Bhat wrote: > >> On 07/16/2012 08:52 PM, Paul E. McKenney wrote: > >>> On Mon, Jul 16, 2012 at 10:42:34AM -0000, Thomas Gleixner wrote: > >>>> The following series implements the infrastructure for parking and > >>>> unparking kernel threads to avoid the full teardown and fork on cpu > >>>> hotplug operations along with management infrastructure for hotplug > >>>> and users. > >>>> > >>>> Changes vs. V2: > >>>> > >>>> Use callbacks for all functionality. Thanks to Rusty for pointing > >>>> that out. It makes the use sites nice and simple and keeps all the > >>>> code which would be duplicated otherwise on the core. > >>> > >>> Hello, Thomas, > >>> > >>> What version should I apply this patchset to? I tried v3.5-rc7, but > >>> got lots of warnings (one shown below) and the watchdog patch did not > >>> apply. > >>> > >> > >> Hi Paul, > >> > >> This patchset applies cleanly on Thomas' smp/hotplug branch in the -tip > >> tree. > > > > Thank you, Srivatsa, works much better. Still get "scheduling while > > atomic", looking into that. > > > > Got a chance to run this patchset now.. Even I am getting "scheduling while > atomic" messages like shown below.. Hmmm... Here is what little I have done so far (lots of completing demands on time this week, but I should have a goodly block of time to focus on this today): 1. The failure is from the softirq modifications. Reverting that commit gets rid of the failures. 2. As one would expect, CONFIG_PREEMPT=n kernels do not have the problem, which of course indicates a preempt_disable() imbalance. 3. I was unable to spot the problem by inspection, but this is not too surprising given the high level of distraction this week. 4. Instrumentation shows that preempt_count() grows slowly with time, but with the upper bits zero. This confirms the preempt_disable imbalance. 5. I am currently placing WARN_ONCE() calls in the code to track this down. When I do find it, I fully expect to feel very stupid about my efforts on #3 above. ;-) Thanx, Paul > Regards, > Srivatsa S. Bhat > > ----------------- > > BUG: sleeping function called from invalid context at kernel/softirq.c:748 > in_atomic(): 1, irqs_disabled(): 0, pid: 3, name: ksoftirqd/0 > INFO: lockdep is turned off. > Pid: 3, comm: ksoftirqd/0 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 > Call Trace: > [<ffffffff8108e7ac>] __might_sleep+0x18c/0x250 > [<ffffffff8105ea4f>] run_ksoftirqd+0x4f/0x70 > [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 > [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 > [<ffffffff8107e7ee>] kthread+0xee/0x100 > [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 > [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 > [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 > [<ffffffff8158eb70>] ? gs_change+0x13/0x13 > BUG: scheduling while atomic: ksoftirqd/0/3/0x1000000b > INFO: lockdep is turned off. > Modules linked in: ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan] > Pid: 3, comm: ksoftirqd/0 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 > Call Trace: > [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90 > [<ffffffff81582818>] __schedule+0x7c8/0x880 > [<ffffffff81092cea>] __cond_resched+0x2a/0x40 > [<ffffffff81582960>] _cond_resched+0x30/0x40 > SELinux: 2048 avtab hash slots, 250818 rules. > [<ffffffff8105ea54>] run_ksoftirqd+0x54/0x70 > [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 > [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 > [<ffffffff8107e7ee>] kthread+0xee/0x100 > [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 > [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 > [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 > [<ffffffff8158eb70>] ? gs_change+0x13/0x13 > > [...] > > BUG: sleeping function called from invalid context at kernel/softirq.c:748 > in_atomic(): 1, irqs_disabled(): 0, pid: 37, name: ksoftirqd/8 > INFO: lockdep is turned off. > Pid: 37, comm: ksoftirqd/8 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 > Call Trace: > [<ffffffff8108e7ac>] __might_sleep+0x18c/0x250 > [<ffffffff8105ea4f>] run_ksoftirqd+0x4f/0x70 > [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 > Welcome to [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 > [<ffffffff8107e7ee>] kthread+0xee/0x100 > Red Hat [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 > [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 > [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 > [<ffffffff8158eb70>] ? gs_change+0x13/0x13 > BUG: scheduling while atomic: ksoftirqd/8/37/0x10000004 > INFO: lockdep is turned off. > Modules linked in: ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan] > Pid: 37, comm: ksoftirqd/8 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 > Call Trace: > Enterprise Linu [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90 > x Server > [<ffffffff81582818>] __schedule+0x7c8/0x880 > [<ffffffff81092cea>] __cond_resched+0x2a/0x40 > [<ffffffff81582960>] _cond_resched+0x30/0x40 > [<ffffffff8105ea54>] run_ksoftirqd+0x54/0x70 > [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0 > [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 > [<ffffffff8107e7ee>] kthread+0xee/0x100 > [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 > [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 > [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 > [<ffffffff8158eb70>] ? gs_change+0x13/0x13 > > [...] > > BUG: scheduling while atomic: watchdog/15/64/0x0000009b > INFO: lockdep is turned off. > Modules linked in: ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat xt_CHECKSUM iptable_mangle bridge stp llc autofs4 cpufreq_powersave sunrpc cpufreq_ondemand acpi_cpufreq freq_table mperf ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror dm_region_hash dm_log dm_round_robin scsi_dh_rdac dm_multipath vhost_net macvtap macvlan tun uinput cdc_ether usbnet mii serio_raw kvm_intel kvm microcode i2c_i801 i2c_core lpc_ich mfd_core shpchp ioatdma dca i7core_edac edac_core bnx2 dm_mod sg ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan] > Pid: 64, comm: watchdog/15 Tainted: G W 3.5.0-rc1-threadhotplugv3-debug #2 > Call Trace: > [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90 > [<ffffffff81582818>] __schedule+0x7c8/0x880 > [<ffffffff81582bf9>] schedule+0x29/0x70 > [<ffffffff81582f77>] schedule_preempt_disabled+0x27/0x40 > [<ffffffff810c8b33>] smpboot_thread_fn+0xf3/0x1f0 > [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80 > [<ffffffff8107e7ee>] kthread+0xee/0x100 > [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10 > [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13 > [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70 > [<ffffffff8158eb70>] ? gs_change+0x13/0x13 > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists