| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <200901311905.24186.arvidjaar@mail.ru>
Date: Sat, 31 Jan 2009 19:05:21 +0300
From: Andrey Borzenkov <arvidjaar@...l.ru>
To: righi.andrea@...il.com
Cc: Geert Uytterhoeven <geert@...ux-m68k.org>,
Linux Frame Buffer Device Development
<linux-fbdev-devel@...ts.sourceforge.net>,
"Antonino A. Daplas" <adaplas@...il.com>,
linux-pm@...ts.linux-foundation.org,
Linux Kernel Development <linux-kernel@...r.kernel.org>
Subject: Re: [Linux-fbdev-devel] [2.6.29-rc2] fb_mmap: circular locking dependency on hibernation
On 31 of January 2009 18:53:23 Andrea Righi wrote:
> On 2009-01-30 11:23, Andrea Righi wrote:
> > On 2009-01-30 05:15, Andrey Borzenkov wrote:
> >> On 29 of January 2009 12:10:11 Geert Uytterhoeven wrote:
> >>> On Tue, 27 Jan 2009, Andrey Borzenkov wrote:
> >>>> I do not know if this is new; I usually do not hibernate at all
> >>>> and was testing another driver actually. I do not remember such
> >>>> issues in 2.6.28 though. Dmesg follows; config attached.
> >>>>
> >>>> [11715.652498]
> >>>> =======================================================
> >>>> [11715.652520] [ INFO: possible circular locking dependency
> >>>> detected ] [11715.652534] 2.6.29-rc2-1avb #11
> >>>> [11715.652541]
> >>>> -------------------------------------------------------
> >>>> [11715.652550] s2disk/25519 is trying to acquire lock:
> >>>> [11715.652562] (&fb_info->lock){--..}, at: [<c0223017>]
> >>>> fb_mmap+0x97/0x170 [11715.652623]
> >>>> [11715.652625] but task is already holding lock:
> >>>> [11715.652632] (&mm->mmap_sem){----}, at: [<c010639e>]
> >>>> sys_mmap2+0x8e/0xc0 [11715.652669]
> >>>> [11715.652671] which lock already depends on the new lock.
> >>>> [11715.652675]
> >>>> [11715.652681]
> >>>> [11715.652683] the existing dependency chain (in reverse order)
> >>>> is: [11715.652691]
> >>>> [11715.652693] -> #1 (&mm->mmap_sem){----}:
> >>>> [11715.652707] [<c0144ecf>] __lock_acquire+0x129f/0x1930
> >>>> [11715.652730] [<c01455bc>] lock_acquire+0x5c/0x80
> >>>> [11715.652742] [<c0181e57>] might_fault+0x77/0xa0
> >>>> [11715.652764] [<c0209325>] copy_from_user+0x35/0x130
> >>>> [11715.652782] [<c0223ef5>] fb_ioctl+0x1d5/0x3f0
> >>>> [11715.652797] [<c01a51d0>] vfs_ioctl+0x20/0x80
> >>>> [11715.652832] [<c01a53e2>] do_vfs_ioctl+0x72/0x570
> >>>> [11715.652845] [<c01a5919>] sys_ioctl+0x39/0x70
> >>>> [11715.652857] [<c0103371>] sysenter_do_call+0x12/0x31
> >>>> [11715.652870] [<ffffffff>] 0xffffffff
> >>>> [11715.652973]
> >>>> [11715.652975] -> #0 (&fb_info->lock){--..}:
> >>>> [11715.652988] [<c014503c>] __lock_acquire+0x140c/0x1930
> >>>> [11715.653000] [<c01455bc>] lock_acquire+0x5c/0x80
> >>>> [11715.653011] [<c030761a>] mutex_lock_nested+0xba/0x2f0
> >>>> [11715.653032] [<c0223017>] fb_mmap+0x97/0x170
> >>>> [11715.653044] [<c0188996>] mmap_region+0x1d6/0x530
> >>>> [11715.653067] [<c0188ec3>] do_mmap_pgoff+0x1d3/0x2f0
> >>>> [11715.653079] [<c01063bd>] sys_mmap2+0xad/0xc0
> >>>> [11715.653091] [<c0103371>] sysenter_do_call+0x12/0x31
> >>>> [11715.653102] [<ffffffff>] 0xffffffff
> >>>> [11715.653116]
> >>>> [11715.653119] other info that might help us debug this:
> >>>> [11715.653122]
> >>>> [11715.653130] 1 lock held by s2disk/25519:
> >>>> [11715.653137] #0: (&mm->mmap_sem){----}, at: [<c010639e>]
> >>>> sys_mmap2+0x8e/0xc0
> >>>> [11715.653156]
> >>>> [11715.653158] stack backtrace:
> >>>> [11715.653170] Pid: 25519, comm: s2disk Not tainted
> >>>> 2.6.29-rc2-1avb #11 [11715.653179] Call Trace:
> >>>> [11715.653190] [<c0306248>] ? printk+0x18/0x20
> >>>> [11715.653202] [<c01437df>] print_circular_bug_tail+0xcf/0xe0
> >>>> [11715.653213] [<c014503c>] __lock_acquire+0x140c/0x1930
> >>>> [11715.653224] [<c014328b>] ? trace_hardirqs_on+0xb/0x10
> >>>> [11715.653242] [<c0309125>] ? _spin_unlock_irqrestore+0x35/0x60
> >>>> [11715.653254] [<c0140dc5>] ? lock_release_holdtime+0x35/0x210
> >>>> [11715.653265] [<c01455bc>] lock_acquire+0x5c/0x80
> >>>> [11715.653277] [<c0223017>] ? fb_mmap+0x97/0x170
> >>>> [11715.653287] [<c030761a>] mutex_lock_nested+0xba/0x2f0
> >>>> [11715.653298] [<c0223017>] ? fb_mmap+0x97/0x170
> >>>> [11715.653309] [<c0223017>] ? fb_mmap+0x97/0x170
> >>>> [11715.653335] [<c019514d>] ? kmem_cache_alloc+0xad/0x100
> >>>> [11715.653347] [<c0223017>] fb_mmap+0x97/0x170
> >>>> [11715.653358] [<c0188996>] mmap_region+0x1d6/0x530
> >>>> [11715.653371] [<c0186a9d>] ?
> >>>> arch_get_unmapped_area_topdown+0x10d/0x170 [11715.653383]
> >>>> [<c0188ec3>] do_mmap_pgoff+0x1d3/0x2f0
> >>>> [11715.653395] [<c01063bd>] sys_mmap2+0xad/0xc0
> >>>> [11715.653406] [<c0103371>] sysenter_do_call+0x12/0x31
> >>>
> >>> Is this fixed by http://patchwork.kernel.org/patch/3259/?
> >>
> >> No. It just makes stack a bit more deep :)
> >
> > yep! :)
> >
> > Probably we should simply use an atomic notifier there, instead of
> > blocking. I'll do some tests and post a new fix.
>
> No, the solution is no so trivial. Some notifier callbacks can sleep,
> so the usage of a blocking notifier is mandatory. It seems we need a
> more complicated solution to totally fix it. At least the previous
> fix avoids the circular locking of fb_info->lock and mm->mmap_sem due
> to copy_from/to_user() called with fb_info->lock held...
>
Well, this is serious regression that must be fixed before release. It
apparently prevents other notifiers from being called, with the effect
that e.g. firmware is no more cached on suspend and applied later on
resume -> controller non-functional.
*Any* workaround would still be useful.
> -Andrea
>
> > Thanks,
> > -Andrea
> >
> >> [ 408.267429]
> >> ======================================================= [
> >> 408.267448] [ INFO: possible circular locking dependency detected
> >> ] [ 408.267462] 2.6.29-rc3-1avb #14
> >> [ 408.267469]
> >> ------------------------------------------------------- [
> >> 408.267478] s2disk/5695 is trying to acquire lock:
> >> [ 408.267487] (&fb_info->lock){--..}, at: [<c02235e7>]
> >> fb_mmap+0x97/0x170
> >> [ 408.267544]
> >> [ 408.267546] but task is already holding lock:
> >> [ 408.267553] (&mm->mmap_sem){----}, at: [<c010639e>]
> >> sys_mmap2+0x8e/0xc0
> >> [ 408.267591]
> >> [ 408.267593] which lock already depends on the new lock.
> >> [ 408.267596]
> >> [ 408.267603]
> >> [ 408.267604] the existing dependency chain (in reverse order)
> >> is: [ 408.267612]
> >> [ 408.267614] -> #3 (&mm->mmap_sem){----}:
> >> [ 408.267628] [<c014520f>] __lock_acquire+0x129f/0x1930
> >> [ 408.267650] [<c01458fc>] lock_acquire+0x5c/0x80
> >> [ 408.267661] [<c0182117>] might_fault+0x77/0xa0
> >> [ 408.267683] [<c0209916>] copy_to_user+0x36/0x120
> >> [ 408.267699] [<c01a60f7>] filldir+0x97/0xe0
> >> [ 408.267735] [<c01de339>] sysfs_readdir+0x129/0x220
> >> [ 408.267751] [<c01a62a6>] vfs_readdir+0x86/0xa0
> >> [ 408.267763] [<c01a63e8>] sys_getdents+0x68/0xc0
> >> [ 408.267775] [<c010344a>] syscall_call+0x7/0xb
> >> [ 408.267787] [<ffffffff>] 0xffffffff
> >> [ 408.267862]
> >> [ 408.267864] -> #2 (sysfs_mutex){--..}:
> >> [ 408.267877] [<c014520f>] __lock_acquire+0x129f/0x1930
> >> [ 408.267889] [<c01458fc>] lock_acquire+0x5c/0x80
> >> [ 408.267899] [<c0307f3a>] mutex_lock_nested+0xba/0x2f0
> >> [ 408.267921] [<c01de6cc>] sysfs_addrm_start+0x2c/0xc0
> >> [ 408.267933] [<c01dec80>] create_dir+0x40/0x90
> >> [ 408.267944] [<c01decfb>] sysfs_create_dir+0x2b/0x50
> >> [ 408.267955] [<c02036bc>] kobject_add_internal+0xbc/0x1b0
> >> [ 408.267984] [<c0203881>] kobject_add_varg+0x31/0x50
> >> [ 408.267996] [<c02038fc>] kobject_add+0x2c/0x60
> >> [ 408.268008] [<c0273448>] device_add+0xa8/0x550
> >> [ 408.268041] [<c0273902>] device_register+0x12/0x20
> >> [ 408.268053] [<c02739bb>] device_create_vargs+0xab/0xc0
> >> [ 408.268065] [<c02739f8>] device_create+0x28/0x30
> >> [ 408.268077] [<c026cc6d>] register_con_driver+0xed/0x130
> >> [ 408.268090] [<c026e18b>] take_over_console+0x1b/0x50
> >> [ 408.268102] [<c023052d>] fbcon_takeover+0x5d/0xb0
> >> [ 408.268125] [<c02311a0>] fbcon_event_notify+0x820/0x900
> >> [ 408.268137] [<c030c3b3>] notifier_call_chain+0x53/0xa0
> >> [ 408.268161] [<c0138ab4>]
> >> __blocking_notifier_call_chain+0x44/0x60
> >> [ 408.268184] [<c0138aea>]
> >> blocking_notifier_call_chain+0x1a/0x20
> >> [ 408.268197] [<c0222f81>]
> >> fb_notifier_call_chain+0x11/0x20 [ 408.268209]
> >> [<c0223f08>] register_framebuffer+0x168/0x220 [ 408.268223]
> >> [<c0414ba2>] vesafb_probe+0x542/0x783 [ 408.268250]
> >> [<c02765af>] platform_drv_probe+0xf/0x20 [ 408.268269]
> >> [<c02757c7>] driver_probe_device+0x87/0x1b0 [ 408.268281]
> >> [<c0275988>] __device_attach+0x8/0x10 [ 408.268291]
> >> [<c0274e0b>] bus_for_each_drv+0x5b/0x80 [ 408.268308]
> >> [<c0275a36>] device_attach+0x76/0x80 [ 408.268319]
> >> [<c0274c27>] bus_attach_device+0x47/0x70 [ 408.268331]
> >> [<c02736c3>] device_add+0x323/0x550 [ 408.268342]
> >> [<c0276f85>] platform_device_add+0x175/0x1c0 [ 408.268358]
> >> [<c0414e7d>] vesafb_init+0x9a/0x1ec [ 408.268370]
> >> [<c010111a>] do_one_initcall+0x2a/0x160 [ 408.268382]
> >> [<c04004d5>] kernel_init+0x83/0xd5
> >> [ 408.268411] [<c0103a77>] kernel_thread_helper+0x7/0x10
> >> [ 408.268423] [<ffffffff>] 0xffffffff
> >> [ 408.268436]
> >> [ 408.268438] -> #1 ((fb_notifier_list).rwsem){----}:
> >> [ 408.268451] [<c014520f>] __lock_acquire+0x129f/0x1930
> >> [ 408.268463] [<c01458fc>] lock_acquire+0x5c/0x80
> >> [ 408.268474] [<c0308879>] down_read+0x49/0x90
> >> [ 408.268484] [<c0138a9a>]
> >> __blocking_notifier_call_chain+0x2a/0x60
> >> [ 408.268497] [<c0138aea>]
> >> blocking_notifier_call_chain+0x1a/0x20
> >> [ 408.268509] [<c0222f81>]
> >> fb_notifier_call_chain+0x11/0x20 [ 408.268521]
> >> [<c022459e>] do_fb_ioctl+0x2ae/0x5d0 [ 408.268533]
> >> [<c02248dd>] fb_ioctl+0x1d/0x20
> >> [ 408.268545] [<c01a54b0>] vfs_ioctl+0x20/0x80
> >> [ 408.268558] [<c01a56c2>] do_vfs_ioctl+0x72/0x570
> >> [ 408.268570] [<c01a5bf9>] sys_ioctl+0x39/0x70
> >> [ 408.268581] [<c0103371>] sysenter_do_call+0x12/0x31
> >> [ 408.268592] [<ffffffff>] 0xffffffff
> >> [ 408.268623]
> >> [ 408.268625] -> #0 (&fb_info->lock){--..}:
> >> [ 408.268638] [<c014537c>] __lock_acquire+0x140c/0x1930
> >> [ 408.268649] [<c01458fc>] lock_acquire+0x5c/0x80
> >> [ 408.268661] [<c0307f3a>] mutex_lock_nested+0xba/0x2f0
> >> [ 408.268672] [<c02235e7>] fb_mmap+0x97/0x170
> >> [ 408.268684] [<c0188c76>] mmap_region+0x1d6/0x530
> >> [ 408.268706] [<c01891a3>] do_mmap_pgoff+0x1d3/0x2f0
> >> [ 408.268718] [<c01063bd>] sys_mmap2+0xad/0xc0
> >> [ 408.268729] [<c0103371>] sysenter_do_call+0x12/0x31
> >> [ 408.268740] [<ffffffff>] 0xffffffff
> >> [ 408.268752]
> >> [ 408.268754] other info that might help us debug this:
> >> [ 408.268758]
> >> [ 408.268767] 1 lock held by s2disk/5695:
> >> [ 408.268773] #0: (&mm->mmap_sem){----}, at: [<c010639e>]
> >> sys_mmap2+0x8e/0xc0
> >> [ 408.268792]
> >> [ 408.268794] stack backtrace:
> >> [ 408.268805] Pid: 5695, comm: s2disk Not tainted 2.6.29-rc3-1avb
> >> #14 [ 408.268814] Call Trace:
> >> [ 408.268825] [<c0306b68>] ? printk+0x18/0x20
> >> [ 408.268836] [<c0143b1f>] print_circular_bug_tail+0xcf/0xe0
> >> [ 408.268847] [<c014537c>] __lock_acquire+0x140c/0x1930
> >> [ 408.268862] [<c01487f0>] ? futex_wait+0x170/0x450
> >> [ 408.268881] [<c011826c>] ? try_to_wake_up+0x10c/0x120
> >> [ 408.268894] [<c0309a45>] ? _spin_unlock_irqrestore+0x35/0x60
> >> [ 408.268906] [<c0141105>] ? lock_release_holdtime+0x35/0x210
> >> [ 408.268917] [<c01458fc>] lock_acquire+0x5c/0x80
> >> [ 408.268928] [<c02235e7>] ? fb_mmap+0x97/0x170
> >> [ 408.268939] [<c0307f3a>] mutex_lock_nested+0xba/0x2f0
> >> [ 408.268950] [<c02235e7>] ? fb_mmap+0x97/0x170
> >> [ 408.268961] [<c02235e7>] ? fb_mmap+0x97/0x170
> >> [ 408.268987] [<c019542d>] ? kmem_cache_alloc+0xad/0x100
> >> [ 408.268999] [<c02235e7>] fb_mmap+0x97/0x170
> >> [ 408.269010] [<c0188c76>] mmap_region+0x1d6/0x530
> >> [ 408.269023] [<c0186d7d>] ?
> >> arch_get_unmapped_area_topdown+0x10d/0x170
> >> [ 408.269035] [<c01891a3>] do_mmap_pgoff+0x1d3/0x2f0
> >> [ 408.269047] [<c01063bd>] sys_mmap2+0xad/0xc0
> >> [ 408.269058] [<c0103371>] sysenter_do_call+0x12/0x31
Download attachment "signature.asc " of type "application/pgp-signature" (198 bytes)
Powered by blists - more mailing lists