| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <10192e43-c21d-44e4-915d-bf77a50c22c4@gmail.com>
Date: Tue, 28 May 2019 11:38:43 +0200
From: Vicente Bergas <vicencb@...il.com>
To: Al Viro <viro@...iv.linux.org.uk>
Cc: <linux-fsdevel@...r.kernel.org>, <linux-kernel@...r.kernel.org>
Subject: Re: d_lookup: Unable to handle kernel paging request
On Wednesday, May 22, 2019 6:29:46 PM CEST, Al Viro wrote:
>...
> IOW, here we have also run into bogus hlist forward pointer or head -
> same 0x1000000 in one case and 0x0000880001000000 in two others.
>
> Have you tried to see if KASAN catches anything on those loads?
> Use-after-free, for example... Another thing to try: slap
> WARN_ON(entry->d_flags & DCACHE_NORCU);
> in __d_rehash() and see if it triggers.
Hi Al,
after 5 days with v5.2-rc1 + KASAN + WARN_ON could not reproduce the issue.
Neither the first day running v5.3-rc2 + WARN_ON. But today 6 times.
So, there is no KASAN and also the WARN_ON, being there, did not trigger.
The first trace hapenned while untaring a big file into tmpfs. The other
five while "git pull -r" severeal repos on f2fs.
Regards,
Vicenç.
Unable to handle kernel paging request at virtual address 0000000001000018
Mem abort info:
ESR = 0x96000004
Exception class = DABT (current EL), IL = 32 bits
SET = 0, FnV = 0
EA = 0, S1PTW = 0
Data abort info:
ISV = 0, ISS = 0x00000004
CM = 0, WnR = 0
user pgtable: 4k pages, 48-bit VAs, pgdp=00000000aeab4000
[0000000001000018] pgd=0000000000000000
Internal error: Oops: 96000004 [#1] SMP
CPU: 4 PID: 1172 Comm: tar Not tainted 5.2.0-rc2 #1
Hardware name: Sapphire-RK3399 Board (DT)
pstate: 00000005 (nzcv daif -PAN -UAO)
pc : __d_lookup+0x58/0x198
lr : d_lookup+0x38/0x68
sp : ffff000012663b90
x29: ffff000012663b90 x28: ffff000012663d58
x27: 0000000000000000 x26: ffff8000ae7cc900
x25: 0000000000000001 x24: ffffffffffffffff
x23: 00000000ce9c8f81 x22: 0000000000000000
x21: 0000000000000001 x20: ffff000012663d58
x19: 0000000001000000 x18: 0000000000000000
x17: 0000000000000000 x16: 0000000000000000
x15: 0000000000000000 x14: 0000000000000000
x13: 0000000000000000 x12: 0000000000000000
x11: fefefefefefefeff x10: b4fea3d0a3a4b4fe
x9 : d237122a91454b69 x8 : a0591ae4450bed6a
x7 : 5845a2c80f79d4e7 x6 : 0000000000000004
x5 : 0000000000000000 x4 : ffff000012663d58
x3 : ffff000010828a68 x2 : ffff000010828000
x1 : ffff8000f3000000 x0 : 00000000000674e4
Call trace:
__d_lookup+0x58/0x198
d_lookup+0x38/0x68
path_openat+0x4a8/0xfb8
do_filp_open+0x60/0xd8
do_sys_open+0x144/0x1f8
__arm64_sys_openat+0x20/0x28
el0_svc_handler+0x74/0x140
el0_svc+0x8/0xc
Code: 92800018 a9025bf5 d2800016 52800035 (b9401a62)
---[ end trace 7fc40d1e6d2ed53e ]---
Unable to handle kernel paging request at virtual address 0000000000fffffc
Mem abort info:
ESR = 0x96000004
Exception class = DABT (current EL), IL = 32 bits
SET = 0, FnV = 0
EA = 0, S1PTW = 0
Data abort info:
ISV = 0, ISS = 0x00000004
CM = 0, WnR = 0
user pgtable: 4k pages, 48-bit VAs, pgdp=000000007af3e000
[0000000000fffffc] pgd=0000000000000000
Internal error: Oops: 96000004 [#1] SMP
CPU: 4 PID: 2124 Comm: git Not tainted 5.2.0-rc2 #1
Hardware name: Sapphire-RK3399 Board (DT)
pstate: 00000005 (nzcv daif -PAN -UAO)
pc : __d_lookup_rcu+0x68/0x198
lr : lookup_fast+0x44/0x2e8
sp : ffff0000130b3b60
x29: ffff0000130b3b60 x28: 00000000ce99d070
x27: ffffffffffffffff x26: 0000000000000026
x25: ffff8000ecec6030 x24: ffff0000130b3c2c
x23: 0000000000000006 x22: 00000026ce99d070
x21: ffff8000811f3d80 x20: 0000000000020000
x19: 0000000001000000 x18: 0000000000000000
x17: 0000000000000000 x16: 0000000000000000
x15: 0000000000000000 x14: 0000000000000000
x13: 0000000000000000 x12: 0000000000000000
x11: fefefefefefefeff x10: e4d0b2e6e2b4b6e9
x9 : 5096e90463dfacb0 x8 : 2b4f8961c30ebc93
x7 : aec349fb204a7256 x6 : 4fd9025392b5761a
x5 : 02ff010101030100 x4 : ffff8000f3000000
x3 : ffff0000130b3d58 x2 : ffff0000130b3c2c
x1 : 00000000000674ce x0 : ffff8000811f3d80
Call trace:
__d_lookup_rcu+0x68/0x198
lookup_fast+0x44/0x2e8
path_openat+0x19c/0xfb8
do_filp_open+0x60/0xd8
do_sys_open+0x144/0x1f8
__arm64_sys_openat+0x20/0x28
el0_svc_handler+0x74/0x140
el0_svc+0x8/0xc
Code: 9280001b 14000003 f9400273 b4000793 (b85fc265)
---[ end trace 6bd1b3b7588a78fe ]---
Unable to handle kernel paging request at virtual address 0000880000fffffc
Mem abort info:
ESR = 0x96000004
Exception class = DABT (current EL), IL = 32 bits
SET = 0, FnV = 0
EA = 0, S1PTW = 0
Data abort info:
ISV = 0, ISS = 0x00000004
CM = 0, WnR = 0
user pgtable: 4k pages, 48-bit VAs, pgdp=00000000867ac000
[0000880000fffffc] pgd=0000000000000000
Internal error: Oops: 96000004 [#2] SMP
CPU: 4 PID: 2183 Comm: git Tainted: G D 5.2.0-rc2 #1
Hardware name: Sapphire-RK3399 Board (DT)
pstate: 00000005 (nzcv daif -PAN -UAO)
pc : __d_lookup_rcu+0x68/0x198
lr : lookup_fast+0x44/0x2e8
sp : ffff00001325ba90
x29: ffff00001325ba90 x28: 00000000ce99f075
x27: ffffffffffffffff x26: 0000000000000007
x25: ffff8000ecec402a x24: ffff00001325bb5c
x23: 0000000000000007 x22: 00000007ce99f075
x21: ffff80007a810c00 x20: 0000000000000000
x19: 0000880001000000 x18: 0000000000000000
x17: 0000000000000000 x16: 0000000000000000
x15: 0000000000000000 x14: 0000000000000000
x13: 0000000000000000 x12: 0000000000000000
x11: fefefefefefefeff x10: d0bbbcbfa6b2b9bc
x9 : 0000000000000000 x8 : ffff80007a810c00
x7 : 6cad9ff29d8de19c x6 : ff94ec6f0ce3656c
x5 : ffff8000ecec402a x4 : ffff8000f3000000
x3 : ffff00001325bc78 x2 : ffff00001325bb5c
x1 : 00000000000674cf x0 : ffff80007a810c00
Call trace:
__d_lookup_rcu+0x68/0x198
lookup_fast+0x44/0x2e8
walk_component+0x34/0x2e0
path_lookupat.isra.0+0x5c/0x1e0
filename_lookup+0x78/0xf0
user_path_at_empty+0x44/0x58
vfs_statx+0x70/0xd0
__se_sys_newfstatat+0x20/0x40
__arm64_sys_newfstatat+0x18/0x20
el0_svc_handler+0x74/0x140
el0_svc+0x8/0xc
Code: 9280001b 14000003 f9400273 b4000793 (b85fc265)
---[ end trace 6bd1b3b7588a78ff ]---
Unable to handle kernel paging request at virtual address 0000880000fffffc
Mem abort info:
ESR = 0x96000004
Exception class = DABT (current EL), IL = 32 bits
SET = 0, FnV = 0
EA = 0, S1PTW = 0
Data abort info:
ISV = 0, ISS = 0x00000004
CM = 0, WnR = 0
user pgtable: 4k pages, 48-bit VAs, pgdp=00000000867ac000
[0000880000fffffc] pgd=0000000000000000
Internal error: Oops: 96000004 [#3] SMP
CPU: 4 PID: 2180 Comm: git Tainted: G D 5.2.0-rc2 #1
Hardware name: Sapphire-RK3399 Board (DT)
pstate: 00000005 (nzcv daif -PAN -UAO)
pc : __d_lookup_rcu+0x68/0x198
lr : lookup_fast+0x44/0x2e8
sp : ffff000012a3ba90
x29: ffff000012a3ba90 x28: 00000000ce99f075
x27: ffffffffffffffff x26: 0000000000000007
x25: ffff8000ecec702a x24: ffff000012a3bb5c
x23: 0000000000000007 x22: 00000007ce99f075
x21: ffff80007a810c00 x20: 0000000000000000
x19: 0000880001000000 x18: 0000000000000000
x17: 0000000000000000 x16: 0000000000000000
x15: 0000000000000000 x14: 0000000000000000
x13: 0000000000000000 x12: 0000000000000000
x11: fefefefefefefeff x10: d0bbbcbfa6b2b9bc
x9 : 0000000000000000 x8 : ffff80007a810c00
x7 : 6cad9ff29d8de19c x6 : ff94ec6f0ce3656c
x5 : ffff8000ecec702a x4 : ffff8000f3000000
x3 : ffff000012a3bc78 x2 : ffff000012a3bb5c
x1 : 00000000000674cf x0 : ffff80007a810c00
Call trace:
__d_lookup_rcu+0x68/0x198
lookup_fast+0x44/0x2e8
walk_component+0x34/0x2e0
path_lookupat.isra.0+0x5c/0x1e0
filename_lookup+0x78/0xf0
user_path_at_empty+0x44/0x58
vfs_statx+0x70/0xd0
__se_sys_newfstatat+0x20/0x40
__arm64_sys_newfstatat+0x18/0x20
el0_svc_handler+0x74/0x140
el0_svc+0x8/0xc
Code: 9280001b 14000003 f9400273 b4000793 (b85fc265)
---[ end trace 6bd1b3b7588a7900 ]---
Unable to handle kernel paging request at virtual address 0000880000fffffc
Mem abort info:
ESR = 0x96000004
Exception class = DABT (current EL), IL = 32 bits
SET = 0, FnV = 0
EA = 0, S1PTW = 0
Data abort info:
ISV = 0, ISS = 0x00000004
CM = 0, WnR = 0
user pgtable: 4k pages, 48-bit VAs, pgdp=0000000073f2f000
[0000880000fffffc] pgd=0000000000000000
Internal error: Oops: 96000004 [#4] SMP
CPU: 4 PID: 2210 Comm: git Tainted: G D 5.2.0-rc2 #1
Hardware name: Sapphire-RK3399 Board (DT)
pstate: 00000005 (nzcv daif -PAN -UAO)
pc : __d_lookup_rcu+0x68/0x198
lr : lookup_fast+0x44/0x2e8
sp : ffff0000132bba90
x29: ffff0000132bba90 x28: 00000000ce99e1a6
x27: ffffffffffffffff x26: 000000000000000c
x25: ffff8000f21dd036 x24: ffff0000132bbb5c
x23: 0000000000000004 x22: 0000000cce99e1a6
x21: ffff800074dd8d80 x20: 0000000000000000
x19: 0000880001000000 x18: 0000000000000000
x17: 0000000000000000 x16: 0000000000000000
x15: 0000000000000000 x14: 0000000000000000
x13: 0000000000000000 x12: 0000000000000000
x11: fefefefefefefeff x10: d0d0d0d0b8fea4b3
x9 : 40bcd8645005512e x8 : c433ade89ebd10f9
x7 : c6b69091eeb194d2 x6 : 848f758ca69635b4
x5 : ffff8000f21dd036 x4 : ffff8000f3000000
x3 : ffff0000132bbc78 x2 : ffff0000132bbb5c
x1 : 00000000000674cf x0 : ffff800074dd8d80
Call trace:
__d_lookup_rcu+0x68/0x198
lookup_fast+0x44/0x2e8
walk_component+0x34/0x2e0
path_lookupat.isra.0+0x5c/0x1e0
filename_lookup+0x78/0xf0
user_path_at_empty+0x44/0x58
vfs_statx+0x70/0xd0
__se_sys_newfstatat+0x20/0x40
__arm64_sys_newfstatat+0x18/0x20
el0_svc_handler+0x74/0x140
el0_svc+0x8/0xc
Code: 9280001b 14000003 f9400273 b4000793 (b85fc265)
---[ end trace 6bd1b3b7588a7901 ]---
Unable to handle kernel paging request at virtual address 0000880000fffffc
Mem abort info:
ESR = 0x96000004
Exception class = DABT (current EL), IL = 32 bits
SET = 0, FnV = 0
EA = 0, S1PTW = 0
Data abort info:
ISV = 0, ISS = 0x00000004
CM = 0, WnR = 0
user pgtable: 4k pages, 48-bit VAs, pgdp=0000000073f2f000
[0000880000fffffc] pgd=0000000000000000
Internal error: Oops: 96000004 [#5] SMP
CPU: 5 PID: 2200 Comm: git Tainted: G D 5.2.0-rc2 #1
Hardware name: Sapphire-RK3399 Board (DT)
pstate: 00000005 (nzcv daif -PAN -UAO)
pc : __d_lookup_rcu+0x68/0x198
lr : lookup_fast+0x44/0x2e8
sp : ffff000013263a90
x29: ffff000013263a90 x28: 00000000ce99e1a6
x27: ffffffffffffffff x26: 000000000000000c
x25: ffff8000f0a6f036 x24: ffff000013263b5c
x23: 0000000000000004 x22: 0000000cce99e1a6
x21: ffff800074dd8d80 x20: 0000000000000000
x19: 0000880001000000 x18: 0000000000000000
x17: 0000000000000000 x16: 0000000000000000
x15: 0000000000000000 x14: 0000000000000000
x13: 0000000000000000 x12: 0000000000000000
x11: fefefefefefefeff x10: d0d0d0d0b8fea4b3
x9 : 40bcd8645005512e x8 : c433ade89ebd10f9
x7 : c6b69091eeb194d2 x6 : 848f758ca69635b4
x5 : ffff8000f0a6f036 x4 : ffff8000f3000000
x3 : ffff000013263c78 x2 : ffff000013263b5c
x1 : 00000000000674cf x0 : ffff800074dd8d80
Call trace:
__d_lookup_rcu+0x68/0x198
lookup_fast+0x44/0x2e8
walk_component+0x34/0x2e0
path_lookupat.isra.0+0x5c/0x1e0
filename_lookup+0x78/0xf0
user_path_at_empty+0x44/0x58
vfs_statx+0x70/0xd0
__se_sys_newfstatat+0x20/0x40
__arm64_sys_newfstatat+0x18/0x20
el0_svc_handler+0x74/0x140
el0_svc+0x8/0xc
Code: 9280001b 14000003 f9400273 b4000793 (b85fc265)
---[ end trace 6bd1b3b7588a7902 ]---
00000000000001c8 <__d_rehash>:
return dentry_hashtable + (hash >> d_hash_shift);
1c8: 90000001 adrp x1, 0 <find_submount>
1c8: R_AARCH64_ADR_PREL_PG_HI21 .data..read_mostly
1cc: 91000022 add x2, x1, #0x0
1cc: R_AARCH64_ADD_ABS_LO12_NC .data..read_mostly
static void __d_rehash(struct dentry *entry)
{
struct hlist_bl_head *b = d_hash(entry->d_name.hash);
WARN_ON(entry->d_flags & DCACHE_NORCU);
1d0: b9400003 ldr w3, [x0]
return dentry_hashtable + (hash >> d_hash_shift);
1d4: f9400025 ldr x5, [x1]
1d4: R_AARCH64_LDST64_ABS_LO12_NC .data..read_mostly
1d8: b9400841 ldr w1, [x2, #8]
1dc: b9402002 ldr w2, [x0, #32]
1e0: 1ac12442 lsr w2, w2, w1
1e4: 8b020ca1 add x1, x5, x2, lsl #3
WARN_ON(entry->d_flags & DCACHE_NORCU);
1e8: 37f00343 tbnz w3, #30, 250 <__d_rehash+0x88>
__READ_ONCE_SIZE;
1ec: f9400023 ldr x3, [x1]
if (READ_ONCE(*p) & mask)
1f0: 37000283 tbnz w3, #0, 240 <__d_rehash+0x78>
1f4: f9800031 prfm pstl1strm, [x1]
1f8: c85ffc23 ldaxr x3, [x1]
1fc: b2400064 orr x4, x3, #0x1
200: c8067c24 stxr w6, x4, [x1]
204: 35ffffa6 cbnz w6, 1f8 <__d_rehash+0x30>
while (unlikely(test_and_set_bit_lock(bitnum, addr))) {
208: 370001c3 tbnz w3, #0, 240 <__d_rehash+0x78>
((unsigned long)h->first & ~LIST_BL_LOCKMASK);
20c: f86278a3 ldr x3, [x5, x2, lsl #3]
hlist_bl_lock(b);
hlist_bl_add_head_rcu(&entry->d_hash, b);
210: 91002004 add x4, x0, #0x8
214: 927ff863 and x3, x3, #0xfffffffffffffffe
struct hlist_bl_node *first;
/* don't need hlist_bl_first_rcu because we're under lock */
first = hlist_bl_first(h);
n->next = first;
218: f9000403 str x3, [x0, #8]
if (first)
21c: b4000043 cbz x3, 224 <__d_rehash+0x5c>
first->pprev = &n->next;
220: f9000464 str x4, [x3, #8]
rcu_assign_pointer(h->first,
224: b2400084 orr x4, x4, #0x1
n->pprev = &h->first;
228: f9000801 str x1, [x0, #16]
rcu_assign_pointer(h->first,
22c: c89ffc24 stlr x4, [x1]
230: f86278a0 ldr x0, [x5, x2, lsl #3]
old &= ~BIT_MASK(nr);
234: 927ff800 and x0, x0, #0xfffffffffffffffe
238: c89ffc20 stlr x0, [x1]
hlist_bl_unlock(b);
}
23c: d65f03c0 ret
240: d503203f yield
244: f9400023 ldr x3, [x1]
} while (test_bit(bitnum, addr));
248: 3707ffc3 tbnz w3, #0, 240 <__d_rehash+0x78>
24c: 17ffffe8 b 1ec <__d_rehash+0x24>
WARN_ON(entry->d_flags & DCACHE_NORCU);
250: d4210000 brk #0x800
preempt_disable();
254: 17ffffe6 b 1ec <__d_rehash+0x24>
...
0000000000002d10 <__d_lookup_rcu>:
{
2d10: a9b97bfd stp x29, x30, [sp, #-112]!
2d14: aa0103e3 mov x3, x1
return dentry_hashtable + (hash >> d_hash_shift);
2d18: 90000004 adrp x4, 0 <find_submount>
2d18: R_AARCH64_ADR_PREL_PG_HI21 .data..read_mostly
{
2d1c: 910003fd mov x29, sp
2d20: a90153f3 stp x19, x20, [sp, #16]
return dentry_hashtable + (hash >> d_hash_shift);
2d24: 91000081 add x1, x4, #0x0
2d24: R_AARCH64_ADD_ABS_LO12_NC .data..read_mostly
{
2d28: a9025bf5 stp x21, x22, [sp, #32]
2d2c: a9046bf9 stp x25, x26, [sp, #64]
const unsigned char *str = name->name;
2d30: a9406476 ldp x22, x25, [x3]
return dentry_hashtable + (hash >> d_hash_shift);
2d34: b9400821 ldr w1, [x1, #8]
2d38: f9400084 ldr x4, [x4]
2d38: R_AARCH64_LDST64_ABS_LO12_NC .data..read_mostly
2d3c: 1ac126c1 lsr w1, w22, w1
__READ_ONCE_SIZE;
2d40: f8617893 ldr x19, [x4, x1, lsl #3]
hlist_bl_for_each_entry_rcu(dentry, node, b, d_hash) {
2d44: f27ffa73 ands x19, x19, #0xfffffffffffffffe
2d48: 54000920 b.eq 2e6c <__d_lookup_rcu+0x15c> // b.none
2d4c: aa0003f5 mov x21, x0
if (dentry_cmp(dentry, str, hashlen_len(hashlen)) != 0)
2d50: d360feda lsr x26, x22, #32
2d54: a90363f7 stp x23, x24, [sp, #48]
2d58: aa0203f8 mov x24, x2
2d5c: d3608ad7 ubfx x23, x22, #32, #3
2d60: a90573fb stp x27, x28, [sp, #80]
2d64: 2a1603fc mov w28, w22
mask = bytemask_from_count(tcount);
2d68: 9280001b mov x27, #0xffffffffffffffff // #-1
2d6c: 14000003 b 2d78 <__d_lookup_rcu+0x68>
2d70: f9400273 ldr x19, [x19]
hlist_bl_for_each_entry_rcu(dentry, node, b, d_hash) {
2d74: b4000793 cbz x19, 2e64 <__d_lookup_rcu+0x154>
2d78: b85fc265 ldur w5, [x19, #-4]
smp_rmb();
2d7c: d50339bf dmb ishld
if (dentry->d_parent != parent)
2d80: f9400a64 ldr x4, [x19, #16]
2d84: d1002260 sub x0, x19, #0x8
2d88: eb0402bf cmp x21, x4
2d8c: 54ffff21 b.ne 2d70 <__d_lookup_rcu+0x60> // b.any
return ret & ~1;
2d90: 121f78b4 and w20, w5, #0xfffffffe
hlist_bl_for_each_entry_rcu(dentry, node, b, d_hash) {
2d94: aa0003e9 mov x9, x0
if (d_unhashed(dentry))
2d98: f9400664 ldr x4, [x19, #8]
2d9c: b4fffea4 cbz x4, 2d70 <__d_lookup_rcu+0x60>
if (unlikely(parent->d_flags & DCACHE_OP_COMPARE)) {
2da0: b94002a4 ldr w4, [x21]
2da4: 37080404 tbnz w4, #1, 2e24 <__d_lookup_rcu+0x114>
if (dentry->d_name.hash_len != hashlen)
2da8: f9401000 ldr x0, [x0, #32]
2dac: eb16001f cmp x0, x22
2db0: 54fffe01 b.ne 2d70 <__d_lookup_rcu+0x60> // b.any
2db4: f9401265 ldr x5, [x19, #32]
const unsigned char *cs = READ_ONCE(dentry->d_name.name);
2db8: 2a1a03e6 mov w6, w26
2dbc: cb050328 sub x8, x25, x5
2dc0: 14000006 b 2dd8 <__d_lookup_rcu+0xc8>
cs += sizeof(unsigned long);
2dc4: 910020a5 add x5, x5, #0x8
if (unlikely(a != b))
2dc8: eb07001f cmp x0, x7
2dcc: 54fffd21 b.ne 2d70 <__d_lookup_rcu+0x60> // b.any
if (!tcount)
2dd0: 710020c6 subs w6, w6, #0x8
2dd4: 54000160 b.eq 2e00 <__d_lookup_rcu+0xf0> // b.none
cs += sizeof(unsigned long);
2dd8: 8b0800a4 add x4, x5, x8
if (tcount < sizeof(unsigned long))
2ddc: 6b1700df cmp w6, w23
static inline unsigned long load_unaligned_zeropad(const void *addr)
{
unsigned long ret, offset;
/* Load word from unaligned pointer addr */
asm(
2de0: f9400087 ldr x7, [x4]
static __no_kasan_or_inline
unsigned long read_word_at_a_time(const void *addr)
{
kasan_check_read(addr, 1);
return *(unsigned long *)addr;
2de4: f94000a0 ldr x0, [x5]
2de8: 54fffee1 b.ne 2dc4 <__d_lookup_rcu+0xb4> // b.any
mask = bytemask_from_count(tcount);
2dec: 531d72e1 lsl w1, w23, #3
return unlikely(!!((a ^ b) & mask));
2df0: ca070000 eor x0, x0, x7
mask = bytemask_from_count(tcount);
2df4: 9ac12361 lsl x1, x27, x1
if (dentry_cmp(dentry, str, hashlen_len(hashlen)) != 0)
2df8: ea21001f bics xzr, x0, x1
2dfc: 54fffba1 b.ne 2d70 <__d_lookup_rcu+0x60> // b.any
*seqp = seq;
2e00: b9000314 str w20, [x24]
}
2e04: aa0903e0 mov x0, x9
2e08: a94153f3 ldp x19, x20, [sp, #16]
2e0c: a9425bf5 ldp x21, x22, [sp, #32]
return dentry;
2e10: a94363f7 ldp x23, x24, [sp, #48]
}
2e14: a9446bf9 ldp x25, x26, [sp, #64]
return dentry;
2e18: a94573fb ldp x27, x28, [sp, #80]
}
2e1c: a8c77bfd ldp x29, x30, [sp], #112
2e20: d65f03c0 ret
if (dentry->d_name.hash != hashlen_hash(hashlen))
2e24: b9402001 ldr w1, [x0, #32]
2e28: 6b01039f cmp w28, w1
2e2c: 54fffa21 b.ne 2d70 <__d_lookup_rcu+0x60> // b.any
tlen = dentry->d_name.len;
2e30: b9402401 ldr w1, [x0, #36]
tname = dentry->d_name.name;
2e34: f9401402 ldr x2, [x0, #40]
smp_rmb();
2e38: d50339bf dmb ishld
return unlikely(s->sequence != start);
2e3c: b85fc264 ldur w4, [x19, #-4]
if (read_seqcount_retry(&dentry->d_seq, seq)) {
2e40: 6b04029f cmp w20, w4
2e44: 54000221 b.ne 2e88 <__d_lookup_rcu+0x178> // b.any
if (parent->d_op->d_compare(dentry,
2e48: f94032a4 ldr x4, [x21, #96]
2e4c: a90627e3 stp x3, x9, [sp, #96]
2e50: f9400c84 ldr x4, [x4, #24]
2e54: d63f0080 blr x4
2e58: a94627e3 ldp x3, x9, [sp, #96]
2e5c: 34fffd20 cbz w0, 2e00 <__d_lookup_rcu+0xf0>
2e60: 17ffffc4 b 2d70 <__d_lookup_rcu+0x60>
2e64: a94363f7 ldp x23, x24, [sp, #48]
2e68: a94573fb ldp x27, x28, [sp, #80]
return NULL;
2e6c: d2800009 mov x9, #0x0 // #0
}
2e70: aa0903e0 mov x0, x9
2e74: a94153f3 ldp x19, x20, [sp, #16]
2e78: a9425bf5 ldp x21, x22, [sp, #32]
2e7c: a9446bf9 ldp x25, x26, [sp, #64]
2e80: a8c77bfd ldp x29, x30, [sp], #112
2e84: d65f03c0 ret
2e88: d503203f yield
__READ_ONCE_SIZE;
2e8c: b85fc265 ldur w5, [x19, #-4]
smp_rmb();
2e90: d50339bf dmb ishld
if (dentry->d_parent != parent)
2e94: f9400c01 ldr x1, [x0, #24]
return ret & ~1;
2e98: 121f78b4 and w20, w5, #0xfffffffe
2e9c: eb15003f cmp x1, x21
2ea0: 54fff681 b.ne 2d70 <__d_lookup_rcu+0x60> // b.any
2ea4: 17ffffbd b 2d98 <__d_lookup_rcu+0x88>
0000000000002ea8 <__d_lookup>:
{
2ea8: a9b97bfd stp x29, x30, [sp, #-112]!
return dentry_hashtable + (hash >> d_hash_shift);
2eac: 90000002 adrp x2, 0 <find_submount>
2eac: R_AARCH64_ADR_PREL_PG_HI21 .data..read_mostly
2eb0: 91000043 add x3, x2, #0x0
2eb0: R_AARCH64_ADD_ABS_LO12_NC .data..read_mostly
{
2eb4: 910003fd mov x29, sp
2eb8: a90573fb stp x27, x28, [sp, #80]
2ebc: aa0103fc mov x28, x1
2ec0: a90153f3 stp x19, x20, [sp, #16]
2ec4: a90363f7 stp x23, x24, [sp, #48]
2ec8: a9046bf9 stp x25, x26, [sp, #64]
2ecc: aa0003fa mov x26, x0
unsigned int hash = name->hash;
2ed0: b9400397 ldr w23, [x28]
return dentry_hashtable + (hash >> d_hash_shift);
2ed4: b9400860 ldr w0, [x3, #8]
2ed8: f9400041 ldr x1, [x2]
2ed8: R_AARCH64_LDST64_ABS_LO12_NC .data..read_mostly
2edc: 1ac026e0 lsr w0, w23, w0
2ee0: f8607833 ldr x19, [x1, x0, lsl #3]
hlist_bl_for_each_entry_rcu(dentry, node, b, d_hash) {
2ee4: f27ffa73 ands x19, x19, #0xfffffffffffffffe
2ee8: 54000320 b.eq 2f4c <__d_lookup+0xa4> // b.none
smp_store_release(&lock->locked, 0);
2eec: 5280001b mov w27, #0x0 // #0
mask = bytemask_from_count(tcount);
2ef0: 92800018 mov x24, #0xffffffffffffffff // #-1
2ef4: a9025bf5 stp x21, x22, [sp, #32]
2ef8: d2800016 mov x22, #0x0 // #0
2efc: 52800035 mov w21, #0x1 // #1
if (dentry->d_name.hash != hash)
2f00: b9401a62 ldr w2, [x19, #24]
2f04: d1002274 sub x20, x19, #0x8
2f08: 6b17005f cmp w2, w23
2f0c: 540001a1 b.ne 2f40 <__d_lookup+0x98> // b.any
2f10: 91014279 add x25, x19, #0x50
2f14: f9800331 prfm pstl1strm, [x25]
2f18: 885fff21 ldaxr w1, [x25]
2f1c: 4a160020 eor w0, w1, w22
2f20: 35000060 cbnz w0, 2f2c <__d_lookup+0x84>
2f24: 88007f35 stxr w0, w21, [x25]
2f28: 35ffff80 cbnz w0, 2f18 <__d_lookup+0x70>
2f2c: 35000521 cbnz w1, 2fd0 <__d_lookup+0x128>
if (dentry->d_parent != parent)
2f30: f9400e82 ldr x2, [x20, #24]
2f34: eb1a005f cmp x2, x26
2f38: 540001a0 b.eq 2f6c <__d_lookup+0xc4> // b.none
2f3c: 089fff3b stlrb w27, [x25]
2f40: f9400273 ldr x19, [x19]
hlist_bl_for_each_entry_rcu(dentry, node, b, d_hash) {
2f44: b5fffdf3 cbnz x19, 2f00 <__d_lookup+0x58>
2f48: a9425bf5 ldp x21, x22, [sp, #32]
struct dentry *found = NULL;
2f4c: d2800008 mov x8, #0x0 // #0
}
2f50: aa0803e0 mov x0, x8
2f54: a94153f3 ldp x19, x20, [sp, #16]
2f58: a94363f7 ldp x23, x24, [sp, #48]
2f5c: a9446bf9 ldp x25, x26, [sp, #64]
2f60: a94573fb ldp x27, x28, [sp, #80]
2f64: a8c77bfd ldp x29, x30, [sp], #112
2f68: d65f03c0 ret
if (d_unhashed(dentry))
2f6c: f9400660 ldr x0, [x19, #8]
2f70: b4fffe60 cbz x0, 2f3c <__d_lookup+0x94>
if (likely(!(parent->d_flags & DCACHE_OP_COMPARE))) {
2f74: b9400340 ldr w0, [x26]
hlist_bl_for_each_entry_rcu(dentry, node, b, d_hash) {
2f78: aa1403e8 mov x8, x20
if (likely(!(parent->d_flags & DCACHE_OP_COMPARE))) {
2f7c: b9402681 ldr w1, [x20, #36]
2f80: 370802e0 tbnz w0, #1, 2fdc <__d_lookup+0x134>
if (dentry->d_name.len != name->len)
2f84: b9400784 ldr w4, [x28, #4]
2f88: 6b04003f cmp w1, w4
2f8c: 54fffd81 b.ne 2f3c <__d_lookup+0x94> // b.any
return dentry_cmp(dentry, name->name, name->len) == 0;
2f90: f9400787 ldr x7, [x28, #8]
static inline int dentry_string_cmp(const unsigned char *cs, const unsigned
char *ct, unsigned tcount)
2f94: 12000881 and w1, w4, #0x7
2f98: f9401265 ldr x5, [x19, #32]
2f9c: cb0500e7 sub x7, x7, x5
2fa0: 14000003 b 2fac <__d_lookup+0x104>
if (!tcount)
2fa4: 71002084 subs w4, w4, #0x8
2fa8: 54000300 b.eq 3008 <__d_lookup+0x160> // b.none
cs += sizeof(unsigned long);
2fac: 8b0700a2 add x2, x5, x7
if (tcount < sizeof(unsigned long))
2fb0: 6b04003f cmp w1, w4
2fb4: f9400046 ldr x6, [x2]
return *(unsigned long *)addr;
2fb8: f94000a0 ldr x0, [x5]
2fbc: 54000340 b.eq 3024 <__d_lookup+0x17c> // b.none
cs += sizeof(unsigned long);
2fc0: 910020a5 add x5, x5, #0x8
if (unlikely(a != b))
2fc4: eb06001f cmp x0, x6
2fc8: 54fffee0 b.eq 2fa4 <__d_lookup+0xfc> // b.none
2fcc: 17ffffdc b 2f3c <__d_lookup+0x94>
queued_spin_lock_slowpath(lock, val);
2fd0: aa1903e0 mov x0, x25
2fd4: 94000000 bl 0 <queued_spin_lock_slowpath>
2fd4: R_AARCH64_CALL26 queued_spin_lock_slowpath
2fd8: 17ffffd6 b 2f30 <__d_lookup+0x88>
return parent->d_op->d_compare(dentry,
2fdc: f9403340 ldr x0, [x26, #96]
2fe0: aa1c03e3 mov x3, x28
2fe4: f9401682 ldr x2, [x20, #40]
2fe8: f90037f4 str x20, [sp, #104]
2fec: f9400c04 ldr x4, [x0, #24]
2ff0: aa1403e0 mov x0, x20
2ff4: d63f0080 blr x4
name) == 0;
2ff8: 7100001f cmp w0, #0x0
2ffc: 1a9f17e0 cset w0, eq // eq = none
3000: f94037e8 ldr x8, [sp, #104]
if (!d_same_name(dentry, parent, name))
3004: 34fff9c0 cbz w0, 2f3c <__d_lookup+0x94>
dentry->d_lockref.count++;
3008: b9405e80 ldr w0, [x20, #92]
smp_store_release(&lock->locked, 0);
300c: 52800001 mov w1, #0x0 // #0
3010: 11000400 add w0, w0, #0x1
3014: b9005e80 str w0, [x20, #92]
3018: 089fff21 stlrb w1, [x25]
}
301c: a9425bf5 ldp x21, x22, [sp, #32]
3020: 17ffffcc b 2f50 <__d_lookup+0xa8>
mask = bytemask_from_count(tcount);
3024: 531d7021 lsl w1, w1, #3
return unlikely(!!((a ^ b) & mask));
3028: ca060000 eor x0, x0, x6
mask = bytemask_from_count(tcount);
302c: 9ac12301 lsl x1, x24, x1
3030: ea21001f bics xzr, x0, x1
3034: 1a9f17e0 cset w0, eq // eq = none
if (!d_same_name(dentry, parent, name))
3038: 34fff820 cbz w0, 2f3c <__d_lookup+0x94>
303c: 17fffff3 b 3008 <__d_lookup+0x160>
0000000000003040 <d_lookup>:
{
3040: a9bd7bfd stp x29, x30, [sp, #-48]!
3044: 910003fd mov x29, sp
3048: a90153f3 stp x19, x20, [sp, #16]
304c: 90000013 adrp x19, 0 <find_submount>
304c: R_AARCH64_ADR_PREL_PG_HI21 .data..cacheline_aligned
3050: aa0103f4 mov x20, x1
3054: 91000273 add x19, x19, #0x0
3054: R_AARCH64_ADD_ABS_LO12_NC .data..cacheline_aligned
3058: a9025bf5 stp x21, x22, [sp, #32]
305c: aa0003f5 mov x21, x0
__READ_ONCE_SIZE;
3060: b9400276 ldr w22, [x19]
if (unlikely(ret & 1)) {
3064: 370001d6 tbnz w22, #0, 309c <d_lookup+0x5c>
smp_rmb();
3068: d50339bf dmb ishld
dentry = __d_lookup(parent, name);
306c: aa1403e1 mov x1, x20
3070: aa1503e0 mov x0, x21
3074: 94000000 bl 2ea8 <__d_lookup>
3074: R_AARCH64_CALL26 __d_lookup
if (dentry)
3078: b50000a0 cbnz x0, 308c <d_lookup+0x4c>
smp_rmb();
307c: d50339bf dmb ishld
} while (read_seqretry(&rename_lock, seq));
3080: b9400261 ldr w1, [x19]
3084: 6b16003f cmp w1, w22
3088: 54fffec1 b.ne 3060 <d_lookup+0x20> // b.any
}
308c: a94153f3 ldp x19, x20, [sp, #16]
3090: a9425bf5 ldp x21, x22, [sp, #32]
3094: a8c37bfd ldp x29, x30, [sp], #48
3098: d65f03c0 ret
309c: d503203f yield
30a0: 17fffff0 b 3060 <d_lookup+0x20>
30a4: d503201f nop
Powered by blists - more mailing lists