| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20150814010903.GA19621@codemonkey.org.uk>
Date: Thu, 13 Aug 2015 21:09:03 -0400
From: Dave Jones <davej@...emonkey.org.uk>
To: daniel.vetter@...el.com
Cc: Linux Kernel <linux-kernel@...r.kernel.org>,
intel-gfx@...ts.freedesktop.org, dri-devel@...ts.freedesktop.org
Subject: i915/kasan: out of bounds access in i915_cmd_parser_init_ring
I finally got around to playing with kasan. It didn't end well.
I added some debugging to validate_cmds_sorted to print out the table
sizes right before the stack traces.
Dave
validate_cmds_sorted: table:ffffffffa1fb4220 cmd_table_count:3
validate_cmds_sorted: table:ffffffffa1fb4220 table->count:12
validate_cmds_sorted: table:ffffffffa1fb4230 table->count:20
validate_cmds_sorted: table:ffffffffa1fb4230 table->count:20
validate_cmds_sorted: table:ffffffffa1fb4240 table->count:18
validate_cmds_sorted: table:ffffffffa1fb41e0 cmd_table_count:2
validate_cmds_sorted: table:ffffffffa1fb41e0 table->count:12
validate_cmds_sorted: table:ffffffffa1fb41f0 table->count:7
validate_cmds_sorted: table:ffffffffa1fb4100 cmd_table_count:3
validate_cmds_sorted: table:ffffffffa1fb4100 table->count:12
validate_cmds_sorted: table:ffffffffa1fb4110 table->count:6
==================================================================
BUG: KASan: out of bounds access in i915_cmd_parser_init_ring+0x66b/0x760 at addr ffffffffa1fb4374
Read of size 4 by task swapper/0/1
Address belongs to variable hsw_blt_cmds+0xb4/0xe0
CPU: 1 PID: 1 Comm: swapper/0 Not tainted 4.2.0-rc6-firewall+ #4
0000000000000002 ffff8801d6baf478 ffffffffa1c0b4fb 0000000000000032
ffff8801d6baf510 ffff8801d6baf4f8 ffffffffa123198f ffff8801d6baf5a8
ffffed003ad75e9b 0000000000000246 ffffffffa1fb4110 0000000010000000
Call Trace:
[<ffffffffa1c0b4fb>] dump_stack+0x4f/0x7b
[<ffffffffa123198f>] kasan_report_error+0x3bf/0x3f0
[<ffffffffa1231a9b>] kasan_report+0x3b/0x40
[<ffffffffa166d7ab>] ? i915_cmd_parser_init_ring+0x66b/0x760
[<ffffffffa1230e06>] __asan_load4+0x66/0xa0
[<ffffffffa166d7ab>] i915_cmd_parser_init_ring+0x66b/0x760
[<ffffffffa16c0459>] intel_init_ring_buffer+0x449/0x680
[<ffffffffa16c61de>] intel_init_blt_ring_buffer+0x38e/0x520
[<ffffffffa1687744>] i915_gem_init_rings+0x74/0x220
[<ffffffffa168c292>] i915_gem_init+0x1e2/0x320
[<ffffffffa1768ff1>] i915_driver_load+0x1571/0x2310
[<ffffffffa11090ee>] ? debug_lockdep_rcu_enabled+0x4e/0x70
[<ffffffffa10e98ce>] ? __lock_acquire+0x97e/0x2710
[<ffffffffa14c8b87>] ? debug_smp_processor_id+0x17/0x20
[<ffffffffa10e8f50>] ? debug_show_all_locks+0x280/0x280
[<ffffffffa1c1279b>] ? __mutex_unlock_slowpath+0x11b/0x1e0
[<ffffffffa10e6bb2>] ? trace_hardirqs_on_caller+0x192/0x2a0
[<ffffffffa1767a80>] ? i915_getparam+0x390/0x390
[<ffffffffa10e69f4>] ? mark_held_locks+0xa4/0xd0
[<ffffffffa1c19538>] ? _raw_spin_unlock_irqrestore+0x58/0x70
[<ffffffffa10e6bb2>] ? trace_hardirqs_on_caller+0x192/0x2a0
[<ffffffffa10b8431>] ? preempt_count_sub+0xc1/0x130
[<ffffffffa1c19523>] ? _raw_spin_unlock_irqrestore+0x43/0x70
[<ffffffffa1612811>] drm_dev_register+0xd1/0x170
[<ffffffffa16166a1>] drm_get_pci_dev+0xf1/0x350
[<ffffffffa10e6bb2>] ? trace_hardirqs_on_caller+0x192/0x2a0
[<ffffffffa163df43>] i915_pci_probe+0x83/0xb0
[<ffffffffa14f522f>] pci_device_probe+0xcf/0x130
[<ffffffffa17756f1>] driver_probe_device+0x1e1/0x410
[<ffffffffa1775920>] ? driver_probe_device+0x410/0x410
[<ffffffffa1775920>] ? driver_probe_device+0x410/0x410
[<ffffffffa17759f6>] __driver_attach+0xd6/0xe0
[<ffffffffa17725e5>] bus_for_each_dev+0xf5/0x160
[<ffffffffa17724f0>] ? bus_remove_file+0xa0/0xa0
[<ffffffffa10f2ee4>] ? do_raw_spin_unlock+0xa4/0x140
[<ffffffffa10b8431>] ? preempt_count_sub+0xc1/0x130
[<ffffffffa1775b80>] driver_attach+0x30/0x40
[<ffffffffa17738e1>] bus_add_driver+0x2b1/0x330
[<ffffffffa17763ce>] driver_register+0xde/0x1b0
[<ffffffffa14f579c>] __pci_register_driver+0xbc/0xd0
[<ffffffffa1616ae7>] drm_pci_init+0x1e7/0x210
[<ffffffffa2975265>] ? do_one_initcall+0x108/0x242
[<ffffffffa2975265>] ? do_one_initcall+0x108/0x242
[<ffffffffa29b732f>] i915_init+0xdb/0xe3
[<ffffffffa29b7254>] ? mipi_dsi_bus_init+0x12/0x12
[<ffffffffa2975384>] do_one_initcall+0x227/0x242
[<ffffffffa297515d>] ? start_kernel+0x4ed/0x4ed
[<ffffffffa10a38db>] ? parse_args+0x5b/0x4f0
[<ffffffffa297562f>] kernel_init_freeable+0x290/0x321
[<ffffffffa1c03ce0>] ? rest_init+0x150/0x150
[<ffffffffa1c03cf4>] kernel_init+0x14/0x100
[<ffffffffa1c03ce0>] ? rest_init+0x150/0x150
[<ffffffffa1c1a3bf>] ret_from_fork+0x3f/0x70
[<ffffffffa1c03ce0>] ? rest_init+0x150/0x150
Memory state around the buggy address:
ffffffffa1fb4200: fa fa fa fa 00 00 00 00 00 00 fa fa fa fa fa fa
ffffffffa1fb4280: 00 00 00 00 fa fa fa fa 00 00 00 00 00 00 00 00
>ffffffffa1fb4300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fa fa
^
ffffffffa1fb4380: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
ffffffffa1fb4400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
==================================================================
==================================================================
BUG: KASan: out of bounds access in i915_cmd_parser_init_ring+0x67e/0x760 at addr ffffffffa1fb4378
Read of size 4 by task swapper/0/1
Address belongs to variable hsw_blt_cmds+0xb8/0xe0
CPU: 1 PID: 1 Comm: swapper/0 Not tainted 4.2.0-rc6-firewall+ #4
0000000000000002 ffff8801d6baf478 ffffffffa1c0b4fb 0000000000000032
ffff8801d6baf510 ffff8801d6baf4f8 ffffffffa123198f 0000000000000010
ffffed0000000000 0000000000000246 fffffbfff43f686e 6666662010000000
Call Trace:
[<ffffffffa1c0b4fb>] dump_stack+0x4f/0x7b
[<ffffffffa123198f>] kasan_report_error+0x3bf/0x3f0
[<ffffffffa1231a9b>] kasan_report+0x3b/0x40
[<ffffffffa166d7be>] ? i915_cmd_parser_init_ring+0x67e/0x760
[<ffffffffa1230e06>] __asan_load4+0x66/0xa0
[<ffffffffa166d7be>] i915_cmd_parser_init_ring+0x67e/0x760
[<ffffffffa16c0459>] intel_init_ring_buffer+0x449/0x680
[<ffffffffa16c61de>] intel_init_blt_ring_buffer+0x38e/0x520
[<ffffffffa1687744>] i915_gem_init_rings+0x74/0x220
[<ffffffffa168c292>] i915_gem_init+0x1e2/0x320
[<ffffffffa1768ff1>] i915_driver_load+0x1571/0x2310
[<ffffffffa11090ee>] ? debug_lockdep_rcu_enabled+0x4e/0x70
[<ffffffffa10e98ce>] ? __lock_acquire+0x97e/0x2710
[<ffffffffa14c8b87>] ? debug_smp_processor_id+0x17/0x20
[<ffffffffa10e8f50>] ? debug_show_all_locks+0x280/0x280
[<ffffffffa1c1279b>] ? __mutex_unlock_slowpath+0x11b/0x1e0
[<ffffffffa10e6bb2>] ? trace_hardirqs_on_caller+0x192/0x2a0
[<ffffffffa1767a80>] ? i915_getparam+0x390/0x390
[<ffffffffa10e69f4>] ? mark_held_locks+0xa4/0xd0
[<ffffffffa1c19538>] ? _raw_spin_unlock_irqrestore+0x58/0x70
[<ffffffffa10e6bb2>] ? trace_hardirqs_on_caller+0x192/0x2a0
[<ffffffffa10b8431>] ? preempt_count_sub+0xc1/0x130
[<ffffffffa1c19523>] ? _raw_spin_unlock_irqrestore+0x43/0x70
[<ffffffffa1612811>] drm_dev_register+0xd1/0x170
[<ffffffffa16166a1>] drm_get_pci_dev+0xf1/0x350
[<ffffffffa10e6bb2>] ? trace_hardirqs_on_caller+0x192/0x2a0
[<ffffffffa163df43>] i915_pci_probe+0x83/0xb0
[<ffffffffa14f522f>] pci_device_probe+0xcf/0x130
[<ffffffffa17756f1>] driver_probe_device+0x1e1/0x410
[<ffffffffa1775920>] ? driver_probe_device+0x410/0x410
[<ffffffffa1775920>] ? driver_probe_device+0x410/0x410
[<ffffffffa17759f6>] __driver_attach+0xd6/0xe0
[<ffffffffa17725e5>] bus_for_each_dev+0xf5/0x160
[<ffffffffa17724f0>] ? bus_remove_file+0xa0/0xa0
[<ffffffffa10f2ee4>] ? do_raw_spin_unlock+0xa4/0x140
[<ffffffffa10b8431>] ? preempt_count_sub+0xc1/0x130
[<ffffffffa1775b80>] driver_attach+0x30/0x40
[<ffffffffa17738e1>] bus_add_driver+0x2b1/0x330
[<ffffffffa17763ce>] driver_register+0xde/0x1b0
[<ffffffffa14f579c>] __pci_register_driver+0xbc/0xd0
[<ffffffffa1616ae7>] drm_pci_init+0x1e7/0x210
[<ffffffffa2975265>] ? do_one_initcall+0x108/0x242
[<ffffffffa2975265>] ? do_one_initcall+0x108/0x242
[<ffffffffa29b732f>] i915_init+0xdb/0xe3
[<ffffffffa29b7254>] ? mipi_dsi_bus_init+0x12/0x12
[<ffffffffa2975384>] do_one_initcall+0x227/0x242
[<ffffffffa297515d>] ? start_kernel+0x4ed/0x4ed
[<ffffffffa10a38db>] ? parse_args+0x5b/0x4f0
[<ffffffffa297562f>] kernel_init_freeable+0x290/0x321
[<ffffffffa1c03ce0>] ? rest_init+0x150/0x150
[<ffffffffa1c03cf4>] kernel_init+0x14/0x100
[<ffffffffa1c03ce0>] ? rest_init+0x150/0x150
[<ffffffffa1c1a3bf>] ret_from_fork+0x3f/0x70
[<ffffffffa1c03ce0>] ? rest_init+0x150/0x150
Memory state around the buggy address:
ffffffffa1fb4200: fa fa fa fa 00 00 00 00 00 00 fa fa fa fa fa fa
ffffffffa1fb4280: 00 00 00 00 fa fa fa fa 00 00 00 00 00 00 00 00
>ffffffffa1fb4300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fa fa
^
ffffffffa1fb4380: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
ffffffffa1fb4400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
==================================================================
validate_cmds_sorted: table:ffffffffa1fb4120 table->count:2
==================================================================
BUG: KASan: out of bounds access in i915_cmd_parser_init_ring+0x6eb/0x760 at addr ffffffffa1fb4374
Read of size 4 by task swapper/0/1
Address belongs to variable hsw_blt_cmds+0xb4/0xe0
CPU: 1 PID: 1 Comm: swapper/0 Not tainted 4.2.0-rc6-firewall+ #4
0000000000000002 ffff8801d6baf478 ffffffffa1c0b4fb 0000000000000032
ffff8801d6baf510 ffff8801d6baf4f8 ffffffffa123198f 0000000000000010
ffffed003ad75e9b 0000000000000246 ffffffffa1fb4120 0000000000000003
Call Trace:
[<ffffffffa1c0b4fb>] dump_stack+0x4f/0x7b
[<ffffffffa123198f>] kasan_report_error+0x3bf/0x3f0
[<ffffffffa1231a9b>] kasan_report+0x3b/0x40
[<ffffffffa166d82b>] ? i915_cmd_parser_init_ring+0x6eb/0x760
[<ffffffffa1230e06>] __asan_load4+0x66/0xa0
[<ffffffffa166d82b>] i915_cmd_parser_init_ring+0x6eb/0x760
[<ffffffffa16c0459>] intel_init_ring_buffer+0x449/0x680
[<ffffffffa16c61de>] intel_init_blt_ring_buffer+0x38e/0x520
[<ffffffffa1687744>] i915_gem_init_rings+0x74/0x220
[<ffffffffa168c292>] i915_gem_init+0x1e2/0x320
[<ffffffffa1768ff1>] i915_driver_load+0x1571/0x2310
[<ffffffffa11090ee>] ? debug_lockdep_rcu_enabled+0x4e/0x70
[<ffffffffa10e98ce>] ? __lock_acquire+0x97e/0x2710
[<ffffffffa14c8b87>] ? debug_smp_processor_id+0x17/0x20
[<ffffffffa10e8f50>] ? debug_show_all_locks+0x280/0x280
[<ffffffffa1c1279b>] ? __mutex_unlock_slowpath+0x11b/0x1e0
[<ffffffffa10e6bb2>] ? trace_hardirqs_on_caller+0x192/0x2a0
[<ffffffffa1767a80>] ? i915_getparam+0x390/0x390
[<ffffffffa10e69f4>] ? mark_held_locks+0xa4/0xd0
[<ffffffffa1c19538>] ? _raw_spin_unlock_irqrestore+0x58/0x70
[<ffffffffa10e6bb2>] ? trace_hardirqs_on_caller+0x192/0x2a0
[<ffffffffa10b8431>] ? preempt_count_sub+0xc1/0x130
[<ffffffffa1c19523>] ? _raw_spin_unlock_irqrestore+0x43/0x70
[<ffffffffa1612811>] drm_dev_register+0xd1/0x170
[<ffffffffa16166a1>] drm_get_pci_dev+0xf1/0x350
[<ffffffffa10e6bb2>] ? trace_hardirqs_on_caller+0x192/0x2a0
[<ffffffffa163df43>] i915_pci_probe+0x83/0xb0
[<ffffffffa14f522f>] pci_device_probe+0xcf/0x130
[<ffffffffa17756f1>] driver_probe_device+0x1e1/0x410
[<ffffffffa1775920>] ? driver_probe_device+0x410/0x410
[<ffffffffa1775920>] ? driver_probe_device+0x410/0x410
[<ffffffffa17759f6>] __driver_attach+0xd6/0xe0
[<ffffffffa17725e5>] bus_for_each_dev+0xf5/0x160
[<ffffffffa17724f0>] ? bus_remove_file+0xa0/0xa0
[<ffffffffa10f2ee4>] ? do_raw_spin_unlock+0xa4/0x140
[<ffffffffa10b8431>] ? preempt_count_sub+0xc1/0x130
[<ffffffffa1775b80>] driver_attach+0x30/0x40
[<ffffffffa17738e1>] bus_add_driver+0x2b1/0x330
[<ffffffffa17763ce>] driver_register+0xde/0x1b0
[<ffffffffa14f579c>] __pci_register_driver+0xbc/0xd0
[<ffffffffa1616ae7>] drm_pci_init+0x1e7/0x210
[<ffffffffa2975265>] ? do_one_initcall+0x108/0x242
[<ffffffffa2975265>] ? do_one_initcall+0x108/0x242
[<ffffffffa29b732f>] i915_init+0xdb/0xe3
[<ffffffffa29b7254>] ? mipi_dsi_bus_init+0x12/0x12
[<ffffffffa2975384>] do_one_initcall+0x227/0x242
[<ffffffffa297515d>] ? start_kernel+0x4ed/0x4ed
[<ffffffffa10a38db>] ? parse_args+0x5b/0x4f0
[<ffffffffa297562f>] kernel_init_freeable+0x290/0x321
[<ffffffffa1c03ce0>] ? rest_init+0x150/0x150
[<ffffffffa1c03cf4>] kernel_init+0x14/0x100
[<ffffffffa1c03ce0>] ? rest_init+0x150/0x150
[<ffffffffa1c1a3bf>] ret_from_fork+0x3f/0x70
[<ffffffffa1c03ce0>] ? rest_init+0x150/0x150
Memory state around the buggy address:
ffffffffa1fb4200: fa fa fa fa 00 00 00 00 00 00 fa fa fa fa fa fa
ffffffffa1fb4280: 00 00 00 00 fa fa fa fa 00 00 00 00 00 00 00 00
>ffffffffa1fb4300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fa fa
^
ffffffffa1fb4380: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
ffffffffa1fb4400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
==================================================================
==================================================================
BUG: KASan: out of bounds access in i915_cmd_parser_init_ring+0x6fb/0x760 at addr ffffffffa1fb4378
Read of size 4 by task swapper/0/1
Address belongs to variable hsw_blt_cmds+0xb8/0xe0
CPU: 1 PID: 1 Comm: swapper/0 Not tainted 4.2.0-rc6-firewall+ #4
0000000000000002 ffff8801d6baf478 ffffffffa1c0b4fb 0000000000000032
ffff8801d6baf510 ffff8801d6baf4f8 ffffffffa123198f 0000000000000010
ffffed0000000000 0000000000000246 fffffbfff43f686e 6666662000000003
Call Trace:
[<ffffffffa1c0b4fb>] dump_stack+0x4f/0x7b
[<ffffffffa123198f>] kasan_report_error+0x3bf/0x3f0
[<ffffffffa1231a9b>] kasan_report+0x3b/0x40
[<ffffffffa166d83b>] ? i915_cmd_parser_init_ring+0x6fb/0x760
[<ffffffffa1230e06>] __asan_load4+0x66/0xa0
[<ffffffffa166d83b>] i915_cmd_parser_init_ring+0x6fb/0x760
[<ffffffffa16c0459>] intel_init_ring_buffer+0x449/0x680
[<ffffffffa16c61de>] intel_init_blt_ring_buffer+0x38e/0x520
[<ffffffffa1687744>] i915_gem_init_rings+0x74/0x220
[<ffffffffa168c292>] i915_gem_init+0x1e2/0x320
[<ffffffffa1768ff1>] i915_driver_load+0x1571/0x2310
[<ffffffffa11090ee>] ? debug_lockdep_rcu_enabled+0x4e/0x70
[<ffffffffa10e98ce>] ? __lock_acquire+0x97e/0x2710
[<ffffffffa14c8b87>] ? debug_smp_processor_id+0x17/0x20
[<ffffffffa10e8f50>] ? debug_show_all_locks+0x280/0x280
[<ffffffffa1c1279b>] ? __mutex_unlock_slowpath+0x11b/0x1e0
[<ffffffffa10e6bb2>] ? trace_hardirqs_on_caller+0x192/0x2a0
[<ffffffffa1767a80>] ? i915_getparam+0x390/0x390
[<ffffffffa10e69f4>] ? mark_held_locks+0xa4/0xd0
[<ffffffffa1c19538>] ? _raw_spin_unlock_irqrestore+0x58/0x70
[<ffffffffa10e6bb2>] ? trace_hardirqs_on_caller+0x192/0x2a0
[<ffffffffa10b8431>] ? preempt_count_sub+0xc1/0x130
[<ffffffffa1c19523>] ? _raw_spin_unlock_irqrestore+0x43/0x70
[<ffffffffa1612811>] drm_dev_register+0xd1/0x170
[<ffffffffa16166a1>] drm_get_pci_dev+0xf1/0x350
[<ffffffffa10e6bb2>] ? trace_hardirqs_on_caller+0x192/0x2a0
[<ffffffffa163df43>] i915_pci_probe+0x83/0xb0
[<ffffffffa14f522f>] pci_device_probe+0xcf/0x130
[<ffffffffa17756f1>] driver_probe_device+0x1e1/0x410
[<ffffffffa1775920>] ? driver_probe_device+0x410/0x410
[<ffffffffa1775920>] ? driver_probe_device+0x410/0x410
[<ffffffffa17759f6>] __driver_attach+0xd6/0xe0
[<ffffffffa17725e5>] bus_for_each_dev+0xf5/0x160
[<ffffffffa17724f0>] ? bus_remove_file+0xa0/0xa0
[<ffffffffa10f2ee4>] ? do_raw_spin_unlock+0xa4/0x140
[<ffffffffa10b8431>] ? preempt_count_sub+0xc1/0x130
[<ffffffffa1775b80>] driver_attach+0x30/0x40
[<ffffffffa17738e1>] bus_add_driver+0x2b1/0x330
[<ffffffffa17763ce>] driver_register+0xde/0x1b0
[<ffffffffa14f579c>] __pci_register_driver+0xbc/0xd0
[<ffffffffa1616ae7>] drm_pci_init+0x1e7/0x210
[<ffffffffa2975265>] ? do_one_initcall+0x108/0x242
[<ffffffffa2975265>] ? do_one_initcall+0x108/0x242
[<ffffffffa29b732f>] i915_init+0xdb/0xe3
[<ffffffffa29b7254>] ? mipi_dsi_bus_init+0x12/0x12
[<ffffffffa2975384>] do_one_initcall+0x227/0x242
[<ffffffffa297515d>] ? start_kernel+0x4ed/0x4ed
[<ffffffffa10a38db>] ? parse_args+0x5b/0x4f0
[<ffffffffa297562f>] kernel_init_freeable+0x290/0x321
[<ffffffffa1c03ce0>] ? rest_init+0x150/0x150
[<ffffffffa1c03cf4>] kernel_init+0x14/0x100
[<ffffffffa1c03ce0>] ? rest_init+0x150/0x150
[<ffffffffa1c1a3bf>] ret_from_fork+0x3f/0x70
[<ffffffffa1c03ce0>] ? rest_init+0x150/0x150
Memory state around the buggy address:
ffffffffa1fb4200: fa fa fa fa 00 00 00 00 00 00 fa fa fa fa fa fa
ffffffffa1fb4280: 00 00 00 00 fa fa fa fa 00 00 00 00 00 00 00 00
>ffffffffa1fb4300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fa fa
^
ffffffffa1fb4380: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
ffffffffa1fb4400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
==================================================================
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists