linux-kernel - Re: [dm-crypt] cryptsetup not working under 3.6

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 29 Oct 2012 20:45:38 +0000
From:	Tvrtko Ursulin <tvrtko@...ulin.net>
To:	Milan Broz <gmazyland@...il.com>
CC:	dm-crypt@...ut.de, linux-kernel@...r.kernel.org,
	okozina@...hat.com, linux-rt-users@...r.kernel.org
Subject: Re: [dm-crypt] cryptsetup not working under 3.6 - RT patch set seem
 to break it

On 29/10/12 20:14, Tvrtko Ursulin wrote:
> On 29/10/12 19:47, Milan Broz wrote:
>> On 10/29/2012 08:31 PM, Tvrtko Ursulin wrote:
>>> Just tried 3.6.4 and it is still broken. Is there anything else I could
>>> try to debug this?
>>
>> See response from response from Ondra on dmcrypt list - your kernel
>> config
>> is perhaps broken.
>
> I don't think that reached me - perhaps I was not copied.
>
> Weird yes, I mean no one can spot anything bad with my config...
>
>> Please use fresh checkout, run make oldconfig and try again.
>> (Many people use 3.6 already and there is no other report.)
>
> ... and I just used it today to configure a fresh 3.6.4 (plus -rt10
> patches) without a problem.
>
> Unless RT patchset is the culprit. Hm.. that would be unexpected, but I
> guess it is worth a shot. I'll let you know what happens without -rt.

Ha, this is exciting, vanilla 3.6.4 works, with -rt10 patch it does not.

Copying in linux-rt-users.

To recap, 3.6 RT seems to break dm-crypt. Try it like this:

cryptsetup --debug --key-file=- luksOpen /some/luks/formatted/file any-label

And kernel will fail like this:

Kernel says this:
      device-mapper: table: 252:1: crypt: Error allocating crypto tfm
      device-mapper: ioctl: error adding target to table

I think crypto modules are not getting auto loaded for some reason, but 
even if I load them manually before hand and still breaks in the same way.

Maybe someone will have a "ah yes" moment? Here is the diff between 
working (vanilla 3.6.4) and non-working (3.6.4-rt10) configs:

--- 1	2012-10-29 20:37:49.472306922 +0000
+++ 2	2012-10-29 20:37:52.319732779 +0000
@@ -892,7 +892,6 @@
  # CONFIG_DEBUG_SECTION_MISMATCH is not set
  # CONFIG_DEBUG_SET_MODULE_RONX is not set
  # CONFIG_DEBUG_SG is not set
-# CONFIG_DEBUG_SHIRQ is not set
  # CONFIG_DEBUG_SPINLOCK is not set
  # CONFIG_DEBUG_STACKOVERFLOW is not set
  # CONFIG_DEBUG_STACK_USAGE is not set
@@ -1810,7 +1809,7 @@
  CONFIG_MTRR_SANITIZER=y
  CONFIG_MTRR=y
  # CONFIG_MULTICORE_RAID456 is not set
-CONFIG_MUTEX_SPIN_ON_OWNER=y
+# CONFIG_MUTEX_SPIN_ON_OWNER is not set
  # CONFIG_MWAVE is not set
  CONFIG_NAMESPACES=y
  CONFIG_NATIONAL_PHY=y
@@ -1842,7 +1841,6 @@
  CONFIG_NET_CLS_TCINDEX=m
  CONFIG_NET_CLS_U32=m
  CONFIG_NET_CLS=y
-# CONFIG_NETCONSOLE is not set
  CONFIG_NET_CORE=y
  CONFIG_NETDEVICES=y
  # CONFIG_NET_DSA is not set
@@ -2260,9 +2258,13 @@
  CONFIG_PPP=y
  # CONFIG_PPS is not set
  CONFIG_PREEMPT_COUNT=y
+# CONFIG_PREEMPT__LL is not set
  # CONFIG_PREEMPT_NONE is not set
  CONFIG_PREEMPT_NOTIFIERS=y
  CONFIG_PREEMPT_RCU=y
+CONFIG_PREEMPT_RT_BASE=y
+# CONFIG_PREEMPT_RTB is not set
+CONFIG_PREEMPT_RT_FULL=y
  # CONFIG_PREEMPT_VOLUNTARY is not set
  CONFIG_PREEMPT=y
  CONFIG_PREVENT_FIRMWARE_BUILD=y
@@ -2368,11 +2370,10 @@
  CONFIG_RTC_INTF_PROC=y
  CONFIG_RTC_INTF_SYSFS=y
  CONFIG_RTC_LIB=y
-# CONFIG_RT_GROUP_SCHED is not set
  CONFIG_RT_MUTEXES=y
  # CONFIG_RT_MUTEX_TESTER is not set
-# CONFIG_RWSEM_GENERIC_SPINLOCK is not set
-CONFIG_RWSEM_XCHGADD_ALGORITHM=y
+CONFIG_RWSEM_GENERIC_SPINLOCK=y
+# CONFIG_RWSEM_XCHGADD_ALGORITHM is not set
  # CONFIG_SAMPLES is not set
  # CONFIG_SATA_ACARD_AHCI is not set
  CONFIG_SATA_AHCI=m
@@ -2851,7 +2852,6 @@
  CONFIG_TRACE_IRQFLAGS_SUPPORT=y
  # CONFIG_TRACE_SINK is not set
  CONFIG_TRACING_SUPPORT=y
-# CONFIG_TRANSPARENT_HUGEPAGE is not set
  CONFIG_TREE_PREEMPT_RCU=y
  # CONFIG_TREE_RCU_TRACE is not set
  CONFIG_TUN=y
@@ -2867,7 +2867,6 @@
  CONFIG_UID16=y
  # CONFIG_UIO is not set
  # CONFIG_ULTRIX_PARTITION is not set
-CONFIG_UNINLINE_SPIN_UNLOCK=y
  CONFIG_UNIX98_PTYS=y
  CONFIG_UNIX_DIAG=y
  # CONFIG_UNIXWARE_DISKLABEL is not set

No crypto related differences.

Tvrtko




--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/