linux-kernel - Re: [dm-crypt] cryptsetup not working under 3.6

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 30 Oct 2012 00:51:03 +0100 (CET)
From:	Thomas Gleixner <tglx@...utronix.de>
To:	Tvrtko Ursulin <tvrtko@...ulin.net>
cc:	Milan Broz <gmazyland@...il.com>, dm-crypt@...ut.de,
	linux-kernel@...r.kernel.org, okozina@...hat.com,
	linux-rt-users@...r.kernel.org
Subject: Re: [dm-crypt] cryptsetup not working under 3.6 - RT patch set seem
 to break it

On Mon, 29 Oct 2012, Tvrtko Ursulin wrote:
> On 29/10/12 20:14, Tvrtko Ursulin wrote:
> > Unless RT patchset is the culprit. Hm.. that would be unexpected, but I
> > guess it is worth a shot. I'll let you know what happens without -rt.
> 
> Ha, this is exciting, vanilla 3.6.4 works, with -rt10 patch it does not.
> 
> Copying in linux-rt-users.
> 
> To recap, 3.6 RT seems to break dm-crypt. Try it like this:
> 
> cryptsetup --debug --key-file=- luksOpen /some/luks/formatted/file any-label
> 
> And kernel will fail like this:
> 
> Kernel says this:
>      device-mapper: table: 252:1: crypt: Error allocating crypto tfm
>      device-mapper: ioctl: error adding target to table
> 
> I think crypto modules are not getting auto loaded for some reason, but even
> if I load them manually before hand and still breaks in the same way.

I don't think it's a module problem. That code is not very helpful due
to _NOT_ showing the error number so it's hard to decode what actually
fails.

I put this on my todo list, but TBH it's low on my radar. If you are
interested then try to capture the failure with the function
tracer. Put a tracing_off() call into the failure code path, i.e.:

        ret = crypt_alloc_tfms(cc, cipher_api);
        if (ret < 0) {
+	   	tracing_off();
                ti->error = "Error allocating crypto tfm";
		        goto bad;
	}

Enable the function tracer before doing the cryptsetup invocation and
after that read out the trace, compress it and put it somewhere to
download or send it to me in private mail if you don't have a place to
put it.

I can decode the trace, but I don't have the capacity at the moment to
setup all the necessary stuff here and do the debugging myself.

Thanks,

	tglx

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/