linux-kernel - Re: [PATCH] Implement prctl(PR_GET

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <19192.35446.970229.391493@pilspetsen.it.uu.se>
Date:	Mon, 9 Nov 2009 22:32:38 +0100
From:	Mikael Pettersson <mikpe@...uu.se>
To:	Andrew Morton <akpm@...ux-foundation.org>
Cc:	Helge Deller <deller@....de>, linux-kernel@...r.kernel.org,
	Anton Blanchard <anton@...ba.org>,
	Paul Mackerras <paulus@...ba.org>
Subject: Re: [PATCH] Implement prctl(PR_GET_ENDIAN) for all architectures

Andrew Morton writes:
 > On Thu, 22 Oct 2009 21:35:33 +0200
 > Helge Deller <deller@....de> wrote:
 > 
 > > The PR_GET_ENDIAN and PR_SET_ENDIAN prctl() calls have been implemented
 > > to allow to switch processes at runtime from big-endian to little-endian
 > > mode (and vice versa) on PowerPC processors. Since the other architectures
 > > don't support this feature, they currently will just fail and return -EINVAL.
 > > 
 > > This patch adds just minimal overhead and implements the PR_GET_ENDIAN
 > > call for all other architectures by returning the native endianess of
 > > the architecture. Furthermore, calling prctl(PR_SET_ENDIAN) with the
 > > native endianess of the architecture will succeed, while trying to
 > > set another (not-supported) endianess, will fail.
 > > 
 > > The patch can be tested with the following program:
 > > 
 > > #include <stdio.h>
 > > #include <linux/prctl.h>
 > > 
 > > int main(int argc, char **argv)
 > > {
 > > 	int endian, ret;
 > > 
 > > 	ret = prctl(PR_GET_ENDIAN, &endian);
 > > 	if (ret)
 > > 		perror("prctl(PR_GET_ENDIAN) not implemented");
 > > 	printf("current process/machine is running in %s endian mode (%d)\n",
 > > 		endian == PR_ENDIAN_LITTLE ? "little":"big", endian);
 > > 
 > > 	/* setting native endianess should succeed */
 > > 	ret = prctl(PR_SET_ENDIAN, endian);
 > > 	printf("prctl(PR_SET_ENDIAN,%d) should succeed: %s\n",
 > > 		endian, ret == 0 ? "OK":"FAIL");
 > > 
 > > 	/* setting foreign endianess should fail */
 > > 	endian = (endian == PR_ENDIAN_LITTLE) ?
 > > 		PR_ENDIAN_BIG : PR_ENDIAN_LITTLE;
 > > 	ret = prctl(PR_SET_ENDIAN, endian);
 > > 	printf("prctl(PR_SET_ENDIAN,%d) should fail: %s\n",
 > > 		endian, ret == 0 ? "OK":"FAIL");
 > > }
 > > 
 > 
 > The changelog forgot to provide any reason for making this change to
 > the kernel.

Why is PR_GET_ENDIAN needed? Surely user-space can detect the current
endianess for itself?

Also, which endianess does this refer to? ARM has had and continues to
invent bizarre endianess rules, where the endianess of some datum depends
not only on the endianess flag in a core control register but also the
functional unit operating on the data and the ARM ISA version. Some of
these endianess properties are unchangeable, some may be changeable in
specific implementations.

What about archs that always run in one endianess but support other-endian
data accesses via special instruction operands? (I believe SPARC falls into
this category.) PR_SET_ENDIAN on the other-endian value must fail, but what
does that mean to the program? That it can't do other-endian accesses? No.

I think the very concept of flipping a thread-global endianess mode flag
is too architecture specific to be treated as a generic thing.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/