full-disclosure - Java integer overflows (was: a really long topic)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Thu Mar 30 01:11:32 2006
From: kf_lists at digitalmunition.com (KF (lists))
Subject: Java integer overflows (was: a really long
	topic)

Tim Hollebeek wrote:

>So you're saying that if I write code that fails to validate
>array bounds, and something bad happens, it is the 
>compiler/runtime's fault, but if I write code that fails to 
>validate integer bounds, it is the programmer's fault?
>
>  
>
Here why don't just use a real example...

Whos fault is this? The coder or the JVM... (its pretty clear in my mind)

[root@...singmatch JNI]# cat HelloWorld.java
class HelloWorld
{
  public static void main(String args[])
  {
        HelloWorld hello = new HelloWorld();
        hello.displayMessage();
  }
  public native void displayMessage();
  static
  {
    System.loadLibrary("HelloWorldImp");
  }
}

compile it
[root@...singmatch JNI]# javac HelloWorld.java

get the jni headers from it
[root@...singmatch JNI]# javah -jni HelloWorld
[root@...singmatch JNI]# cat HelloWorld.h
/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class HelloWorld */

#ifndef _Included_HelloWorld
#define _Included_HelloWorld
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class: HelloWorld
 * Method: displayMessage
 * Signature: ()V
 */
JNIEXPORT void JNICALL Java_HelloWorld_displayMessage
  (JNIEnv *, jobject);

#ifdef __cplusplus
}
#endif
#endif

Make a .c file with the methods to run when the JNI call is made
[root@...singmatch JNI]# cat HelloWorld.c
#include <stdio.h>
#include <stdlib.h>
#include "HelloWorld.h" // this header file was generated by javah
JNIEXPORT void JNICALL Java_HelloWorld_displayMessage(JNIEnv *env,
jobject obj)
{
  char fixed[20];
  char *test = getenv("TEST");
  sprintf(fixed, "%s", test);
  printf("%s\n", fixed);
}

Compile this .c file as a shared object.
[root@...singmatch JNI]# cc -o libHelloWorldImp.so -shared HelloWorld.c
-I/usr/dlc/java/jdk130/include/

Now we have a .class and a .so ... we should be ready to roll
[root@...singmatch JNI]# ls -al
total 32
drwxr-xr-x 2 root root 4096 Jun 27 16:36 .
drwxrwxrwt 6 root root 4096 Jun 27 16:36 ..
-rw-r--r-- 1 root root 203 Jun 27 16:31 HelloWorld.c
-rw-r--r-- 1 root root 473 Jun 27 16:35 HelloWorld.class
-rw-r--r-- 1 root root 395 Jun 27 16:31 HelloWorld.h
-rw-r--r-- 1 root root 235 Jun 27 16:30 HelloWorld.java
-rwxr-xr-x 1 root root 5415 Jun 27 16:36 libHelloWorldImp.so

[root@...singmatch JNI]# export TEST=`perl -e 'print "A" x 50'`
[root@...singmatch JNI]# java HelloWorld
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
SIGSEGV 11 (*) segmentation violation
    si_signo [11]: SIGSEGV: (*) segmentation violation
    si_errno [0]: Success
    si_code [1]: SEGV_MAPERR [addr: 0x33]

        stackpointer=0xbffc1ae4
Writing java dump to javacore14629.1025210621.txt ...

At this point the session totally hung... (telnet)
the javacore14629.blah.txt was never written to...
-rw-r--r-- 1 root root 0 Jun 27 16:43
javacore14629.1025210621.txt

oddly enough I tryed a HUGE value ... and got totally different results.

[root@...singmatch JNI]# export TEST=`perl -e 'print "A" x 5000'`

[root@...singmatch JNI]# java HelloWorld
AAAAAAAAAAAAAAAAAAAAA.......
(process hung)

login on different terminal... type ps -ef

root 14863 2877 0 16:50 ? 00:00:00
/usr/dlc/java/jdk130/jre/bin/exe
root 14864 2877 0 16:50 ? 00:00:00
/usr/dlc/java/jdk130/jre/bin/exe
root 14865 14786 99 16:50 pts/3 00:00:03
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
root 14874 14865 0 16:50 pts/3 00:00:00
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
root 14875 14874 0 16:50 pts/3 00:00:00
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
root 14876 14874 0 16:50 pts/3 00:00:00
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
root 14877 14874 0 16:50 pts/3 00:00:00
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

[root@...singmatch JNI]# gdb blah 14865
GNU gdb Red Hat Linux 7.x (5.0rh-15) (MI_OUT)
Copyright 2001 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...blah: No such file or
directory.

/tmp/JNI/14865: No such file or directory.
Attaching to process 14865
0x40266aae in ?? ()
(gdb) bt
#0 0x40266aae in ?? ()
#1 0x4030379d in ?? ()
#2 0x4030378a in ?? ()
#3 0x40303a76 in ?? ()
#4 0x40033517 in ?? ()
#5 <signal handler called>
#6 0x40307d03 in ?? ()
#7 0x40299bbb in ?? ()
#8 0x40266abf in ?? ()
#9 0x4030379d in ?? ()
#10 0x4030378a in ?? ()
#11 0x40303a76 in ?? ()
#12 0x40033517 in ?? ()
#13 <signal handler called>
#14 0x40307d03 in ?? ()
#15 0x40299bbb in ?? ()
#16 0x40266abf in ?? ()
#17 0x4030379d in ?? ()
#18 0x4030378a in ?? ()
#19 0x40303a76 in ?? ()
#20 0x40033517 in ?? ()
#21 <signal handler called>

-KF