ONJava.com    
 Published on ONJava.com (http://www.onjava.com/)
 See this if you're having trouble printing code examples


Subverting Java Access Protection for Unit Testing

by Ross Burton
11/12/2003

As everyone knows, unit testing is a great way of ensuring that code actually does what it claims it does, and that over time, as the system changes, it carries on doing the same thing. Formal processes like Extreme Programming (XP) depend heavily on consistent unit testing.

When used together, object-oriented design and comprehensive unit tests can lead to a very clean design, as the test-first methodology implies a user-based interface design, resulting in a public interface that is simple yet efficient.

However, when it comes to testing, sometimes these clean interfaces are not as good as they could be. There are often member variables that the test suite would like to access but that have been scoped private or protected, and making these members public would expose the internals, ruining the clean design. C++ has a way of working around this: by declaring the test suite as a friend class, the access protection is sidelined. In Java, a similar approach can be used by making certain members package scope and putting the test classes into the same package. However, this leads to an unsatisfactory design, as some members are private or protected for good reasons, and then an arbitrary set of members are package scope solely for the current test suite.

However, there is a third option available to Java programmers. The Java Reflection API allows the program to introspect arbitrary classes at runtime, accessing members at will. Most competent Java programmers should know this much, but what many don't know is that the security layer preventing you from accessing private members is only optional, and can be turned off. This allows the system code to be written without explicitly planning the unit tests, and if the unit tests really do need to access a private or protected member, they can.

Example Files

Download java-reflection-examples.tar.gz for the example code found in this article.

Throughout this article, I shall deal with the simplest case: fields. In Java Reflection, fields, methods, and constructors are all treated in a very similar manner, so whenever I say "field" here, it can generally be replaced with "method" and the examples will still work.

Does private Mean Private?

Let's begin by proving to ourselves that a private field is really a private field, and this is not just something we've all been told. First we need a class with some private members; this is the test class I shall use for the following examples.

class FieldTest {
    public String publicString = "Foobar";
    private String privateString = "Hello, World!";
}

Here we have a simple class with two fields: one private field and one public field. We would assume that arbitrary code can access the public field, but not the private field.

public class Test1 {
  public static void main(String args[]) {
    System.out.println(new FieldTest().publicString);
    System.out.println(new FieldTest().privateString);
  }
}

When we compile this we get:

Test1.java:4: privateString has private access in FieldTest
    System.out.println(new FieldTest().privateString);
                       ^
1 error

This shows that private fields really are private fields, as the Java compiler won't allow access to them.

The Class Object

In Java 1.0, the java.lang.Class object was fairly trivial. However, in Java 1.1, the Reflection API was added. A cursory glance at the documentation reveals several interesting methods:

These methods return Field instances (arrays, in the latter two) that allow us to see the name of the field and its type, and more importantly, get its value.

It seems that if we can get a Class instance, we can call getField() to get a Field instance, which we can use to get at the value we want. And the easiest way to get a Class object for the class Foo is to write Foo.class, a construct called a class literal.

Calling getField() looks like this:

import java.lang.reflect.Field;

public class Test2 {
  public static void main(String args[])
    throws Exception {
    Field f;
    
    f = FieldTest.class.getField("publicString");
    System.out.println("Public Field: " + f);

    f = FieldTest.class.getField("privateString");
    System.out.println("Private Field: " + f);
  }
}

Running this gives us:

Public Field: public java.lang.String FieldTest.publicString
Exception in thread "main"
  java.lang.NoSuchFieldException: privateString
	at java.lang.Class.getField0(Class.java:1735)
	at java.lang.Class.getField(Class.java:900)
	at Test2.main(Test2.java:10)

So what happened here? The code managed to get a reference to publicString but failed when trying to get a reference to privateString. The exception thrown was NoSuchFieldException, but I know it does exist, since I created it. However, the fine print in the API documentation for Class.getField() clearly states, "... the specified public member field ...". Time to try getFields() and getDeclaredFields().

Java Extreme Programming Cookbook

Related Reading

Java Extreme Programming Cookbook
By Eric M. Burke, Brian M. Coyner

Enumerating Fields

At first glance, getFields() and getDeclaredFields() seem very similar. Nevertheless, a closer read of the API documentation reveals that they are very different.

The method getFields() reflects (no pun intended) what the Java programmer conceptually sees when programming: it enumerates all publicly accessible fields in the class and all of its superclasses.

On the other hand, getDeclaredFields() reveals how the class is constructed. It enumerates fields, but only if they are actually declared in that class; any inherited fields are ignored.

The reason for the existence of two methods (instead of a single method that returns all fields, including inherited ones) seems to be so that simple dynamic lookup of public fields can be achieved easily (using getField() and getFields()) and generally does the right thing. If a program wants to see the private fields, it will probably want to handle inherited fields specially (for example, an object-oriented debugger).

Exercising these methods is always a good thing, to check they do what we expect. Here is a test for getFields().

import java.lang.reflect.Field;

public class Test3 {
  public static void main(String args[]) {
    final Field fields[] =
        FieldTest.class.getFields();
    for (int i = 0; i < fields.length; ++i) {
      System.out.println("Field: " + fields[i]);
    }
  }
}

The output is rather predicable:

Field: public java.lang.String FieldTest.publicString

Now let's try the same test, but use getDeclaredFields() instead.

import java.lang.reflect.Field;

public class Test4 {
  public static void main(String args[]) {
    final Field fields[] =
        FieldTest.class.getDeclaredFields();
    for (int i = 0; i < fields.length; ++i) {
      System.out.println("Field: " + fields[i]);
    }
  }
}

We hope that this will yield all of the fields, both public and private:

Field: public java.lang.String FieldTest.publicString
Field: private java.lang.String FieldTest.privateString

Life is indeed good. Now that we can enumerate all of the fields in a class, we can get the specific field we are after and hopefully manipulate it however we want.

Using Field Objects

Time to investigate what the Field object can do for us. There are the classic get() and set() methods, a large array of convenience methods to handle the getting/setting of a particular primitive type (to avoid having to wrap the primitive in an object manually), and a few methods to determine what the field is (getName(), getModifiers()).

It is the get() method that we need, as we want to retrieve the value of a String field. This method takes a reference to the object that that we want to get the field from (this object is ignored if the field is static). So this next example should work fine:

import java.lang.reflect.Field;

public class Test5 {
  public static void main(String args[])
    throws Exception {
    final Field fields[] =
        FieldTest.class.getDeclaredFields();
    for (int i = 0; i < fields.length; ++i) {
      if ("privateString".equals(fields[i].getName())) {
        System.out.println(
            fields[i].get(new FieldTest()));
        break;
      }
    }
  }
}

However, running this produces an exception:

Exception in thread "main" java.lang.IllegalAccessException:
  Class Test5 can not access a member of class FieldTest
  with modifiers "private"
   at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:57)
   at java.lang.reflect.Field.doSecurityCheck(Field.java:811)
   at java.lang.reflect.Field.getFieldAccessor(Field.java:758)
   at java.lang.reflect.Field.get(Field.java:228)
   at Test5.main(Test5.java:8)

And everything was looking so good. However, there is hope. Another reading of the fine print in the API documentation leads to the interesting line "If this Field object enforces Java language access control, and the underlying field is inaccessible, the method throws an IllegalAccessException" (emphasis added). If enforcing access control is optional, there has to be a way to turn it off.

The Security Manager

The Security Manager in Java is designed to stop arbitrary code from performing unwanted actions. It is primarily used in controlled environments, such as Java applets, where you wouldn't trust the applet to read files from your disk (since it could send your password database to a cracker). However, in the common case of a Java application executed from the command line, the security manager imposes few restrictions.

The exception we're getting comes from Java denying access to a private field, as the field is "not accessible." This check doesn't involve the Security Manager, but simply calls isAccessible() on the superclass of Field, AccessibleObject. The accessibility of a private field defaults to false, but a call to setAccessible() will perform the appropriate security checks on my credentials and hopefully give me access.

import java.lang.reflect.Field;

public class Test6 {
  public static void main(String args[])
    throws Exception {
    final Field fields[] =
        FieldTest.class.getDeclaredFields();
    for (int i = 0; i < fields.length; ++i) {
      if ("privateString".equals(fields[i].getName())) {
        fields[i].setAccessible(true);
        System.out.println(
            fields[i].get(new FieldTest()));
        break;
      }
    }
  }
}

The only difference between this and Test5.java is the addition of the setAccessible() call. When we run this test we see:

Hello, World!

Which is exactly what we were after: we can access a private field and see its value. Very similar code should allow us to edit this field as well.

import java.lang.reflect.Field;

public class Test7 {
  public static void main(String args[])
    throws Exception {
    final Field fields[] =
        FieldTest.class.getDeclaredFields();
    for (int i = 0; i < fields.length; ++i) {
      if ("privateString".equals(fields[i].getName())) {
        FieldTest fieldTest = new FieldTest();
        Field f = fields[i];
        f.setAccessible(true);
        System.out.println(f.get(fieldTest));
        f.set(fieldTest, "Modified Field");
        System.out.println(f.get(fieldTest));
        break;
      }
    }
  }
}

This code gives the predictable outcome:

Hello, World!
Modified Field

Note that no amount of code will allow you to change the value of a final field.

The Security Manager, Part 2

As I said earlier, the Java Security Manager only lets you access private fields if you have appropriate rights. Code that I run on my desktop is trusted in some sense, as I have to explicity run it. However, Java applets can be hidden on web pages and executed without the user knowing, so these are untrusted.

This can be demonstrated by re-writing Test6.java as an applet. This involves two parts, the actual Java code:

import java.lang.reflect.Field;
import java.applet.Applet;
import java.awt.Label;

public class Test8 extends Applet {
  public Test8() {
    super();
    String s = "Field not found";
    try {
      final Field fields[] =
        FieldTest.class.getDeclaredFields();
      for (int i = 0; i < fields.length; ++i) {
        if ("privateString".equals(fields[i].getName())) {
          fields[i].setAccessible(true);
          s = (String)fields[i].get(new FieldTest());
          break;
        }
      }
    } catch (Exception ex) {
      s = ex.getMessage();
    }
    add(new Label(s));
  }
}

and an HTML page to load it:

<html>
  <body>
    <applet width="100" height="50"
        code="Test8.class">
    </applet>
  </body>
</html>

It's not exactly a great example of applet programming, but it does the job. Now, either load this HTML file into either a Java-enabled web browser, or use the appletviewer that is supplied with the JDK. The result is shown in Figure 1.

Figure 1
Figure 1. Accessing private fields from an applet

As expected, attempting to access the private field in a secure context has been blocked. The only way around this is to "sign" the applet, so that the user can see who wrote the code, and decide whether they trust them or not. However, signing .jar files is beyond the scope of this article, and best saved for another day.

Methods and Constructors

So far I have only shown examples of accessing fields via reflection. However, to prove that this concept does work for methods and constructors, here's an example of calling a private method.

import java.lang.reflect.Method;

class MethodTest {
  private final String sayHello(final String name) {
    return "Hello, " + name;
  }
}

public class Test9 {
  public static void main(String args[])
    throws Exception {
    MethodTest test = new MethodTest();
    final Method[] methods =
        test.getClass().getDeclaredMethods();
    for (int i = 0; i < methods.length; ++i) {
      if (methods[i].getName().equals("sayHello")) {
        final Object params[] = {"Ross"};
        methods[i].setAccessible(true);
        Object ret = methods[i].invoke(test, params);
        System.out.println(ret);
      }
    }
  }
}

As you can see, getting a list of the declared methods is straightforward, using getDeclaredMethods(). Then the array is searched as before, and the correct method made accessible. The method that does the magic is invoke(). You call this on a Method object, passing it the instance it should call the method on (unless it is a static method, in which case you pass null), and an Object[] of parameters to the method to be called. If the method is void, then either an empty array or null is acceptable. Java will also automatically marshal to and from primitive types, so, for example, if you have a method int round (final float f) that takes a float and returns an int, the parameter array would have a single instance of java.lang.Float and an instance of java.lang.Integer would be returned.

Conclusion

Now that the Reflection API has been well exercised, let's bring it back to the field of unit testing. Say I wanted to write a unit test for FieldTest. Checking that the public field is the correct string is trivial:

import java.lang.reflect.Field;
import junit.framework.TestCase;

public class Test10 extends TestCase {
  public Test10(final String name) {
    super(name) ;
  }
  
  public void test_reflection() throws Exception {
    FieldTest f = new FieldTest();
    assertEquals (f.publicString, "Foobar");
  }
}

When this test case is executed, it works as expected.

$ java Test10
.
Time: 0.036
 
OK (1 test)

I could copy the loop I've been using to get a reference to the private field into the test case, but that would be very ugly if there were 10 private fields I wanted to check. Instead, a utility class is in order. Here is PrivateAccessor.java:

import java.lang.reflect.Field;
import junit.framework.Assert;

/**
 * Provides access to private members in classes.
 */
public class PrivateAccessor {
  public static Object getPrivateField (Object o,
  String fieldName) {
    /* Check we have valid arguments */
    Assert.assertNotNull(o);
    Assert.assertNotNull(fieldName);
    /* Go and find the private field... */
    final Field fields[] =
        o.getClass().getDeclaredFields();
    for (int i = 0; i < fields.length; ++i) {
      if (fieldName.equals(fields[i].getName())) {
        try {
          fields[i].setAccessible(true);
          return fields[i].get(o);
        } catch (IllegalAccessException ex) {
          Assert.fail ("IllegalAccessException accessing " +
            fieldName);
        }
      }
    }
    Assert.fail ("Field '" + fieldName +
                 "' not found");
    return null;
  }
}

The core of PrivateAccessor.java is the same as the previous examples, but the code is more reliable, as it checks the inputs and uses the Assert class to integrate with JUnit test cases. With this class, when a test case needs access to a private field, it can use getPrivateField to get a reference to the field, and the test case does the right thing automatically if the field was not found or the arguments were not valid. The test case can then assume that is has been given a valid object, removing some error checking code. I now present the final example, a JUnit test suite that tests FieldTest and uses PrivateAccessor:

import java.lang.reflect.Field;
import junit.framework.TestCase;
 
public class Test11 extends TestCase {
  public Test11(final String name) {
    super(name) ;
  }

  public void test_reflection() throws Exception {
    FieldTest f = new FieldTest();
    /* Assert we have a valid object */
    assertNotNull(f);

    /* Assert that the public field is "Foobar" */
    assertEquals (f.publicString, "Foobar");

    String s =
        (String) PrivateAccessor.getPrivateField(f,
                         "privateString");
    assertEquals (s, "Hello, World!");
  }
 
  public static void main(String args[]) {
    junit.textui.TestRunner.run (Test11.class);
  }
}

And to prove that this works:

$ java Test11
.
Time: 0.039
 
OK (1 test)

By using PrivateAccessor, we can write our classes without having to compromise the visibility of fields and other members just for the sake of JUnit tests. This way, we can do right by our sense of good object-oriented design, and still expose our classes to automated testing code.

Links

The following links will be helpful for background or further reading.

Ross Burton codes Java and embedded systems on Linux and enjoys Python, C, GTK+, and Debian GNU/Linux.


Return to ONJava.com.

Copyright © 2009 O'Reilly Media, Inc.