Java 6 compiler API tutorial

One of the new features in Java 6 is compiler API. It let’s you compile any Java source code file on the fly and after compilation you can load compiled class using a class loader and start using it. Because Java compiler doesn’t know anything about classes compiled on the fly, so you can use them only using java reflections.

In the first step we need to prepare a class that is going to be compiled using Compiler API. I prepared the following simple class:

package com.blogspot.mike_java;

public class ToBeCompiled {

public void callMe() {
System.out.println("Method callMe has been called");
}

}
And saved it to the directory “d:\javafiles\src” (absolute path to the ToBeCompiled.java file is “d:\javafiles\src\com\blogspot\mike_java\ ToBeCompiled.java”) and prepared directory for compiled files “d:\javafiles\bin”. The next step is to prepare a class that compiles ToBeCompiled class, loads it and runs method callMe() on an instance of it. I prepared in my favorite IDE Eclipse following class:

package com.blogspot.mike_java;

public class CompilerAPITest {

public static void main(String[] args) {
// source code is going to be here
}
}

All following source code lines will be in the main method of the CompilerAPITest class.

At first we need to get instance of JavaCompiler and StandardJavaFileManager:

JavaCompiler jc = ToolProvider.getSystemJavaCompiler();
StandardJavaFileManager sjfm = jc.getStandardFileManager(null, null, null);

The next step is to get Iterable collection of files to be compiled and prepare options for compiler (you can find list of compiler options at http://java.sun.com/javase/6/docs/technotes/tools/windows/javac.html):

File javaFile = new File("d:/javafiles/src/com/blogspot/mike_java/ToBeCompiled.java");
Iterable fileObjects = sjfm.getJavaFileObjects(javaFile);
String[] options = new String[]{"-d", "d:/javafiles/bin"};

The next step is to compile Iterable collection of java files and close file manager:

jc.getTask(null, null, null, Arrays.asList(options), null, fileObjects).call();
sjfm.close();
System.out.println("Class has been successfully compiled");

After this step all compiled classes would be saved in directory “d:\javafiles\bin”. The next step is to load compiled class using class loader and run method callMe. At first we need to load compiled class:

URL[] urls = new URL[]{ new URL("file://d:/javafiles/bin/") };
URLClassLoader ucl = new URLClassLoader(urls);
Class clazz = ucl.loadClass("com.blogspot.mike_java.ToBeCompiled");
System.out.println("Class has been successfully loaded");

And get the method callMe using reflections:

Method method = clazz.getDeclaredMethod("callMe", null);

Finally we need to create new instance of the just loaded class and call method callMe on it:

Object object = clazz.newInstance();
method.invoke(object, null);

You can download source codes at http://rapidshare.com/files/102030988/compiler_api.zip. I hope that this tutorial was useful for you.

Simple POP3 client in Java (tutorial)

This article is intended for people who have no or very little experience with networking in Java.

Today, my aim is to show you how to make your own simple POP3 client in Java. Some people are scared of creating a network application in Java because they think it have to be difficult to handle such topic, but opposite is true. It’s easier than you could think as you will see later. I chose the POP3, which is used to access your mailbox on a server, because it’s quite simple protocol. It has just a few commands. We are not going to implement all of them of course, just a few.

Before creating a first class in Java, let’s see the RFC 1939 specification of the POP3 protocol that can be found at http://www.faqs.org/rfcs/rfc1939.html. It’s always important to go through a specification of a protocol you are going to implement and to understand it well. You should read paragraph “3. Basic operation” where you would find information about default port for POP3 protocol (110) and two possible responses for any POP3 command: -OK indicating success and –ERR indicating a problem on the server. By the way response from the server can be multi-line (typically when server sends an email content) or single-line (e.g. response for command “USER”).

Now, when you are familiar with basic protocol operation we can start and create a first Java class, let’s give it name ‘POP3Client’. What fields would the class need to have? Firstly an instance of Socket (java.net.Socket) class for network communication and secondly reader as well as writer for receiving and sending network data. That’s could be enough, but for debugging feature let’s add boolean debug field and POP3’s default port value field. You should have something similar to the following:

public class POP3Client {

private Socket socket;

private boolean debug = false;

private BufferedReader reader;
private BufferedWriter writer;

private static final int DEFAULT_PORT = 110;

public boolean isDebug() {
return debug;
}

public void setDebug(boolean debug) {
this.debug = debug;
}
}

For simplicity I won’t deal with exceptions and state (with exception of connection establishment) below. You should have on mind that POP3 is state protocol and you should always check for the state before trying to do anything (e.g. client can’t logout from the server while not connected). The first method would handle connecting to the server and initializing needed fields. Let’s name the method connect(…) and implement it:

public void connect(String host, int port) throws IOException {
socket = new Socket();
socket.connect(new InetSocketAddress(host, port));
reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
writer = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
if (debug)
System.out.println("Connected to the host");
}

The first line creates new socket, the second connects it to the specified host and address, the third and fourth sets reader and writer and the last prints debug message. Let’s create second method that would use the DEFAULT_PORT field

public void connect(String host) throws IOException {
connect(host, DEFAULT_PORT);
}

Once the socket is connected to the server, you would need to somehow check for establishment of connection before disconnecting. Let’s implement isConnected() method:

public boolean isConnected() {
return socket != null && socket.isConnected();
}

Now we can implement disconnect() method that would disconnect socket from the server:

public void disconnect() throws IOException {
if (!isConnected())
throw new IllegalStateException("Not connected to a host");
socket.close();
reader = null;
writer = null;
if (debug)
System.out.println("Disconnected from the host");
}

Implementation of the method is quite straightforward, so let’s move a bit. Now the most important part will take place. We need somehow to send commands to the server and receive response. For sending data will use “writer” and for receiving data “reader”. Sending data is simple as long as receiving. To send data to server, you just call writer’s write(…) method and to receive you just call reader’s readLine() method. Let’s create and implement method readResponseLine() that will read and return one line of data sent by the server:

protected String readResponseLine() throws IOException{
String response = reader.readLine();
if (debug) {
System.out.println("DEBUG [in] : " + response);
}
if (response.startsWith("-ERR"))
throw new RuntimeException("Server has returned an error: " + response.replaceFirst("-ERR ", ""));
return response;
}

The first line reads the line of text sent by the server, the second prints it while in debug mode and the third checks for an error according to RFC specification. When we have got the method for receiving data from server, we need the method for sending data – commands - to the server. Let’s name it sendCommand(…) and implement it:

protected String sendCommand(String command) throws IOException {
if (debug) {
System.out.println("DEBUG [out]: " + command);
}
writer.write(command + "\n");
writer.flush();
return readResponseLine();
}

The first line prints command being sent while in debug mode, the second sends it and the third flushes the socket’s buffer. It’s one of the most important things and you should never forget to do it. If you don’t flush the buffer then the data are held in it (until the buffer is full) and not sent to the server. The last line reads and returns server’s response to our command.

You should know from RFC specification that just after connecting to the server, the server sends welcome string (something like “+OK Hello, this is POP3 server v. 2.3.16.”), so you have to receive it. Let’s modify existing connect method:

public void connect(String host, int port) throws IOException {
…
readResponseLine();
}

The next thing we need to do is to add support for logging in and out from the server. Let’s create two methods - login(…) and logout() - and implement them:

public void login(String username, String password) throws IOException {
sendCommand("USER " + username);
sendCommand("PASS " + password);
}

public void logout() throws IOException {
sendCommand("QUIT");
}

You can find in the RFC specification that to login to the server you need to send the combination of commands USER and PASS. The USER command takes as an argument username to an email account (e.g. john.dow@myserver.com) and PASS command takes as an argument the password. To logout from the server you need to send QUIT command. Once a user is logged to the server he can start manipulating his email messages. Firstly let’s create a method that will check for number of messages on the server:

public int getNumberOfNewMessages() throws IOException {
String response = sendCommand("STAT");
String[] values = response.split(" ");
return Integer.parseInt(values[1]);
}

I chose the STAT command (there’s also LIST command) that returns number of messages and theirs overall size in bytes (response has format: “+OK nn mm”, where nn is the number of messages and mm the size of messages in bytes). Now it’s the right time to prepare class for holding a message so let’s create it:

public class Message {

private final Map<String, List<String>> headers;

private final String body;

protected Message(Map<String, List<String>> headers, String body) {
this.headers = Collections.unmodifiableMap(headers);
this.body = body;
}

public Map<String, List<String>> getHeaders() {
return headers;
}

public String getBody() {
return body;
}

}

As you can suspect, every message has a list of headers (each header has a name and one or more values) and a body. The next method we are going to implement is method that gets a message from the server. Let’s name it getMessage(..):

protected Message getMessage(int i) throws IOException {
String response = sendCommand("RETR " + i);
Map<String, List<String>> headers = new HashMap<String, List<String>>();
String headerName = null;
// process headers
while ((response = readResponseLine()).length() != 0) {
if (response.startsWith("\t")) {
continue; //no process of multiline headers
}
int colonPosition = response.indexOf(":");
headerName = response.substring(0, colonPosition);
String headerValue;
if (headerName.length() > colonPosition) {
headerValue = response.substring(colonPosition + 2);
} else {
headerValue = "";
}
List<String> headerValues = headers.get(headerName);
if (headerValues == null) {
headerValues = new ArrayList<String>();
headers.put(headerName, headerValues);
}
headerValues.add(headerValue);
}
// process body
StringBuilder bodyBuilder = new StringBuilder();
while (!(response = readResponseLine()).equals(".")) {
bodyBuilder.append(response + "\n");
}
return new Message(headers, bodyBuilder.toString());
}

The method uses RETR command to get a message from the server. After sending the command, the server starts sending a message. Firstly headers terminated by empty line (its length is 0) followed by body terminated by line containing “.” character. The first loop goes through the all headers and saves it to the map “headers”. Each header is in the following format:

headerName: headerValue

The second loop goes through the all body response lines until the termination character is found. Finally the last method we are going to implement is method that will get complete list of messages from the server. Let’s name it getMessages():

public List<Message> getMessages() throws IOException {
int numOfMessages = getNumberOfNewMessages();
List<Message> messageList = new ArrayList<Message>();
for (int i = 1; i <= numOfMessages; i++) {
messageList.add(getMessage(i));
}
return messageList;
}

I hope that it’s quite straightforward after all we did.

Now it’s the right time to test our implementation, let’s create some class and use the client in it:

public static void main(String[] args) throws IOException {
POP3Client client = new POP3Client();
client.setDebug(true);
client.connect("pop3.myserver.com");
client.login("name@myserver.com", "password");
System.out.println("Number of new emails: " + client.getNumberOfNewMessages());
List<Message> messages = client.getMessages();
for (int index = 0; index < messages.size(); index++) {
System.out.println("--- Message num. " + index + " ---");
System.out.println(messages.get(index).getBody());
}
client.logout();
client.disconnect();
}


Because the debug mode was turned on you should see a list of POP3 commands and its responses.

I hope that this tutorial was hopeful for you and that you enjoyed it as well as I did.

You can download source codes at http://rapidshare.com/files/100278017/pop3client.zip

Groovy performance test

A few days ago I came across an article about Groovy (http://groovy.codehaus.org). It took me so I decided to go through useful Groovy tutorial (you can find it here http://www.asert.com/pubs/Groovy/Groovy.pdf). Groovy is a scripting language based on Java. You can write your Java code in much simpler way using Groovy and you can whenever use legacy Java code in your Groovy code if you want to. Groovy adds to Java language closures, domain specific languages, metaprogramming and extends existing Java classes too (e.g. Groovy adds method eachLine(Closure closure) to standard java.io.File class). Groovy has also direct support for SQL, SOAP, web services, COM scripting, servlets and much more. Groovy seems to be very development oriented so I decided to do basic performance testing to see Groovy’s performance because performance is the most important question for every developer (I hope at least :-)).

Before testing itself it would be worthy to show you briefly how Groovy works e.g. with collections. Let’s have the following Groovy snippet:

names = ["Ted", "Fred", "Jed", "Ned"]
println names
shortNames = names.findAll{ it.size() <= 3 } println shortNames.size() shortNames.each{ println it }

You should notice that Groovy doesn’t force you to use semicolons. Now let’s move to the code itself. The first line declares and defines a list of four strings – “Ted”, “Fred”, “Jed” and “Ned”. The second line prints the list to the standard output. The third line uses closures (finally something more interesting ;-)). It takes each value in list ‘names’ and run a closure { it.size() <= 3 } over it. ‘it’ in the closure refers to a parameter to the closure. 'findAll' method finds and returns all values in ‘names’ list for which the closure returns value ‘true’. So ‘shortNames’ would be a list containing values from list ‘names’ with size lower or equal to 3. The fourth line prints the size of the list ‘shortNames’. Finally the fifth line prints all the values in list ‘shortNames’. The output is:

["Ted", "Fred", "Jed", "Ned"]
3
Ted
Jed
Ned

I hope you enjoyed short Groovy example and if you want to try on your own then simply download and install Groovy, run GroovyConsole, copy there the snippet above and run it.

Maybe you are wondering how the Groovy works. Groovy source code is converted into common Java code and then compiled using compiler to the Java byte code. In the Groovy source code the Groovy specific code (non normal Java one) is wrapped during conversion into Groovy's comon Java classes to get work. This wrapping certainly has a performance impact on your application and that’s the reason why I decided to do a performance test. Groovy has its own executable for running source codes (groovy.exe in Windows) and the executable accepts (among others) the same arguments as Java’s executable (java.exe). I run all the tests with arguments –Xms64m and –Xmx512m to ensure that memory conditions are the same.

The first feature I would like to test was file access. The test creates a new file, writes numbers 0 to 25000 into it (one number per line), closes the file then opens again, read all the lines and print the sum of the numbers to ensure that everything was all right. Source code in Java is:

File file = new File("temp.txt");
file.createNewFile();
PrintStream printer = new PrintStream(file);
long before = System.currentTimeMillis();
for (int i = 0; i <= 25000; i++) { printer.println(i); } long after = System.currentTimeMillis(); System.out.println("Writing done in: " + (after - before)); printer.close(); long result = 0; BufferedReader reader = new BufferedReader(new FileReader(file)); for (int i = 0; i <= 25000; i++) { result += Integer.parseInt(reader.readLine()); } after = System.currentTimeMillis(); System.out.println("Reading done in: " + (after - before)); reader.close(); System.out.println("Result: " + result); file.delete();

And corresponding code in Groovy is:

file = new File("temp.txt")
before = System.currentTimeMillis()
for (i in 0..25000) file.append(i + "\n")
after = System.currentTimeMillis()
println "Writing done in: " + (after - before)
long result = 0
before = System.currentTimeMillis()
file.eachLine( { result += Integer.parseInt(it)} )
after = System.currentTimeMillis()
println "Result: " + result
file.delete()
println "Reading done in: " + (after - before)

The result was partially unexpected because while writing performance was much better in Java, reading performance was better in Groovy although I was using BufferedReader in Java. Even setting the size of the buffer to higher value had no effect. I tried to find source code of Groovy’s java.io.File class but although the Groovy is open source, I had no success. Could anyone help me please? You can see the test results below.

The second feature I would like to test was collection access. I chose List (concretely ArrayList). The test creates new collection, adds 100 000 values and then removes the first 1000 values. Removing a value results in shifting remaining values to the left thus removing all 100 000 values would take very much of time. Source code in Java is:

List stringList = new ArrayList();
long before = System.currentTimeMillis();
for (int i = 0; i <= 100000; i++) { stringList.add(String.valueOf(i*i)); } long after = System.currentTimeMillis(); System.out.println("Added in: " + (after - before)); before = System.currentTimeMillis(); for (int i = 0; i <= 1000; i++) { stringList.remove(i); } after = System.currentTimeMillis(); System.out.println("Removed in: " + (after - before));

And corresponding code in Groovy is:

def stringList = []
before= System.currentTimeMillis()
for (int i in 0..1000000) stringList.add(String.valueOf(i*i))
after=System.currentTimeMillis()
println "Added in: " + (after - before) + " ms"
before= System.currentTimeMillis()
for (int i in 0..1000) stringList.remove(i)
after=System.currentTimeMillis()
println "Removed in: " + (after - before) + " ms"

The result didn’t surprise me at all, in both cases had the Java much better performance (adding values was even 65x faster) than Groovy. You can see the result below.

The final feature I would like to test was database access. I chose the 100% pure Java database – HSQLDB about which I’m going to write an article in the future. The test itself inserts into a simple table 25 000 records and then selects them and count the sum. Source code in Java is:

try {
Class.forName("org.hsqldb.jdbcDriver");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
Connection dbConnection = null;
Statement statement = null;
try {
dbConnection = DriverManager.getConnection("jdbc:hsqldb:hsql://localhost/mydb", "sa", "sa");
statement = dbConnection.createStatement();
statement.executeUpdate("DELETE FROM \"test\"");
long before = System.currentTimeMillis();
for (int i = 0; i <= 25000; i++) { statement.executeUpdate("INSERT INTO \"test\" VALUES (" + i + ", " + i + ")"); } long after = System.currentTimeMillis(); System.out.println("Data inserting: " + (after - before) + " ms"); ResultSet data = statement.executeQuery("SELECT VALUE FROM \"test\""); long result = 0; before = System.currentTimeMillis(); while (data.next()) { int value = data.getInt(1); result += value; } after = System.currentTimeMillis(); System.out.println("Data selecting: " + (after - before) + " ms"); data.close(); statement.close(); dbConnection.close(); System.out.println("result: " + result); } catch (SQLException e) { e.printStackTrace(); }

And corresponding code in Groovy is:

import groovy.sql.Sql

def sql = Sql.newInstance("jdbc:hsqldb:hsql://localhost/mydb", "sa", "sa", "org.hsqldb.jdbcDriver")
sql.executeUpdate("DELETE FROM \"test\"")
before = System.currentTimeMillis()
for (i in 0..25000) {
sql.executeUpdate("INSERT INTO \"test\" VALUES (${i}, ${i})")
}
after = System.currentTimeMillis()
println "Data inserting: " + (after - before) + " ms"
long result = 0;
before = System.currentTimeMillis()
sql.eachRow("select value from \"test\"") { result += it.value }
after = System.currentTimeMillis()
println "Data selecting: " + (after - before) + " ms"
println "Result: " + result

You can see that code in Groovy is really much shorter and simpler, but what about the performance? Again had the Java much better performance than Groovy, you can see the result below.

Conclusion

In the beginning I was really very curios how fast is Groovy going to be and in the end I was very disappointed. Of course that I expected lower performance than in Java, but I didn’t such huge performance impact. On the other hand Groovy is surely appealing project that just need a performance tuning to be done. I can recommend you trying Groovy but I don’t recommend you to develop a high performance application using it.