Using JAXB in custom Ant tasks on recent Java versions

Apache Ant 1.10.14 was released a few weeks ago https://lists.apache.org/thread/9vhk51nkw9wjxzm7xk2q9xm6s803prmr. Like noted in that announcement, apart from the regular bug fixes, this release also has an important change which allows it to be used in the recently released Java 21 https://inside.java/2023/09/19/the-arrival-of-java-21/. One major goal of Ant project is to make sure that it can be used to build projects using latest versions of Java. As such, the Ant project team keeps a watch on any changes in the Java releases that could affect Ant and does necessary changes in Ant and releases them in its 1.10.x release series.

The announcement of 1.10.14 and this changelog https://github.com/apache/ant/blob/rel/1.10.14/WHATSNEW contains the details of what’s exactly changed in this release, so I won’t go into those details again. In this post I’ll however go through one interesting issue that was brought to my notice by more than one projects, when using recent versions of Java. The issue specifically relates to custom Ant tasks that use JAXB. However, in general, it applies to some of the APIs that have been removed from recent versions of Java (details in https://openjdk.org/jeps/320)

JAXB as noted in the reference doc https://docs.oracle.com/javase/tutorial/jaxb/intro/arch.html is an API and implementation for XML binding. Up until Java 11, JAXB API and implementation classes were shipped as part of the Java runtime. What it meant was that any application code, like custom developed Ant tasks, could just use the JAXB APIs in their code without having to explicitly specific any external library dependency. In Java 11, JAXB along with few other modules was removed from the JDK. The release notes of JDK 11 lists this change https://www.oracle.com/java/technologies/javase/11-relnote-issues.html#JDK-8190378. Additionally, JEP-320 https://openjdk.org/jeps/320 has all the details related to this removal. When using JAXB APIs,  the usage of these modules from the JDK was transparent to the application. So although the application may not explicitly have referred to these module names, it was still reliant on them because they were providing public APIs which the application had references to. Effectively, if a project was using some Ant task which used JAXB, then those projects when they switch to any Java version >= 11 will now start seeing issues due to missing compile and runtime dependency on JAXB. Let’s now consider a simple custom Ant task and see what kind of errors it might encounter. But if you are just interested in the shorter answer and some sample code to fix these classloading issues with JAXB, then please check the end of this article, starting here. For the complete details, please read on.

We will use a trivial custom Ant task implemented by a class called org.myapp.HelloTask, which looks like:

package org.myapp;

import org.apache.tools.ant.Task;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;

public class HelloTask extends Task {

    private String message;

    public void setMessage(final String message) {
        this.message = message;
    }

    @Override
    public void execute() {
        try {
            final JAXBContext context = JAXBContext.newInstance(DataContainer.class);
            System.out.println("Created JAXB context " + context);
        } catch (JAXBException e) {
            throw new RuntimeException(e);
        }

        System.out.println(this.message);
    }

    private static class DataContainer {
        private String data;
    }
}

The HelloTask is just here for demonstration and in practice doesn’t provide any real value. This task allows for a message attribute to be specified. Furthermore, in its execute() method it creates a javax.xml.bind.JAXBContext for the application specific DataContainer class and just prints that context. Additionally, it also prints the message that was set when the task is launched. Let’s assume that this task has been compiled and packaged as a jar file and made available to the build process. Now consider this trivial build.xml file which declares this task and then launches it:

<project default="invoke-task">
    <property name="build.dir" value="build"/>
    <property name="jar.name" value="hellotask.jar"/>

    <taskdef name="hello" classname="org.myapp.HelloTask">
        <classpath>
            <pathelement location="${build.dir}/${jar.name}"/>
        </classpath>    
    </taskdef>  

    <target name="invoke-task" description="invokes the HelloTask">
        <hello message="hello world"/>
    </target>       

</project>

There’s not much in this build.xml. All it does is, use a taskdef to define the HelloTask with the classpath containing (just the) jar file containing the HelloTask. Then in the invoke-task target we launch this hello task by passing it a message.

Let’s now run this build against Java 8:

export JAVA_HOME=<path-to-JDK-8>
ant invoke-task

When you run this (on Java 8) you should see the output similar to:

invoke-task:
    [hello] Created JAXB context jar:file:/<path-to-jdk-8>/jre/lib/rt.jar!/com/sun/xml/internal/bind/v2/runtime/JAXBContextImpl.class Build-Id: ...
    [hello] Classes known to this context:
    [hello]   [B
    [hello]   boolean
    [hello]   byte
    [hello]   char
    [hello]   com.sun.xml.internal.bind.api.CompositeStructure
    [hello]   double
    [hello]   float
    [hello]   int
...
    [hello]   long
    [hello]   org.myapp.HelloTask$DataContainer
    [hello]   short
    [hello]   void
    [hello] 
    [hello] hello world

You’ll see that the JAXB usage in the task was successful and the build completed successfully and we didn’t have to configure any classpath to include any JAXB jar files. As noted previously, this is because the Java 8 runtime ships with the relevant JAXB API and implementation classes.

Now, without changing any code in the task or in the build.xml, let’s just switch to a recent Java version. Let’s say Java 17 and run the build:

export JAVA_HOME=<path-to-JDK-17>
ant invoke-task

When you do this, you will now see:

BUILD FAILED
build.xml:5: taskdef A class needed by class org.myapp.HelloTask cannot be found: javax/xml/bind/JAXBException
 using the classloader AntClassLoader[build/hellotask.jar]

You’ll notice that the build now fails and the error message states that the class javax/xml/bind/JAXBException cannot be found in the classpath which as defined in the build.xml only included the hellotask.jar (which just has the org.myapp.HelloTask). As noted previously, this is because the JAXB API and implementation is no longer shipped in the JDK. Applications, like this project, are expected to now include the JAXB API and implementation jars in the application classpath. JEP-320 https://openjdk.org/jeps/320 lists the potential Maven co-ordinates to find such jar(s). In case of JAXB it suggests the JAXB reference implementation available in Maven central repo com.sun.xml.bind:jaxb-ri as a potential candidate. Do note that, just like several other Java EE APIs and implementations, JAXB too has several vendors which implement the JAXB API. A “reference implementation” as the name states is meant to demonstrate the implementation of the specified API. There can, and are, several other vendor implementations for such APIs. It’s upto the applications to choose the ones that they desire to use. In this demonstration, we will use the 2.3.8 version of com.sun.xml.bind:jaxb-ri dependency (available at https://repo.maven.apache.org/maven2/com/sun/xml/bind/jaxb-ri/2.3.8/). There’s no specific reason for my choice of this specific version - it’s only for demo.

Now that we have this dependency made available, let’s include it in the classpath of the taskdef of HelloTask and our build.xml (snippet) will now look like:

...
    <taskdef name="hello" classname="org.myapp.HelloTask">
        <classpath>
            <pathelement location="${build.dir}/${jar.name}"/>
            <!-- JAXB dependencies -->
            <pathelement location="${lib.dir}/jakarta.activation.jar"/>
            <pathelement location="${lib.dir}/jakarta.xml.bind-api.jar"/>
            <pathelement location="${lib.dir}/jaxb-impl.jar"/>
        </classpath>    
    </taskdef>  
...

You’ll notice that the classpath of the taskdef now includes the JAXB related dependency jars. Now let’s rerun the build on Java 17, like previously:

export JAVA_HOME=<path-to-JDK-17>
ant invoke-task

When you run this, you will now see something that starts like this:

BUILD FAILED
build.xml:17: java.lang.RuntimeException: javax.xml.bind.JAXBException: Implementation of JAXB-API has not been found on module path or classpath.
 - with linked exception:
[java.lang.ClassNotFoundException: com.sun.xml.bind.v2.ContextFactory]
    at org.myapp.HelloTask.execute(Unknown Source)
    

So the build still fails, but unlike previously where it had failed when trying to define the HelloTask itself, this time it fails in the execute() implementation of the HelloTask.

So why does it fail even with the JAXB jars in the classpath. This has to do with JAXB (and several other Java EE APIs), which rely on thread context classloader. Several of these APIs, including this call to:

final JAXBContext context = JAXBContext.newInstance(DataContainer.class);

relies on thread context classloader to find the JAXB related classes and resources. It expects the thread context classloader, whichever it is, to be able to load these JAXB classes. Thread context classloaders are specific to the thread that is currently executing the code. So let’s quickly see which classloaders are in play in the execute() method of the HelloTask. To see that, let’s add some trivial debug messages in the code, whose snippet will now look like:

    @Override
    public void execute() {
        System.out.println(HelloTask.class + " was loaded by classloader: " + HelloTask.class.getClassLoader());
        final ClassLoader tccl = Thread.currentThread().getContextClassLoader();
        System.out.println("Context classloader of current thread is " + tccl);
        if (tccl instanceof java.net.URLClassLoader) {
            // let's additionally print the classpath of the URLClassLoader
            final java.net.URL[] classpath = ((java.net.URLClassLoader) tccl).getURLs();
            System.out.println("Context classloader's classpath is " + java.util.Arrays.toString(classpath));
        }
        try {
            final JAXBContext context = JAXBContext.newInstance(DataContainer.class);
            System.out.println("Created JAXB context " + context);
        } catch (JAXBException e) {
            throw new RuntimeException(e);
        }

        System.out.println(this.message);
    }

So what we have done here is that added a few System.out.println messages which prints the classloader which loaded the HelloTask and also prints the current thread’s context classloader. Additionally if the context classloader is a java.net.URLClassLoader, we even print the classpath used by the URLClassLoader. The goal of these debug messages is to see what classloaders are in play when using that JAXB API. Let’s rerun the build again on Java 17 - it’s still expected to fail like previously, but this time we should see these debug messages:

export JAVA_HOME=<path-to-JDK-17>
ant invoke-task

This fails with the same exception stacktrace as previously, but this time you should also see, something like:

[hello] class org.myapp.HelloTask was loaded by classloader: AntClassLoader[build/hellotask.jar:lib/jakarta.activation.jar:lib/jakarta.xml.bind-api.jar:lib/jaxb-impl.jar]
[hello] Context classloader of current thread is java.net.URLClassLoader@6d6f6e28
[hello] Context classloader's classpath is [file:/apache-ant-1.10.14/lib/ant-commons-net.jar, file:/apache-ant-1.10.14/lib/ant-xz.jar, file:/apache-ant-1.10.14/lib/ant-junit4.jar, file:/apache-ant-1.10.14/lib/ant-jai.jar, file:/apache-ant-1.10.14/lib/ant-apache-resolver.jar, file:/apache-ant-1.10.14/lib/ant-jdepend.jar, file:/apache-ant-1.10.14/lib/ant-apache-regexp.jar, file:/apache-ant-1.10.14/lib/ant-apache-log4j.jar, file:/apache-ant-1.10.14/lib/ant-javamail.jar, file:/apache-ant-1.10.14/lib/ant-apache-bcel.jar, file:/apache-ant-1.10.14/lib/ant.jar, file:/apache-ant-1.10.14/lib/ant-netrexx.jar, file:/apache-ant-1.10.14/lib/ant-swing.jar, file:/apache-ant-1.10.14/lib/ant-jsch.jar, file:/apache-ant-1.10.14/lib/ant-junitlauncher.jar, file:/apache-ant-1.10.14/lib/ant-jakartamail.jar, file:/apache-ant-1.10.14/lib/ant-junit.jar, file:/apache-ant-1.10.14/lib/ant-imageio.jar, file:/apache-ant-1.10.14/lib/ant-launcher.jar, file:/apache-ant-1.10.14/lib/ant-antlr.jar, file:/apache-ant-1.10.14/lib/ant-testutil.jar, file:/apache-ant-1.10.14/lib/ant-apache-oro.jar, file:/apache-ant-1.10.14/lib/ant-jmf.jar, file:/apache-ant-1.10.14/lib/ant-apache-xalan2.jar, file:/apache-ant-1.10.14/lib/ant-apache-bsf.jar, file:/apache-ant-1.10.14/lib/ant-commons-logging.jar]

You’ll see that the HelloTask was loaded using an instance of AntClassLoader (which is internal implementation detail of the Ant project) and this classloader has the relevant JAXB jars in its classpath (as seen in the message above). You’ll also notice in the log message that the thread’s context classloader is an instance of URLClassLoader:

[hello] Context classloader of current thread is java.net.URLClassLoader@6d6f6e28

and this instance of URLClassLoader has a classpath which has only Ant specific jars and nothing related to JAXB jars. Now when the specific call to JAXBContext.newInstance(...) gets made it ends up using the URLClassLoader (since it is the thread context classloader) which doesn’t have JAXB jars. Effectively, you end up seeing the classloading failures and the build fails.

So how do we fix this. The important bit here is that the thread context classloader should be the one which has the JAXB classes available, so that it can load them. Java’s java.lang.Thread class allows the context classloader to be changed/switched. In fact, in the Java EE ecosystem, frameworks, servers and other implementations (typically not the application code), switch the thread’s context classloader to a “relevant” classloader at the “right place” and then switch it back to the old context classloader when the operation completes. We will need a similar implementation here in the custom task’s execute() method. Here’s what the snippet will now look like (we no longer need the debug logging, so that’s now been removed):

@Override
    public void execute() {
        // get the current context classloader
        final ClassLoader tccl = Thread.currentThread().getContextClassLoader();
        try {
            // change the thread context classloader to this task's classloader
            // before using the JAXB API
            Thread.currentThread().setContextClassLoader(HelloTask.class.getClassLoader());
            final JAXBContext context = JAXBContext.newInstance(DataContainer.class);
            System.out.println("Created JAXB context " + context);
        } catch (JAXBException e) {
            throw new RuntimeException(e);
        } finally {
            // restore back the old context classloader
            Thread.currentThread().setContextClassLoader(tccl);
        }

        System.out.println(this.message);
    }

Notice that we get hold of the current context classloader and then before calling JAXBContext.newInstance(...) we change the context classloader to the HelloTask’s classloader. The HelloTask’s classloader, as we have seen so far, has the necessary JAXB jars (since we defined it in the classpath of the taskdef in the build.xml) from which it should be able to load the JAXB classes. Finally, and very importantly, in a finally block we restore the context classloader to whatever it was previously - that way rest of the code isn’t impacted by switching the thread context classloader. Let’s now build the project again on Java 17:

export JAVA_HOME=<path-to-JDK-17>
ant invoke-task

When you now run this, you should see:

invoke-task:
    [hello] Created JAXB context jar:file:/lib/jaxb-impl.jar!/com/sun/xml/bind/v2/runtime/JAXBContextImpl.class Build-Id: 2.3.8
    [hello] Classes known to this context:
    [hello]   [B
    [hello]   boolean
    [hello]   byte
    [hello]   char
    [hello]   com.sun.xml.bind.api.CompositeStructure
    [hello]   double
    [hello]   float
    [hello]   int
    ...
    [hello]   org.myapp.HelloTask$DataContainer
    [hello]   short
    [hello]   void
    [hello] 
    [hello] hello world

So the build now succeeds.

To summarize, recent versions of JDK have removed certain APIs from the JDK. Some of such APIs are available as external libraries which can be configured in the application classpath. In context of Ant, if a task that is shipped by Ant makes use of such APIs, then the Ant release note and the manual of that task will make a note of such change and will also note what needs to be done to get that task functional. In other cases, where custom tasks are involved and depend on APIs that are no longer part of the JDK, on most occasions they will need to update their build files to include those dependencies in their taskdef’s classpath. In some additional cases, like this very specific JAXBContext API usage, they might even have to do changes to the task’s code to use the right classloader. Do note that I decided to use this specific API of JAXB only to demonstrate the classloader change that would be needed in the task. Not all tasks that use JAXB would need this change in the task’s code - the build.xml classpath changes are expected. Also note that switching of classloaders shouldn’t be done blindly as it can cause other classloading issues. It should only be done when the specific API call in question has semantics which specify the use of a context classloader.

Some of you would be wondering if Ant itself should be doing a change where it sets the “right” thread context classloader before invoking the tasks. It wouldn’t be right for Ant to be doing such a change - depending on what the custom task does, there could be different answers to what is the “right” thread context classloader to use and for what duration. Answers to either of those questions are task specific and as such should be handled/implemented in the (custom) tasks.