Integration Tests with Arquillian
Lead technology consultant working for msg systems ag in Germany, Markus Eisele, examines the Arquillian tool for testing application code.
Since the early days of Java Enterprise Edition, testing of container components and their interaction has been a pain-point. Their cooperation with other services has been traditionally difficult to simulate. With the increasing functionality of the container (dependency injection, etc.) cross-links and dependencies have also grown. A fully automated integration test suite previously consisted of many JUnit tests. Now, Arquillian aims to make this task easier by offering the first complete and integrated test suite for integration testing.
The mission of the JBoss project Arquillian is easy. It’s about providing a simple test framework. The aim is to remove all container lifecycle and deployment tasks from the actual test logic and enable developers to build a broad range of integration tests for their enterprise Java applications. Integration tests should be a breeze with Arquillian. Since the early days of Java EE, testing enterprise applications has been a major challenge for any project. The isolated testing of technical components was impossible in some cases, since simple unit tests were frequently insufficient. One main reason for this, was the design of the components. They are rarely completely isolated and often depended on both their runtime and other components.
In the overall context, these aspects are as important as the technical work performed by a component. Even with a clean layer separation, a combination of so-called Mocks (interface simulation) and unit testing is required to integration test the interfaces and the implemented logic of the components. And despite all the potential problems, the actual behaviour of the target container remains completely untested. Neither transaction control, dependency injection, or other declarative services can be tested adequately. It is usually one of those final, open questions whether the right services are bound or the transferred data is actually in the proper format. Ensuring this, however, is an essential part of development processes in companies, and it is known as integration testing. The art of designing an automated integration test and regularly carrying them out, is something only a few experienced developers are able to do. This results in many stability problems for projects. The JBoss project Arquillian tries to eliminate this deficiency. It allows developers to easily run integration tests for container components. It does not matter whether these are deployed on a local (embedded) or remote container. The idea behind Arquillian is to make integration tests as simple as unit tests.
Arquillian (Figure 1) connects a unit testing framework (JUnit or TestNG), ShrinkWrap, and one or more supported containers (Java EE container, servlet container, Java SE CDI environment, etc.).
In essence it’s a custom test runner for JUnit and TestNG, which transfers control of the test lifecycle of the respective test implementation to Arquillian. From there Arquillian delegates to the respective environments for testing within the container or against the runtime. An Arquillian test case still looks almost exactly like a normal JUnit or TestNG test case. The testcases basically cover three different aspects. The testcase itself, the deployment, which has to be packaged into an archive and, last but not least, the deployment to the runtime.
Neither the deployment process nor the actual container control is very magical. The packaging of the testcases into the right format for the target runtime is done by ShrinkWrap. This is also the true start of an Arquillian test lifecycle. It defines the basis of the test case, a suitable archive at runtime and deploys it on the target container. After that the actual execution of the test case is initiated and results are collected. Finally, the archive will be removed from the runtime (undeployed). Up to now, there are only a couple of open-source containers supported. JBoss (> = 5), GlassFish (> = 3), Jetty (>= 6.1), Tomcat 6, Weld SE and EE 1.0 or 1.1, 1.0, IronJacamar 1.0, OpenWebBeans Apache 1.0 and Apache OpenEJB 3.1 are among them. According to the manufacturer future support will also cover commercial platforms such as Oracle’s WebLogic or IBM’s WebSphere.
If you want to start with Arquillian today, you should be aware that this is still an alpha release. You can download a complete archive from git or build it from sources. The following examples use the Arquillian Examples project, which is based on Arquillian 1.0.0.Alpha4. If you are trying out this alpha release you are not an early follower but a leader. You need the complete stack of brand new technologies.
In addition to Maven 3.0 a current 1.6 Java Development Kit (JDK) is also required. The best way to install the necessary parts is to install the complete development environment and extract the ZIP file with the examples into any directory. The most interesting use case certainly is the testing of Enterprise Java Beans (EJBs). In conjunction with the GlassFish Embedded Server 3.1 this is already included as a pre-configured example. Find it in the folder ejb31-gfembedded. All examples run with Maven. To use the embedded GlassFish as a container, create the required profile called “glassfish-embedded-3” in the projects pom.xml. The example is executed using the “mvn test-Pglassfish-embedded-3” command. A simple one-liner responds any runs or errors. If everything runs smoothly you will see: Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9828 seconds. If this does not work right away, you have to check for some potential problems and solve them. The number one reason for any errors are unknown or missing Maven repository entries. The Arquillian libraries are in the Public JBoss Repository. This should be referenced from your pom.xml or even settings.xml. If you are experiencing any java.net.ConnectExceptions the reason could be a known bug in Embedded GlassFish. It prevents builds greater than 3.1 b04 from working offline because it is checking for required xml schema files. The same will happen, if you sit behind a proxy or firewall. In this case you need to add some configuration parameters via the Surefire plugin (SystemProperties, http.proxyHost and http.proxyPort). But there are other circumstances, that prevent the most recent Arquillian Alpha from running with the latest GlassFish bits.
During development of GlassFish 3.1 the embedded API changed. This should be reflected with the next Arquillian release. So if you are in need of more recent builds, you should wait a little bit longer for Arquillian to provide an updated version.
If you experience other errors, you are basically helpless at first. Arquillian log messages are generally rare. So you need to dig a little deeper into the configuration. By default, the root directory of GlassFish stays in the target folder which was created by Maven. Each test run creates a new directory there. You can change this behavior by making changes to the arquillian.xml configuration file. It belongs to the srctestresources folder and should look like this:
Using this configuration, the GlassFish directory used each time a test is run, is always in the same place. You are now free to make any additional entries to the embedded GlassFish configuration by changing the domain.xml. But are you still missing the verbose logging? Then you have to get your hands on the logging.properties file. Simply copy it from an existing GlassFish domain and add it to your development project (For example srctestgf). Parse it on as a system property (java.util.logging.config.file) to the Surefire plugin. If you now rerun your tests you can see the container log messages.
Another very helpful artifact would be to have a version of the packaged archive that Arquillian is deploying. This is simple, too. Add the following entry to the arquillian.xml.
Besides satisfying your curiosity, this can be especially helpful in future debugging. Particularly if the generated archive can be successfully deployed to a standalone glassfish, it ensures that you are most likely stumbling over some problems that aren’t the project developer’s fault.
First Simple Testcase
When all preparations are complete, it is time to devote time to the actual test functionality. To test a component in the container, it must be present. A simple Stateless Session Bean HelloEJB should be enough as the first example:
The world’s best known example EJB is now being reviewed with a corresponding test case:
The @RunWith JUnit annotation ensures that the Arquillian test runner is used. After that you simply inject the HelloEJB. The @Deployment annotation declares the method that wraps the actual EJB archive. The actual test case is initiated by the annotation @Test in JUnit by known style. The assertEquals checks the return value against a fixed input string. If the two strings match, the test is a success. All other cases produce an error. This simple error doesn’t say much about the error in general at a first glance. So you have to check the targetsurefire-reports folder for details. Each testcase produces a result in text and xml files. This isn’t very handy, but you have to blame Surefire for that, not Arquillian. If you change the EJB call into anything to force an error (e.g. sayHelloEJB (“Markus”) you can find the specific error:
More Container Functions
In a similar manner as described above you can test almost any container functionality with Arquillian, with the Stateless EJBs being the simplest case. It becomes more complicated when the runtime with the appropriate resources (eg a database or interface) must be supplied or if Context and Dependency Injection (CDI) is required. But even for such requirements Arquillian has a solution. The simplest case is the use of CDI. To enable CDI with your testcase you simply have to add an empty beans.xml file to the test archive. This is achieved by:
From now on CDI functions are accessible because test.jar contains the required beans.xml in the META-INF folder. Another option is the use of an appropriate beans.xml with content. For this purpose, it has to be put into the project’s resources directory and referenced accordingly:
Up to now everything we tested was done using the GlassFish Embedded API. If you are hoping to test your Java Persistence API (JPA) applications you have to switch the container because these are only supported with the remote GlassFish container. This requires the entire Arquillian configuration to be changed. The remote deployment is backed by the JSR-88 API. If you are looking for some good examples you can consult the Arquillian User Guide.
Arquillian basically manages to test everything which could be directly injected or by facilitating the CDI features. In addition to the previous example, you can also use it to test JMS connections. For this it’s as simple as adding the appropriate queues, topics and connection factories:
Arquillian as a Client
Having tests for the external system representations, is as important as focusing on application testing from the inside. You should spend as much time on the interfaces which people or programs use to interact with your application. Typically it should be ensured that each application and its execution path is fully tested. Third parties can interact with the application through various interfaces, such as Web Services, EJBs, or remote via http. If you are taking care of this you have to consider a lot of things, like object serialization and network connections. In order to test this view onto the environment Arquillian provides two modes: IN_CONTAINER and AS_CLIENT. The IN_CONTAINER mode corresponds to the examples shown above. The AS_CLIENT allows checking the outside view of an application. Running the AS_CLIENT mode is as simple as adding another Annotation to your testcase:
In a direct comparison with the fashion IN_CONTAINER the client tries to fashion as few tasks to perform in the container and natural to act as a client. The @Deployment method simply controls the container’s lifecycle.
Debugging, and Eclipse Integration
If you don’t like the painful cmd-line maven execution of your Arquillian tests, you are free to integrate it into your favorite IDE as long as it is Eclipse or NetBeans or has any Maven integration at all. Eclipse requires you to use the m2eclipse plugin. You need to activate the relevant Maven profile, and you can start using Run As | JUnit Test to run individual tests from the IDE. NetBeans doesn’t require a separate plugin. It has Maven support since version 6.8. Here too, the active Maven Profile needs to be added. The configuration is already done. Debugging basically follows the same characteristics as with other deployed applications. Test-runner and tests run in a different JVM than the container itself. Therefore, the debugger has to be attached to the correct JVM. Details about this can be found in the container documentation of your choice.
It’s lots of fun to use Arquillian. There is nothing comparable on the market in terms of seamless container integration. It’s simple enough to use and not too hard to configure. Simple test cases can be created in minutes. Tradeoff is, that it quickly becomes complex. The configuration of the container is only one area. The more complex your deployments get (e.g. dependent libraries, special classpath configuration) the harder the setup. And Arquillian is still an Alpha version which is not finished at the time of writing. Some things simply don’t work at the moment. If you are running into trouble it could consume your time very quickly.
To find out where exactly the problem is, you need time and knowledge. The core of Arquillian doesn’t log too much. To find a real fault is not easy. But the project is not complete yet. A couple of features are already announced. In addition to Method Interceptors the so-called Arquillian Resources should become available with the next release. That will make any container-specific objects (such as the initial context) inject able via a generic @ArquillianResource annotation. Support for other containers is also on the list of things to come.
It will be exciting and worth further observation. The project is slightly behind schedule and it should come up with the next development milestone soon.