Go forth and test in-container!

Arquillian: A Component Model for Integration Testing

DanAllen
A-component-Model-for-Integration-Testing

“Why the best intentions to write tests are so often foiled and how to prevent it.”

If you’ve ever attended a No Fluff Just Stuff event, you likely came away with the message ingrained in your head that testing is important… correction, crucial, and you should do it. Period.

As Venkat Subramaniam, a long-time speaker on the No Fluff Just Stuff symposium series, often reminds audiences:

Testing is the engineering rigor of software development.

In other words, if you’re going to carry around the title Software Engineer, you had better put ample effort into engineering a plethora of tests.

But this article is not about the importance of testing. At this point, we’ll assume its importance is well established. What I’m here to address is why the best intentions to write tests are so often foiled and how to prevent it.The main obstacle is that your tests live in a different world than your application. To overcome this discrepancy, I’ll introduce you to the idea of having a component model for your tests, an architecture that is provided by Arquillian. Arquillian is a container-oriented testing framework developed in Java that brings your test to the runtime rather than requiring you to manage the runtime from the test. This strategy eliminates setup code and allows the test to behave more like the components it’s testing. The end result is that integration testing becomes no more complex than unit testing. You’ll believe it when you see how much power you get out of a simple Arquillian-based test.

I want to start by studying this transition from unit to integration tests and establish an understanding as to why making this transition knocks the wind out of the sails of so many testing initiatives. We’ll then look at how a component model for tests can ease this transition. Of course, we’ll look at some examples of how to use Arquillian along the way.

Why Don’t We Do It?

Why don’t we test? (For those of you who test religiously, think about a scenario when you’ve hesitated or decided to not write a test). I’ve asked this question at several NFJS events and it doesn’t take long before I hear all the responses I’m seeking:

• Too hard
• Not possible
• Not enough time
• Too slow
• Not enjoyable
• No proper tools

Unless you have an enlightened manager who embraces agile software practices (perhaps because she attended a NFJS event), you likely fear her walking by to find you working on something that isn’t the application code (i.e., tests). Worse is if she catches you writing your own testing framework, a tangent even by an enlightened manager’s standards. So we can generalize why testing gets the axe by saying there isn’t enough time.

The reason there isn’t enough time is because we don’t have the proper tools. I’m not talking about those fancy graphical tools. Those likely just compound the problem (because they require additional learning.) What we’re missing is a tool in the form of a programming model that makes us efficient at solving the other testing challenges listed above.

But wait, isn’t that what a unit testing framework is for? It is – to a point. Unit testing frameworks, such as JUnit and TestNG, provide a programming model for writing unit tests. They don’t specifically address integration testing concerns (though they can drive integration tests).What lies between these two classifications is a large chasm.

The Testing Bandgap

Unit tests are fine-grained, simple and fast. They exercise individual components while ignoring the rest of the system. Since the test plugins for IDEs were designed with this type of test in mind, executing them in the IDE is easy.

A majority of books and articles that cover testing demonstrate the concept with unit testing. Indeed, it looks so simple and promises to keep regression errors from creeping in that you feel motivated to run back to your application and start adding tests. But when you give it a try, you soon discover it’s harder than it looks.

Where things fall apart is when you transition to integration testing. Integration tests are coarse-grained and generally perceived as complex and slow (though this article will hopefully change that perception in your mind).They exercise multiple components from a subset of the application, verifying their communication and interactions.

Once you start involving multiple components, the task of isolating the code being tested from the rest of the system becomes more difficult. On top of that, getting components to interact often requires bootstrapping the environment they need in order to function (e.g., dependency injection). All of a sudden, the task of writing a test isn’t so simple.

I describe the sharp increase in configuration complexity between unit and integration testing as the testing bandgap, represented in Figure ALL-1.

The shift to integration testing requires a significant increase in mental effort when any non-trivial setup is required (which is typical). This interrupts your flow.

Work stops, your browser history becomes flooded with search results, the build script multiplies and the code-build-test cycle becomes a code-build…-catch up on social timeline-test cycle. Testing becomes tedious and costly, two things that directly lead to its demise. We’ve got to find a way past this complexity barrier because integration is all around us.

 

                                                                                 

Don’t Mock, Integrate

In a paper titled “What Is Software Testing? And Why Is It So Hard?” James Whittaker offers a well-defined classification of the various levels of testing (i.e., unit, integration, system, functional, etc). However, he struggles with the popular question of when and how to draw the line between unit and integration testing:

Unit testing sometimes requires the construction of throwaway driver code and stubs and is often performed in a debugger.

Some components can’t be unit tested because they weren’t designed with testing in mind, meaning they have lots of dependencies that are hard to mock. Even if they can be mocked, how much effort should you put into creating mocks? And how many mocks are permitted before a test loses its value?

I believe that anything more than a couple of lines of throwaway driver code and stubs is time thrown away. Mocks are not a substitute for integration testing.

We also shouldn’t be satisfied with only being able to use a debugger during unit testing. Debuggers are crucial for productivity. Why can’t we use them for integration testing as well? Or why is it so uncommon? Are integration tests that esoteric?

That brings us back to the lack of a programming model for writing and executing integration tests (and on up, including system and functional tests).We don’t want to beat around the bush by using mocks. We just want integration tests to be:

• as easy as writing unit tests
• executed in the IDE (incremental builds, debugging, etc)
• ramped up gradually (instead of “big bang” integration)
• portable

By providing a component model for tests, Arquillian eliminates the testing bandgap and flattens the configuration complexity curve as it graduates from unit to integration tests and beyond. The result is the testing continuum shown in Figure ALL-2. With Arquillian in your toolbox, you’ll be able to move comfortably between unit and integration testing.

Now that you’ve had your day’s worth of theory, let’s step through an example to see where a component model fits with testing and learn how Arquillian is able to leverage it to maintain this testing continuum.

 

Graduating from Unit to Integration Testing and Beyond

We’ll work from a unit test to an integration test to give you a true appreciation for just how smooth Arquillian makes this transition. When you put them side-by-side, the Arquillian test is going to look very similar to the unit test, and that’s the goal.

1 Arquillian supports both JUnit 4 and TestNG 5 in nearly the same way. To make the example easy to follow, we’ll just select JUnit 4.

A Unit to Test

To keep it simple, we’ll make our unit to test a calculator class. Basic addition isn’t very interesting as a service, though, so let’s make it a calculator that computes the monthly payment amount for a fixed term mortgage. The example class is shown in Listing ALL-1. Currency calculations can get a little dicey in Java if you don’t take into account precision, scale and rounding. So it’s worthy of testing. Let’s write a JUnit 4-based unit test case (Listing ALL-2) that ensures our calculator isn’t going to be the source of any bank errors.

The test case in Listing ALL-2 can be executed using any JUnit 4 plugin.

 

Testing Components

Notice that the test case is initiating MortgageCalculator itself. If this were just a class from a library, that would be reasonable. But let’s assume that the class functions as an EJB component (i.e., a service). EJB 3.1 makes this shift as simple as adding a single annotation to the class, as we’ve done by adding the @Stateless annotation in Listing ALL-3.

Don’t be turned off by the use of EJB for this example. I’ve selected EJB because it’s a recognizable component model. Rest assured, Arquillian can support any programming model, not just EJB. Our unit test will still work with this annotation present. That’s because all the logic is self-contained in the business method and the component doesn’t use any container features. But let’s put that fact aside for the moment and assume that we do want to test this class acting in its role as a component. We want to test an EJB instance.

The definition James Whittaker offers for software testing in the aforementioned paper applies well here (emphasis mine):

Software testing is the process of executing a software system to determine whether it matches its specification and executes in its intended environment.

Unit testing can help test inputs and outputs, but what it lacks is a mechanism by which we can execute a component in its intended environment. That’s the way other components will use it and we want to make sure that part works. We want to bridge theory and practice and test the component for real. We want integration testing.

That brings us to the question, “How do you test an EJB?” We need to start an EJB container and get it to create an instance and hand it to us. That means we need a container running in our test case. Here’s where the component model for tests starts to come into play.

You may have read about the embedded EJB container that was introduced in EJB 3.1 primarily for the purpose of testing. That’s good news! Using it would certainly satisfy the requirement of getting an EJB container setup in our test. Listing ALL-4 shows an example of how to boot the embedded EJB 3.1 container and use it to lookup an instance of our mortgage calculator component:

At first, the approach in Listing ALL-4 seems pretty reasonable (and a significant improvement over the EJB of old). But there are a number of problems with it:

You need to add before/after logic so the container is only created once per suite because startup isn’t instantaneous. You have to look up the component manually, rather than using the more convenient @EJB field/method injection.

The embedded EJB container is going to be scanning all over your project for EJBs – maybe not want you want.

The test is relying on a JNDI name for the component that is specific to the embedded environment.

An embedded environment isn’t the real deal.

The test is tied to the embedded EJB container.

Let’s step away from the embedded EJB container for a moment and look at a much simpler, declarative style that Arquillian’s component model brings to this test.

There’s A Component Model in My Test!

Anxious to see what a component model for a test looks like? Direct your attention to Listing ALL-5. You should immediately recognize several differences in the Arquillian version of this test. But, before addressing what’s present, let’s look at what isn’t. Notice there are no setup or tear down methods. We are going to push all the work of starting and stopping the container to Arquillian. And no controller code means a portable test. More on that later. The other three differences in this test are stamped by their corresponding annotations:

1. @RunWith

2. @Deployment

3. @EJB (and similar)*

* The @EJB annotation represents the link that Arquillian is providing to the application component model. If the application were employing JSR-299, you could use @Inject to get a reference to the component to test. Arquillian is not tied to EJB. 

Let’s study how each of these annotations contribute to the test’s component model. 

The Arquillian Test Runner

Scanning from top to bottom, the first thing you’ll notice is that the test class is now annotated with @RunWith(Arquillian.class). This annotation describes its purpose well.

The test class is run with Arquillian. Delegation to the Arquillian runner occurs immediately after the test is launched. As the JavaDoc for @RunWith explains, JUnit will invoke the class [that @RunWith] references to run the tests in [the annotated] class instead of the runner built into JUnit.

The Arquillian class is also a listener that hooks into the JUnit lifecycle to provide in-container testing functionality. It handles three responsibilities:

  • Control the lifecycle of the target container (i.e., runtime).
  • Orchestrate deployment of test artifacts to the container.
  • Execute tests inside the container, giving them access to the deployed components.

Don’t worry, the delegation to Arquillian won’t change how you run the test. All your standard JUnit plugins will still function. They have no idea any of this extra activity is going on under the covers. As far as they are concerned, this is just a plain old test case.

                         The Deployment

The second difference is the @Deployment method. Since Arquillian is a container-oriented testing framework, it needs to be able to give the container something it can digest. For example, Java EE containers accept EARs, WARs and (EJB) JARs. In general, we can say that containers speak in archives.

In the past, integration testing (or perhaps system testing, depending on where you draw the line) meant having to rely on a build process to assemble an archive to deploy. Arquillian takes a different approach. Rather than having to package these archives somewhere on the file system, Arquillian leverages ShrinkWrap to allow you to produce the archive within the test itself. ShrinkWrap is a fluent API for creating archives as Java objects that exist in memory. Within the @Deployment method, the developer hand picks classes, resources, libraries and/or preexisting archives needed by the test, stuffs them into a ShrinkWrap archive and returns the result. Arquillian then takes over and begins the process of deploying the archive to the container.

The immediate benefit of using ShrinkWrap is that we don’t have to rely on a build process to create the test archive that we are going to pass to the container. The test essentially prepares its own archive. Another benefit of using ShrinkWrap is that Arquillian can leverage its container integrations to more efficiently deploy the archive without having to write it out to the file system (not available with all containers).

After reading the archive from the @Deployment method, Arquillian appends its own resources, such as the test class, the test framework and extensions, and performs any other post-processing necessary. If the target container is managed by Arquillian, and the container is not yet started, Arquillian will setup and start the container (once per test suite). Once the container is ready, the Arquillian and ShrinkWrap adaptors for the container work together to translate the ShrinkWrap archive into a resource the container can read, perhaps by exporting it to a file or passing a stream to the container’s deployment API.

Once the test archive is deployed, depending on the type (or profile) of the container, Arquillian may use either a remote or local protocol to invoke the test methods. Containers are a topic for another article, so we’ll just briefly cover the different types so that you understand how the test gets executed.

2 Since Arquillian has an opportunity to modify the test archive prior to deployment, it means developers don’t have to add any special configuration (e.g., Servlet filters in web.xml) in order to get Arquillian to work. It just works.

3 Arquillian also has a client execution mode, which you activate by adding the @Run(AS_CLIENT) annotation to the test class. In this mode, Arquillian deploys the test archive, but leaves the test case behind to act as a remote client of the components being tested. This mode is useful for web testing, such as requesting a sequence of web pages or REST resources.

4 Test enrichment is not available in the client execution mode at the moment.

Standalone containers, such as the embedded EJB container, are executed in the same thread (and JVM) as the test runner. In this case, Arquillian employs a local protocol, which means it simply intercepts the test invocation, enriches the test case instance (which we’ll talk about next), and allows execution to proceed. Containers with a web component, such as a Java EE container, require Arquillian to work over a remote protocol (the most common being HTTP). Arquillian forwards execution of each test method to the server, typically communicating with an HTTP Servlet that Arquillian plants into the test archive before it gets deployed. On the remote end, Arquillian bootstraps the test runner, creates a new instance of the test case, enriches it, invokes the test method and returns the outcome of the test, including any exceptions, back to the client (typically via HTML).

The final link in the chain is bridging the test case to the application’s component model. That’s the role of the @EJB annotation in this example.

The Component Model Link

In Java EE, the @EJB annotation is used as a hint on a field (or setter method) of an EJB to tell the container to assign (or inject) a reference to another EJB. Only, in this case, it’s being used within the test class. And it’s Arquillian that’s doing the injecting. Arquillian intercepts the test invocation and satisfies the injection points (fields, setter methods and test method arguments) using a model consistent with the container being used. For instance, in a test case using JSR-299, Contexts and Dependency Injection for Java EE (CDI), Arquillian will satisfy the fields and methods annotated with @Inject. This step is called test enrichment.

By honoring the @EJB injection points in the test class, we’ve effectively brought the component model used in the application to the test environment. The test case is now one with the application. It speaks the same language as other consumers of the EJB we are testing. That’s important because it simplifies the test, removes the use of a potentially non-portable JNDI name and allows us to test the class as an EJB component running in its intended environment.

This link is not just about dependency injection, though. It’s about a shared component model made possible by running the test inside the container. There, the test has access to any resource the container provides. In a Java EE container, that includes resources in the naming directory (JNDI), environment entries, data sources, JMS destinations and so forth. In fact, these resources can be injected directly into the test using the @Resource annotation, also supported by Arquillian’s EJB test enricher. 

Let’s consider a scenario that relies on a containerprovided resource. Assume we want to change the monthly payment calculation in the mortgage calculator so it reads the interest rate from an environment entry. First, we add the @Resource injection to one of the component’s fields, shown in Listing ALL-6: 

Next, we set the value of the interest rate environment variable using an ejb-jar.xml descriptor, which we create in the test resources directory. The resulting structure of this descriptor is displayed in Listing ALL-7. 

We then package this descriptor into a test archive using ShrinkWrap, as shown in Listing ALL-8. Note we’ve switched to a WebArchive to work around a shortcoming in Arquillian. 

We test the mortgage calculator component just as before (Listing ALL-5), and the container management and injection of the environment entry just works. That’s why we say you can have as little or as much integration as you want. Since the end goal of testing is results, let’s see how Arquillian hands back control to the test runner to report the outcome of the test. 

              

The Verdict

When it’s all said and done, Arquillian shares the result of the test method with the original test runner and the rest of the lifecycle plays out just like with a unit test, where the results are reported to the command line, file or IDE plugin window.

As the test runner is wrapping things up, Arquillian will shut down the container if necessary. Again, the JUnit test plugin has no idea any remote communication or container lifecycle management is occurring. You can even mix Arquillian tests and plain unit tests in the same test suite. Arquillian’s minimal footprint is really the beauty of it.

Running the Arquillian Test

Notice that despite talking a lot about incontainer testing using Arquillian, we haven’t mentioned which container we are using to test. That’s because it’s a runtime decision.

Arquillian tests can be executed in any container that is compatible with the component model used in the test (that is, as long as the container has an Arquillian adaptor). The example above can be executed in any container that provides the EJB component model. You could run the test in an embedded EJB container for fast execution or easy debugging. You could also execute it in a full Java EE 6 container, perhaps to verify it works in the same container that you run your production code. Basically, your test is a free agent ;)

How do you select a container? Having code in hand, you’re likely itching to see it work. There are only three steps involved:

  • Add the Arquillian container adapter for the target container to the test classpath.
  • Add the container runtime (for an embedded container) or client (for a managed or remote container) to the test classpath.
  • Execute the test as usual, using a JUnit (or TestNG) plugin (Eclipse, IntelliJ, NetBeans, Ant, Maven, etc).

Arquillian selects the container to target based on what is available on the classpath. That’s the secret sauce. Arquillian requires no other configuration (though you can configure certain elements of the target container using the arquillian.xml descriptor). In Maven, the classpath is controlled using profiles. One profile is setup for each container we are targeting. Listing ALL-9 shows the profile for OpenEJB 3.1 embedded, the standalone EJB 3.0+container developed by Apache. Now we’ll run the tests in OpenEJB embedded by activating this profile using the command in Listing ALL-10. 

Here’s the really powerful part. We can run the exact same test on any container that supports EJB 3.0, such as JBoss AS 5 and above. Let’s assume there’s a Maven profile named arq-jbossas-managed defined in the pom.xml that includes the JBoss AS server manager library and corresponding Arquillian container adapter library. We can then run our test on JBoss AS by activating this profile when we run the Maven test goal:

Consult the Arquillian reference guide to learn more about integrating an Arquillian test suite into your build or IDE. In truth, though, there’s not much to it.

Now that you’ve got one Arquillian test under your belt, let’s take a quick tour of what else it can be used to test.

Where We’re Going, We Don’t Need EJB

All this talk about EJB may have left you with the impression that Arquillian is just for EJB testing. That’s certainly not the case. Neither Arquillian nor ShrinkWrap are tied to Java EE. Not its containers, archives or component model.

EJB is just one of the component models already supported. Similar integration can be offered for any container and (at the moment, Java) programming model, simplifying integration tests to three annotations (or more) and some ShrinkWrap packaging.

The Java EE 5 and Java EE 6 stacks (EJB, JPA, JTA, CDI, JMS, JAX-RS, JSF, Servlet, etc) are already well supported. Other component models supported include JSR-299 (CDI) standalone, OSGi and JSR-322 (JCA) standalone. Additional integrations are under development, such as JPA (standalone), Hibernate, Spring and JBoss MC . Suppor t for other JVM languages could open the door for Arquillian to drive integration tests beyond Java, such as Ruby on Rails. Arquillian is already being used to black box test JRuby-based TorqueBox applications using the client execution mode.

All of this is possible because Arquillian didn’t start out by trying to address a narrow problem. Instead of focusing on building a test infrastructure for a specific programming model like Servlets or EJBs, Arquillian focuses on providing a component model for your tests through an extensible set of SPIs.

Arquillian is extensible in nearly every way, so adding a new container and / or programming model is just a matter of implementing a handful of Arquillian SPIs. Figure ALL-3 shows how the SPI implementations fit into the Arquillian stack.

There’s nothing stopping you from rewriting one of the existing SPI implementations or container adapters if you want to change the behavior, though we encourage you to join the community and contribute your improvements for everyone’s benefit.

One of the ways Arquillian manages complexity as you move to higher level tests is to treat auxiliary testing frameworks, such as JSFUnit for testing JSF pages, as an extension of the tests’ component model.

Do you know what you have to do to integrate JSFUnit into your test suite? With Arquillian, it’s just a matter of adding a single JAR file to the classpath, arquillian-framework-jsfunit.jar. JSFUnit becomes part of the component model for your test. Expect similar transparent integrations with tools such as DBUnit, Selenium, HTTPUnit and so on. Arquillian will continue to surprise us because it’s truly an open test automation platform.

Learn before You Test

In learning that Arquillian is a testing framework, you may be inclined to write it off as something that’s only useful when you are ready to lay down tests for your application. In doing so, you overlook one of the most powerful and effective uses of Arquillian; as a learning tool.

It’s a common misconception that a developer can just start out on day one writing application code or even tests, especially when learning a new programming model, container or domain model.

Consider a building contractor who just purchased a new saw. Do you think he is going to head straight to the job site and start cutting material that costs $5/ft? Not likely. He’s first going to take the saw to his work bench and practice on scrap wood to get a feel for how it cuts. He’ll try it one way then another and so on until the saw becomes a natural extension of his hands. Only then is he going to head to the job site and start making “production” cuts. It’s the same way with the application you are going to work on. You want to figure things out before you start laying down production code.

Let’s say I’m teaching you a new programming model, perhaps EJB or CDI. The components you create get placed into a runtime and invoked as backend services. By themselves, they are not executable. So how can you see if they work? One solution is using some sort of user interface (UI) that can invoke the component. In a Java EE application, the UI is typically a web page rendered by a Servlet.

How many times have you created a web page with a button and a few inputs just to experiment with a backend service? And I bet you had to do more than just create the web page. You likely had to assemble a script to package the WAR and manually deploy it to the container. That’s a heck of a lot of work just to stand up a component. Now don’t forget to delete that web page so it doesn’t leave a back door open into your application!

Wouldn’t it be nice if you could invoke the component directly without having to do all that busy work? That’s precisely the infrastructure Arquillian provides for you. Perhaps you now appreciate that Arquillian’s utility goes well beyond writing tests. It’s an opportunity to explore the target platform and get comfortable writing code for that environment. It’s about figuring stuff out. It’s a framework for creating a “Hello world” at each step along your learning path. Combine that with the debugging capabilities of your IDE, which you can leverage since Arquillian tests can be launched from the test plugin in your IDE, and you can master a programming model in no time.

You can even try the same code on multiple containers to make sure that you aren’t relying on non-compliant (or buggy) behavior of a single container. Don’t stop there. Use Arquillian to create prototypes to share with the customer during the design phase. Nowhere does it say demos must be user interfaces. Your business logic may be performing calculations (such as our mortgage calculator), parsing text, reading and writing to the database and so forth. You can use Arquillian to show this functionality is working, for real. Arquillian even has Spock integration, so you can embrace the Behavior Driven Design (BDD) methodology.

My advice to you (as well as technical authors): use Arquillian as a learning device. You can keep those tests around or you can throw them away. Either way, the experience you gain from experimenting with Arquillian is timeless.

Integration Testing as it Should Be

Build free and in-container. That’s the way integration testing should be and it’s what makes Arquillian’s approach so different. Arquillian can make the transition from unit to integration smooth, without losing the ability to leverage the IDE and debugger, because:

• The setup is minimal.

• There is no base class to extend.

• You can leverage the native features of the testing framework.

• You run or debug the test just like a regular unit test.

• You have control over which classes go in the deployment.

• The test has access to the component model of the application.

• The test is portable to any container that provides the component model in use.

If you choose to adopt Arquillian now, there’s no doubt it will continue to grow to match your testing requirements. With its extensible SPIs, there’s no telling all the ways Arquillian may help enrich your test suite.

Go forth and test in-container!

Interested to learn more? Be sure to check out the follow-up article appearing in the April issue of NFJS, The Magazine. The article will describe how to design your tests so they run in multiple containers, then showcases numerous scenarios and technologies that Arquillian enables you to test with ease. By the end, you’ll be ready to test everything from managed beans to web pages.

You can also join me at JAX London to learn about the future of Java enterprise testing using Arquillian.

Author
DanAllen
As a Principal Software Engineer at JBoss, by Red Hat, Dan works as the community liaison and member of the Seam, Weld and Arquillian projects. He
Comments
comments powered by Disqus