Arquillian: A Component Model for Integration Testing
Why the best intentions to write tests are so often foiled and how to prevent it.
If you’ve ever attended a No Fluff Just Stuff event, you likely
came away with the message ingrained in your head that testing is
important… correction, crucial, and you should do it. Period.
As Venkat Subramaniam, a long-time speaker on the No Fluff Just
Stuff symposium series, often reminds audiences:
Testing is the engineering rigor of software
In other words, if you’re going to carry around the title
Software Engineer, you had better put ample effort into engineering
a plethora of tests.
But this article is not about the importance of testing. At this
point, we’ll assume its importance is well established. What I’m
here to address is why the best intentions to write tests are so
often foiled and how to prevent it.The main obstacle is that your
tests live in a different world than your application. To overcome
this discrepancy, I’ll introduce you to the idea of having a
component model for your tests, an architecture that is provided by
Arquillian. Arquillian is a container-oriented testing
framework developed in Java that brings your test to the runtime
rather than requiring you to manage the runtime from the test. This
strategy eliminates setup code and allows the test to behave more
like the components it’s testing. The end result is that
integration testing becomes no more complex than unit testing.
You’ll believe it when you see how much power you get out of a
simple Arquillian-based test.
I want to start by studying this transition from unit to
integration tests and establish an understanding as to why making
this transition knocks the wind out of the sails of so many testing
initiatives. We’ll then look at how a component model for tests can
ease this transition. Of course, we’ll look at some examples of how
to use Arquillian along the way.
Why Don’t We Do It?
Why don’t we test? (For those of you who test religiously, think
about a scenario when you’ve hesitated or decided to not write a
test). I’ve asked this question at several NFJS events and it
doesn’t take long before I hear all the responses I’m seeking:
• Too hard
• Not possible
• Not enough time
• Too slow
• Not enjoyable
• No proper tools
Unless you have an enlightened manager who embraces agile
software practices (perhaps because she attended a NFJS event), you
likely fear her walking by to find you working on something that
isn’t the application code (i.e., tests). Worse is if she catches
you writing your own testing framework, a tangent even by an
enlightened manager’s standards. So we can generalize why testing
gets the axe by saying there isn’t enough time.
The reason there isn’t enough time is because we don’t have the
proper tools. I’m not talking about those fancy graphical tools.
Those likely just compound the problem (because they require
additional learning.) What we’re missing is a tool in the form of a
programming model that makes us efficient at solving the other
testing challenges listed above.
But wait, isn’t that what a unit testing framework is for? It is
– to a point. Unit testing frameworks, such as JUnit and TestNG,
provide a programming model for writing unit tests. They don’t
specifically address integration testing concerns (though they can
drive integration tests).What lies between these two
classifications is a large chasm.
The Testing Bandgap
Unit tests are fine-grained, simple and fast. They exercise
individual components while ignoring the rest of the system. Since
the test plugins for IDEs were designed with this type of test in
mind, executing them in the IDE is easy.
A majority of books and articles that cover testing demonstrate
the concept with unit testing. Indeed, it looks so simple and
promises to keep regression errors from creeping in that you feel
motivated to run back to your application and start adding tests.
But when you give it a try, you soon discover it’s harder than it
Where things fall apart is when you transition to integration
testing. Integration tests are coarse-grained and generally
perceived as complex and slow (though this article will hopefully
change that perception in your mind).They exercise multiple
components from a subset of the application, verifying their
communication and interactions.
Once you start involving multiple components, the task of
isolating the code being tested from the rest of the system becomes
more difficult. On top of that, getting components to interact
often requires bootstrapping the environment they need in order to
function (e.g., dependency injection). All of a sudden, the task of
writing a test isn’t so simple.
I describe the sharp increase in configuration complexity
between unit and integration testing as the testing bandgap,
represented in Figure ALL-1.
The shift to integration testing requires a significant increase
in mental effort when any non-trivial setup is required (which is
typical). This interrupts your flow.
Work stops, your browser history becomes flooded with search
results, the build script multiplies and the code-build-test cycle
becomes a code-build…-catch up on social timeline-test cycle.
Testing becomes tedious and costly, two things that directly lead
to its demise. We’ve got to find a way past this complexity barrier
because integration is all around us.
Don’t Mock, Integrate
In a paper titled “What Is Software Testing? And Why Is It So
Hard?” James Whittaker offers a well-defined classification of the
various levels of testing (i.e., unit, integration, system,
functional, etc). However, he struggles with the popular question
of when and how to draw the line between unit and integration
Unit testing sometimes requires the construction of throwaway
driver code and stubs and is often performed in a debugger.
Some components can’t be unit tested because they weren’t
designed with testing in mind, meaning they have lots of
dependencies that are hard to mock. Even if they can be mocked, how
much effort should you put into creating mocks? And how many mocks
are permitted before a test loses its value?
I believe that anything more than a couple of lines of throwaway
driver code and stubs is time thrown away. Mocks are not a
substitute for integration testing.
We also shouldn’t be satisfied with only being able to use a
debugger during unit testing. Debuggers are crucial for
productivity. Why can’t we use them for integration testing as
well? Or why is it so uncommon? Are integration tests that
That brings us back to the lack of a programming model for
writing and executing integration tests (and on up, including
system and functional tests).We don’t want to beat around the bush
by using mocks. We just want integration tests to be:
• as easy as writing unit tests
• executed in the IDE (incremental builds, debugging, etc)
• ramped up gradually (instead of “big bang” integration)
By providing a component model for tests, Arquillian eliminates
the testing bandgap and flattens the configuration complexity curve
as it graduates from unit to integration tests and beyond. The
result is the testing continuum shown in Figure ALL-2. With
Arquillian in your toolbox, you’ll be able to move comfortably
between unit and integration testing.
Now that you’ve had your day’s worth of theory, let’s step
through an example to see where a component model fits with testing
and learn how Arquillian is able to leverage it to maintain this
Graduating from Unit to Integration Testing and
We’ll work from a unit test to an integration test to give you a
true appreciation for just how smooth Arquillian makes this
transition. When you put them side-by-side, the Arquillian test is
going to look very similar to the unit test, and that’s the
1 Arquillian supports both JUnit 4 and TestNG 5
A Unit to Test
To keep it simple, we’ll make our unit to test a calculator
class. Basic addition isn’t very interesting as a service, though,
so let’s make it a calculator that computes the monthly payment
amount for a fixed term mortgage. The example class is shown in
Listing ALL-1. Currency calculations can get a little dicey in Java
if you don’t take into account precision, scale and rounding. So
it’s worthy of testing. Let’s write a JUnit 4-based unit test case
(Listing ALL-2) that ensures our calculator isn’t going to be the
source of any bank errors.
The test case in Listing ALL-2 can be executed using any JUnit 4
Notice that the test case is initiating MortgageCalculator
itself. If this were just a class from a library, that would be
reasonable. But let’s assume that the class functions as an EJB
component (i.e., a service). EJB 3.1 makes this shift as simple as
adding a single annotation to the class, as we’ve done by adding
the @Stateless annotation in Listing ALL-3.
Don’t be turned off by the use of EJB for this example. I’ve
selected EJB because it’s a recognizable component model. Rest
assured, Arquillian can support any programming model, not just
EJB. Our unit test will still work with this annotation present.
That’s because all the logic is self-contained in the business
method and the component doesn’t use any container features. But
let’s put that fact aside for the moment and assume that we do want
to test this class acting in its role as a component. We want to
test an EJB instance.
The definition James Whittaker offers for software testing in
the aforementioned paper applies well here (emphasis mine):
Software testing is the process of executing a software
system to determine whether it matches its specification and
executes in its intended environment.
Unit testing can help test inputs and outputs, but what it lacks
is a mechanism by which we can execute a component in its intended
environment. That’s the way other components will use it and we
want to make sure that part works. We want to bridge theory and
practice and test the component for real. We want integration
That brings us to the question, “How do you test an EJB?” We
need to start an EJB container and get it to create an instance and
hand it to us. That means we need a container running in our test
case. Here’s where the component model for tests starts to come
You may have read about the embedded EJB container that was
introduced in EJB 3.1 primarily for the purpose of testing. That’s
good news! Using it would certainly satisfy the requirement of
getting an EJB container setup in our test. Listing ALL-4 shows an
example of how to boot the embedded EJB 3.1 container and use it to
lookup an instance of our mortgage calculator component:
At first, the approach in Listing ALL-4 seems pretty reasonable
(and a significant improvement over the EJB of old). But there are
a number of problems with it:
You need to add before/after logic so the container is only
created once per suite because startup isn’t instantaneous. You
have to look up the component manually, rather than using the more
convenient @EJB field/method injection.
The embedded EJB container is going to be scanning all over your
project for EJBs – maybe not want you want.
The test is relying on a JNDI name for the component that is
specific to the embedded environment.
An embedded environment isn’t the real deal.
The test is tied to the embedded EJB container.
Let’s step away from the embedded EJB container for a moment and
look at a much simpler, declarative style that Arquillian’s
component model brings to this test.
There’s A Component Model in My Test!
Anxious to see what a component model for a test looks like?
Direct your attention to Listing ALL-5. You should immediately
recognize several differences in the Arquillian version of this
test. But, before addressing what’s present, let’s look at what
isn’t. Notice there are no setup or tear down methods. We are going
to push all the work of starting and stopping the container to
Arquillian. And no controller code means a portable test. More on
that later. The other three differences in this test are stamped by
their corresponding annotations:
3. @EJB (and similar)*
* The @EJB annotation represents the link that Arquillian is
providing to the application component model. If the application
were employing JSR-299, you could use @Inject to get a reference to
the component to test. Arquillian is not tied to
Let’s study how each of these annotations contribute to the
test’s component model.
The Arquillian Test Runner
Scanning from top to bottom, the first thing you’ll notice is
that the test class is now annotated with
@RunWith(Arquillian.class). This annotation describes its purpose
The test class is run with Arquillian. Delegation to the
Arquillian runner occurs immediately after the test is launched. As
the JavaDoc for @RunWith explains, JUnit will invoke the class
[that @RunWith] references to run the tests in [the annotated]
class instead of the runner built into JUnit.
The Arquillian class is also a listener that hooks into the
JUnit lifecycle to provide in-container testing functionality. It
handles three responsibilities:
- Control the lifecycle of the target container (i.e.,
- Orchestrate deployment of test artifacts to the container.
- Execute tests inside the container, giving them access to the
Don’t worry, the delegation to Arquillian won’t change how you
run the test. All your standard JUnit plugins will still function.
They have no idea any of this extra activity is going on under the
covers. As far as they are concerned, this is just a plain old test
The second difference is the @Deployment method. Since
Arquillian is a container-oriented testing framework, it needs to
be able to give the container something it can digest. For example,
Java EE containers accept EARs, WARs and (EJB) JARs. In general, we
can say that containers speak in archives.
In the past, integration testing (or perhaps system testing,
depending on where you draw the line) meant having to rely on a
build process to assemble an archive to deploy. Arquillian takes a
different approach. Rather than having to package these archives
somewhere on the file system, Arquillian leverages ShrinkWrap to
allow you to produce the archive within the test itself. ShrinkWrap
is a fluent API for creating archives as Java objects that exist in
memory. Within the @Deployment method, the developer hand picks
classes, resources, libraries and/or preexisting archives needed by
the test, stuffs them into a ShrinkWrap archive and returns the
result. Arquillian then takes over and begins the process of
deploying the archive to the container.
The immediate benefit of using ShrinkWrap is that we don’t have
to rely on a build process to create the test archive that we are
going to pass to the container. The test essentially prepares its
own archive. Another benefit of using ShrinkWrap is that Arquillian
can leverage its container integrations to more efficiently deploy
the archive without having to write it out to the file system (not
available with all containers).
After reading the archive from the @Deployment method,
Arquillian appends its own resources, such as the test class, the
test framework and extensions, and performs any other
post-processing necessary. If the target container is managed by
Arquillian, and the container is not yet started, Arquillian will
setup and start the container (once per test suite). Once the
container is ready, the Arquillian and ShrinkWrap adaptors for the
container work together to translate the ShrinkWrap archive into a
resource the container can read, perhaps by exporting it to a file
or passing a stream to the container’s deployment API.
Once the test archive is deployed, depending on the type (or
profile) of the container, Arquillian may use either a remote or
local protocol to invoke the test methods. Containers are a topic
for another article, so we’ll just briefly cover the different
types so that you understand how the test gets executed.
4 Test enrichment is not available in the
Standalone containers, such as the embedded EJB container, are
executed in the same thread (and JVM) as the test runner. In this
case, Arquillian employs a local protocol, which means it simply
intercepts the test invocation, enriches the test case instance
(which we’ll talk about next), and allows execution to proceed.
Containers with a web component, such as a Java EE container,
require Arquillian to work over a remote protocol (the most common
being HTTP). Arquillian forwards execution of each test method to
the server, typically communicating with an HTTP Servlet that
Arquillian plants into the test archive before it gets deployed. On
the remote end, Arquillian bootstraps the test runner, creates a
new instance of the test case, enriches it, invokes the test method
and returns the outcome of the test, including any exceptions, back
to the client (typically via HTML).
The final link in the chain is bridging the test case to the
application’s component model. That’s the role of the @EJB
annotation in this example.
The Component Model Link
In Java EE, the @EJB annotation is used as a hint on a field (or
setter method) of an EJB to tell the container to assign (or
inject) a reference to another EJB. Only, in this case, it’s being
used within the test class. And it’s Arquillian that’s doing the
injecting. Arquillian intercepts the test invocation and satisfies
the injection points (fields, setter methods and test method
arguments) using a model consistent with the container being used.
For instance, in a test case using JSR-299, Contexts and Dependency
Injection for Java EE (CDI), Arquillian will satisfy the fields and
methods annotated with @Inject. This step is called test
By honoring the @EJB injection points in the test class, we’ve
effectively brought the component model used in the application to
the test environment. The test case is now one with the
application. It speaks the same language as other consumers of the
EJB we are testing. That’s important because it simplifies the
test, removes the use of a potentially non-portable JNDI name and
allows us to test the class as an EJB component running in its
This link is not just about dependency injection, though. It’s
about a shared component model made possible by running the test
inside the container. There, the test has access to any resource
the container provides. In a Java EE container, that includes
resources in the naming directory (JNDI), environment entries, data
sources, JMS destinations and so forth. In fact, these resources
can be injected directly into the test using the @Resource
annotation, also supported by Arquillian’s EJB test
Let’s consider a scenario that relies on a containerprovided
resource. Assume we want to change the monthly payment calculation
in the mortgage calculator so it reads the interest rate from an
environment entry. First, we add the @Resource injection to one of
the component’s fields, shown in Listing
Next, we set the value of the interest rate environment variable
using an ejb-jar.xml descriptor, which we create in the test
resources directory. The resulting structure of this descriptor is
displayed in Listing ALL-7.
We then package this descriptor into a test archive using
ShrinkWrap, as shown in Listing ALL-8. Note we’ve switched to a
WebArchive to work around a shortcoming in
We test the mortgage calculator component just as before
(Listing ALL-5), and the container management and injection of the
environment entry just works. That’s why we say you can have as
little or as much integration as you want. Since the end goal of
testing is results, let’s see how Arquillian hands back control to
the test runner to report the outcome of the
When it’s all said and done, Arquillian shares the result of the
test method with the original test runner and the rest of the
lifecycle plays out just like with a unit test, where the results
are reported to the command line, file or IDE plugin window.
As the test runner is wrapping things up, Arquillian will shut
down the container if necessary. Again, the JUnit test plugin has
no idea any remote communication or container lifecycle management
is occurring. You can even mix Arquillian tests and plain unit
tests in the same test suite. Arquillian’s minimal footprint is
really the beauty of it.
Running the Arquillian Test
Notice that despite talking a lot about incontainer testing
using Arquillian, we haven’t mentioned which container we are using
to test. That’s because it’s a runtime decision.
Arquillian tests can be executed in any container that is
compatible with the component model used in the test (that is, as
long as the container has an Arquillian adaptor). The example above
can be executed in any container that provides the EJB component
model. You could run the test in an embedded EJB container for fast
execution or easy debugging. You could also execute it in a full
Java EE 6 container, perhaps to verify it works in the same
container that you run your production code. Basically, your test
is a free agent ;)
How do you select a container? Having code in hand, you’re
likely itching to see it work. There are only three steps
- Add the Arquillian container adapter for the target container
to the test classpath.
- Add the container runtime (for an embedded container) or client
(for a managed or remote container) to the test classpath.
- Execute the test as usual, using a JUnit (or TestNG) plugin
(Eclipse, IntelliJ, NetBeans, Ant, Maven, etc).
Arquillian selects the container to target based on what is
available on the classpath. That’s the secret sauce. Arquillian
requires no other configuration (though you can configure certain
elements of the target container using the arquillian.xml
descriptor). In Maven, the classpath is controlled using profiles.
One profile is setup for each container we are targeting. Listing
ALL-9 shows the profile for OpenEJB 3.1 embedded, the standalone
EJB 3.0+container developed by Apache. Now we’ll run the tests in
OpenEJB embedded by activating this profile using the command in
Here’s the really powerful part. We can run the exact same test
on any container that supports EJB 3.0, such as JBoss AS 5 and
above. Let’s assume there’s a Maven profile named
arq-jbossas-managed defined in the pom.xml that includes the JBoss
AS server manager library and corresponding Arquillian container
adapter library. We can then run our test on JBoss AS by activating
this profile when we run the Maven test goal:
Consult the Arquillian reference guide to learn more about
integrating an Arquillian test suite into your build or IDE. In
truth, though, there’s not much to it.
Now that you’ve got one Arquillian test under your belt, let’s
take a quick tour of what else it can be used to test.
Where We’re Going, We Don’t Need EJB
All this talk about EJB may have left you with the impression
that Arquillian is just for EJB testing. That’s certainly not the
case. Neither Arquillian nor ShrinkWrap are tied to Java EE. Not
its containers, archives or component model.
EJB is just one of the component models already supported.
Similar integration can be offered for any container and (at the
moment, Java) programming model, simplifying integration tests to
three annotations (or more) and some ShrinkWrap packaging.
The Java EE 5 and Java EE 6 stacks (EJB, JPA, JTA, CDI, JMS,
JAX-RS, JSF, Servlet, etc) are already well supported. Other
component models supported include JSR-299 (CDI) standalone, OSGi
and JSR-322 (JCA) standalone. Additional integrations are under
development, such as JPA (standalone), Hibernate, Spring and JBoss
MC . Suppor t for other JVM languages could open the door for
Arquillian to drive integration tests beyond Java, such as Ruby on
Rails. Arquillian is already being used to black box test
JRuby-based TorqueBox applications using the client execution
All of this is possible because Arquillian didn’t start out by
trying to address a narrow problem. Instead of focusing on building
a test infrastructure for a specific programming model like
Servlets or EJBs, Arquillian focuses on providing a component model
for your tests through an extensible set of SPIs.
Arquillian is extensible in nearly every way, so adding a new
container and / or programming model is just a matter of
implementing a handful of Arquillian SPIs. Figure ALL-3 shows how
the SPI implementations fit into the Arquillian stack.
There’s nothing stopping you from rewriting one of the existing
SPI implementations or container adapters if you want to change the
behavior, though we encourage you to join the community and
contribute your improvements for everyone’s benefit.
One of the ways Arquillian manages complexity as you move to
higher level tests is to treat auxiliary testing frameworks, such
as JSFUnit for testing JSF pages, as an extension of the tests’
Do you know what you have to do to integrate JSFUnit into your
test suite? With Arquillian, it’s just a matter of adding a single
JAR file to the classpath, arquillian-framework-jsfunit.jar.
JSFUnit becomes part of the component model for your test. Expect
similar transparent integrations with tools such as DBUnit,
Selenium, HTTPUnit and so on. Arquillian will continue to surprise
us because it’s truly an open test automation platform.
Learn before You Test
In learning that Arquillian is a testing framework, you may be
inclined to write it off as something that’s only useful when you
are ready to lay down tests for your application. In doing so, you
overlook one of the most powerful and effective uses of Arquillian;
as a learning tool.
It’s a common misconception that a developer can just start out
on day one writing application code or even tests, especially when
learning a new programming model, container or domain model.
Consider a building contractor who just purchased a new saw. Do
you think he is going to head straight to the job site and start
cutting material that costs $5/ft? Not likely. He’s first going to
take the saw to his work bench and practice on scrap wood to get a
feel for how it cuts. He’ll try it one way then another and so on
until the saw becomes a natural extension of his hands. Only then
is he going to head to the job site and start making “production”
cuts. It’s the same way with the application you are going to work
on. You want to figure things out before you start laying down
Let’s say I’m teaching you a new programming model, perhaps EJB
or CDI. The components you create get placed into a runtime and
invoked as backend services. By themselves, they are not
executable. So how can you see if they work? One solution is using
some sort of user interface (UI) that can invoke the component. In
a Java EE application, the UI is typically a web page rendered by a
How many times have you created a web page with a button and a
few inputs just to experiment with a backend service? And I bet you
had to do more than just create the web page. You likely had to
assemble a script to package the WAR and manually deploy it to the
container. That’s a heck of a lot of work just to stand up a
component. Now don’t forget to delete that web page so it doesn’t
leave a back door open into your application!
Wouldn’t it be nice if you could invoke the component directly
without having to do all that busy work? That’s precisely the
infrastructure Arquillian provides for you. Perhaps you now
appreciate that Arquillian’s utility goes well beyond writing
tests. It’s an opportunity to explore the target platform and get
comfortable writing code for that environment. It’s about figuring
stuff out. It’s a framework for creating a “Hello world” at each
step along your learning path. Combine that with the debugging
capabilities of your IDE, which you can leverage since Arquillian
tests can be launched from the test plugin in your IDE, and you can
master a programming model in no time.
You can even try the same code on multiple containers to make
sure that you aren’t relying on non-compliant (or buggy) behavior
of a single container. Don’t stop there. Use Arquillian to create
prototypes to share with the customer during the design phase.
Nowhere does it say demos must be user interfaces. Your business
logic may be performing calculations (such as our mortgage
calculator), parsing text, reading and writing to the database and
so forth. You can use Arquillian to show this functionality is
working, for real. Arquillian even has Spock integration, so you
can embrace the Behavior Driven Design (BDD) methodology.
My advice to you (as well as technical authors): use Arquillian
as a learning device. You can keep those tests around or you can
throw them away. Either way, the experience you gain from
experimenting with Arquillian is timeless.
Integration Testing as it Should Be
Build free and in-container. That’s the way integration testing
should be and it’s what makes Arquillian’s approach so different.
Arquillian can make the transition from unit to integration smooth,
without losing the ability to leverage the IDE and debugger,
• The setup is minimal.
• There is no base class to extend.
• You can leverage the native features of the testing
• You run or debug the test just like a regular unit test.
• You have control over which classes go in the deployment.
• The test has access to the component model of the
• The test is portable to any container that provides the
component model in use.
If you choose to adopt Arquillian now, there’s no doubt it will
continue to grow to match your testing requirements. With its
extensible SPIs, there’s no telling all the ways Arquillian may
help enrich your test suite.
Go forth and test in-container!
Interested to learn more? Be sure to check out the follow-up
article appearing in the April issue of NFJS, The Magazine. The article will describe how to
design your tests so they run in multiple containers, then
showcases numerous scenarios and technologies that Arquillian
enables you to test with ease. By the end, you’ll be ready to test
everything from managed beans to web pages.
You can also join me at JAX London
to learn about the future of Java enterprise testing using