days
1
3
hours
0
7
minutes
1
7
seconds
2
7
Consumer-driven contract testing with Pact

Microservices: Agreements must be kept

Tobias Bayer and Hendrik Still
Network image via Shutterstock

“Pacta sunt servanda“, or in English, “agreements must be kept“. What was true in the middle ages is mandatory in the modern world of software development. Utilising (API-) contracts that are defined by several partners instead of just one, microservice architectures can be tested and developed easily and efficiently.

“Microservices” – it’s one of the major buzzwords today when talking or writing about modern software architecture. It stands for an architectural style where business capabilities of an application are distributed among several small services. Each service runs in its own process and is typically responsible for exactly one clearly defined business function.

Services communicate via web technologies like HTTP/REST or over a message bus. A microservice application fulfils requirements by orchestration of several services. For example there might be microservices for description, price, reviews and stockpile in an application for managing products.

Challenges in testing

The advantages of a microservice architecture include scalability (of software and team as well), availability, independence in languages and technologies and increased flexibility. But such a distributed system also has disadvantages, like more complicated operation, code duplication, network latency, distributed transactions, additional interfaces and trickier testability. Solutions exist for all of these problems. In this article we want to further explore consumer-driven contracts as a possible solution for the disadvantages of additional interfaces and trickier testability.

When testing an application that consists of microservices, there are several special challenges in contrast to a monolithic architecture. Low-level tests (like unit tests) can be developed as usual. But for a microservice application to work well, it is always necessary for the services to interact. As soon as interfaces to other microservices come into play or responses from other microservices are necessary for testing the functionality of a microservice, you have to ask yourself how to provide these dependencies.

Usually you do not want to launch your complete service environment to test a single microservice. In doing so, many of the major advantages of microservices would be rendered invalid. You would no longer be able to develop, maintain and deploy your microservices independently. A single service should be tested with as much isolation as possible. On the other hand, you want to make sure your services interact well with each other. This dilemma can be solved using the consumer-driven contract pattern.

Provider contracts vs. consumer contracts

Consumer contracts are not a new concept. In any case, they get more important again regarding microservice architectures. A consumer contract defines the expected structure of a response from the point of view of a service caller.

In contrast, provider contracts are better known. Here a service describes which responses it will deliver for defined requests. For example, this can be the WSDL of a web service or the Swagger documentation of a REST service. Provider contracts often constitute an ample documentation of all the abilities of a service. The providing service creates this interface contract by itself.

In contrast, a consumer contract is the specification of an excerpt from the complete specification of service functionality that is required by a denoted caller for fulfilling its own requirements.

Figure 1: Consumer contracts and consumer-driven contract

Figure 1: Consumer contracts and consumer-driven contract

The consumer-driven contract of a service is composed of all the consumer contracts of its callers (Figure 1). This implies that a service might not have to implement its complete provider contract because at a given time no caller exists for a specific functionality. Such a microservice architecture supports the YAGNI principle. A service only must implement specified or planned functionality as soon as a caller of this functionality appears in its consumer-driven contract.

This method is especially useful for environments where the implementation of consumer and provider is in hand and the services are both implemented by teams belonging to the same organization.

Testing with consumer contracts

Consumer contracts can be used for testing in two ways. When testing a consumer, a provider stub can be created which delivers the expected responses to the consumer. When testing a provider, requests can be generated out of the contract and fired against the provider’s interface for checking the responses (Figure 2). If both tests are successful, consumer and provider can be deployed to production. If the provider test fails, the provider cannot be deployed because it does not fulfill all the requirements (any more). This situation might occur when a provider removes a field from the response structure while there is still a consumer depending on that field.

If consumer tests fail, a consumer cannot cope (any more) with the responses a provider will still deliver and must not be deployed as well.

Figure 2: Testing with Consumer Contracts

Figure 2: Testing with Consumer Contracts

This strategy allows testing the interaction of services independently without having to launch all interacting services. These tests usually provide quick feedback and also retain high confidence about the integrity of the whole application.

For being able to rely on this fact all consumers must keep their contracts up to date and provide them to the service providers for their tests without delay. This can be done using a central repository where all consumers publish their contracts and which is queried before testing the providers.

Implementation in practice

How can consumer contract tests be implemented in practice? Inside a monolith one could simply use JUnit tests. In the case of microservices things are a bit harder because not only single objects and method calls have to be mocked but complete REST services called via HTTP. There are open source tools available that can solve this problem. One of the better known representatives is Pact. Originally developed in Ruby it has been ported to several platforms including JavaScript, .NET and the JVM.

We will use a simple application as an example of how to test the interface between microservices with Pact-JVM.

Use Case

We will test the interface between the ProductDetailsService which provides detailed information, such as a description, about a product and the ProductService which queries for this information. This makes theProductService the consumer and the ProductDetailsService the provider. Communication takes place via a REST interface, provided by the ProductDetailsService.

For creating and consuming a service we use Spring Boot in our example and create a Gradle project for each of the services. For taking a deeper look into the code both projects have been placed on GitHub.

Implementing the consumer

For we are utilizing the consumer-driven approach, we are going to develop the consumer (ProductService) first. The most interesting part of the implementation is the call to the REST interface because we want to test the communication between the consumer and the provider. As you can see in Listing 1 this interface is encapsulated inside the class ProductDetailsFetcher and only contains the single method fetchDetails. It uses a RestTemplate for loading the resource from the given URI and creating an instance of ProductDetails out of the response data. Error handling was omitted in Listing 1 to keep the code short and simple.

public class ProductDetailsFetcher {
  public ProductDetails fetchDetails(URI productDetailsUri) {
    return new RestTemplate().getForObject(productDetailsUri,
    ProductDetails.class);
  }
}

But how to test this part of the application? The expected interface has to be defined from the point of view of the consumer because no provider exists yet. This is where Pact comes into play. We create the class ProductDetailsServiceConsumerTest (Listing 2). It inherits from ConsumerPactTest which is provided by Pact-JVM. The test demands an implementation of createFragment. This method returns a PactFragmentcontaining all the interactions that occur during the test, i. e. every request and the expected responses. The interactions can be defined easily by using the ConsumerPactBuilder DSL. The builder expects a method call to uponReceiving, followed by willRespondWith. In our example we expect the provider to reply to a GET request to /productdetails/1 with status 200 and a defined JSON body.

By creating a PactFragment we defined the expectation of the consumer against the provider. Now we can test if our implementation of ProductDetailsFetcher fulfils the contract by implementing runTest. This method is called after Pact has started the provider stub based on the PactFragment. The address of the stub is injected via the parameter url.

Testing is done as usual by executing the components and checking the results with asserts. If the consumer executes an unknown request the test will fail. It is also tested if the consumer can cope with the response from the stub. For collating the contract to a consumer and provider their names have to be defined via the methodsproviderName and consumerName. We have to include the dependency testCompile au.com.dius:pact-jvm-consumer-junit_2.11:2.1.7 in the Gradle project ProductDetailsService in order to be able to execute the test.

public class ProductDetailsServiceConsumerTest extends ConsumerPactTest {
  @Override
  protected PactFragment createFragment(ConsumerPactBuilder.PactDslWithProvider builder) {
    Map<String, String> headers = new HashMap<>();
    headers.put("Content-Type", "application/json;charset=UTF-8");
    return builder.uponReceiving("a request for product details")
    .path("/productdetails/1")
    .method("GET")
    .willRespondWith()
    .headers(headers)
    .status(200)
    .body("{\"id\":1,\"description\":\"This is the description for product 1\"}")
    .toFragment();
  }
  @Override
  protected String providerName() {
    return "Product_Details_Service";
  }
  @Override
  protected String consumerName() {
    return "Product_Service";
  }
  @Override
  protected void runTest(String url) {
    URI productDetailsUri = URI.create(String.format("%s/%s/%s", url, "productdetails", 1));
    ProductDetailsFetcher productDetailsFetcher = new ProductDetailsFetcher();
    ProductDetails productDetails = productDetailsFetcher.fetchDetails(productDetailsUri);
    assertEquals(productDetails.getId(), 1);
  }
}

The Pact file

The ProductDetailsServiceConsumerTest includes all information to be able to create a contract for the interface and test its own components which access this interface. The test can now be executed by running gradle test inside the directory of the ProductService. This starts the provider stub and the components are tested against it. Additionally, Pact creates a JSON file during the test. This is the Pact file. In our example it can be found at ./target/pacts/Product_Service-Product_Details_Service.json. This file is the actual contract and can be used for checking if the provider fulfills the contract. It contains all requests and responses that were defined in the consumer test.

Listing 3 shows the Pact file for the contract between ProductDetailsService and ProductService. This platform independent file can also be used for microservices that have been developed in other programming languages than Java. Thus, Pact supports the development of polyglot applications, which is a frequently mentioned advantage of the microservice architectural style.

{
  "provider" : {
    "name" : "Product_Details_Service"
  },
  "consumer" : {
    "name" : "Product_Service"
  },
  "interactions" : [ {
    "description" : "a request for product details",
    "request" : {
      "method" : "GET",
      "path" : "/productdetails/1"
    },
    "response" : {
      "status" : 200,
      "headers" : {
        "Content-Type" : "application/json;charset=UTF-8"
      },
      "body" : {
        "id" : 1,
        "description" : "This is the description for product 1"
      }
    }
  } ],
  "metadata" : {
    ...
  }
}

Implementing the Provider

The Pact file represents the consumer contract. We can use it to implement the matching provider. The class Application inside the ProductDetailsService project creates a REST interface for this purpose. This is quite simple using Spring (Listing 4). The method fetchProductDetails will return a serialised ProductDetails object for every request. For this simple example all products have a static text for their description.

@RestController
public class Application {
  @RequestMapping(value = "/productdetails/{id}", method = RequestMethod.GET)
  public ProductDetails fetchProductDetails(@PathVariable final long id) {
    return new ProductDetails(id, "This is the description for product " + id);
  }
  public static void main(final String[] args) {
    SpringApplication.run(Application.class, args);
  }
}

To check if our provider fulfills the given contract, the requests from the Pact file have to be executed against the REST interface. The contract is fulfilled if the service replies as defined in the Pact file. Including the verification in the build process of the service is recommended. Pact offers a Gradle plugin for this purpose. It is included with apply plugin: au.com.dius.pact in the Gradle project of the ProductDetailsService. Listing 5 shows how to define a pact using the plugin. Locations of the provider service and the Pact file have to be defined. Besides using local files you can also access remote files with url(). This allows providing Pact files on a central web server for example.

We have to start our service before Pact can test it. With Spring Boot we can simply type gradle bootRun to start it. As soon as the service is reachable we start the verification with gradle pactVerify. We have now tested the interface of the ProductDetailsService decoupled from the ProductService.

pact {
  serviceProviders {
    productDetailsServiceProvider {
      protocol = 'http'
      host = 'localhost'
      port = 10100
      path = '/'
      hasPactWith('productServiceConsumer') {
        pactFile = file("../product-service/target/pacts/Product_Service-Product_Details_Service.json")
      }
    }
  }
}

Conclusion

Consumer-driven contracts mitigate one of the largest challenges in microservice architectures. The danger of losing the advantages of independent deployability by having strongly coupled tests is reduced a lot by using this approach of interface specification and test strategy. Despite the decoupling of tests high confidence about the correct interaction of services remains. Our example showed that Pact is a rather mature tool for supporting consumer contract testing in (polyglot) microservice environments.

Links:

https://github.com/inovex/pact-example

Author

Tobias Bayer and Hendrik Still

Tobias Bayer is a software architect and senior developer at inovex GmbH. He focuses on the development of scalable web applications in Java and mobile apps for iOS. Additionally he devotes himself to functional programming in Clojure.

 

Hendrik Still is a student employee in the area of application development at inovex GmbH. At the moment he is researching about challenges in microservice architectures and implementation with Docker.


Comments
comments powered by Disqus