JAX London 2014: A retrospective
DIY Gradle Plugin Development

Plugging into Gradle Plugins

TimBerglund
gradle

Tim Berglund explores build automation tool Gradle and how it can be enhanced to manage changes in relational databases, via the use of Gradle plugins

Gradle uses a Groovy-based DSL for its build configuration files. Since Gradle build files are also Groovy scripts, build masters have essentially unlimited flexibility to extend Gradle in whatever ways they want, just by writing Groovy code in the build file. While this kind of power is attractive, and while every Gradle user is likely to sketch some Groovy code into a build at some point, this isn’t a sustainable way to grow the functionality of the build. The only thing worse than a build tool that resists your customizations is one that gives you a blank slate for as much ill-considered, poorly-factored, and untested code as you can write. Gradle has a better way.

Much of the out-of-the-box functionality provided by Gradle is given through the means of Gradle plugins. Building Java code, packaging executable applications, or running code quality analysis through Sonar is as easy as applying a core plugin with a single line of code. Knowing these plugins is an essential part of using Gradle for ordinary builds, and they are well-documented in the Gradle User Guide that comes with every release.

But what happens when your needs outgrow the functionality of the core plugins? What happens when you, like any other build master building real applications, need to write code to do things the creators of Gradle couldn’t anticipate? When you get to this point, it’s time to write a plugin.

In this article, we’ll enhance Gradle with the ability to manage changes in a relational database. Our goals will be to extend the Gradle DSL with a small new syntax, while keeping imperative Groovy code entirely out of the build.

  

The Liquibase Plugin

Liquibase is an open-source database refactoring tool. It provides a means for developers to collaborate on the design of a relational database schema, and to publish a time-ordered stream of schema changes to multiple database instances. It’s very good at keeping multiple developer sandboxes in sync, and for providing an automated mechanism for tracking and deploying database change to shared staging, QA, and production servers. A tool like this is virtually begging to be scripted into a well-constructed build. Liquibase is a Java-based tool with a convenient command-line wrapper built in, but even “convenient” command-line Java is not all that pleasant to run. Liquibase needs a Gradle plugin.

Applying a core plugin is easy: you simply include the statement apply plugin: ‘plugin-id’ in your build script. Because core plugins are a part of the Gradle release, their IDs are reg- istered and their classes are available to the Gradle classload- er. Applying an external plugin is very similar, but we have to take one more step to tell Gradle where to find the plugin.

The buildscript method lets us add jars to the classpath of the build itself, and even declare the repositories Gradle should use to fetch those jars. To include the liquibase plugin in a build, we’d do the following (listing 1).

Listing 1: Applying the Liquibase plugin using the +buildscript+ method.

 

apply plugin: ‚liquibase‘
buildscript { 
        repositories {
                mavenCentral() 
        }
        dependencies {
                classpath ‚com.augusttechgroup:gradle-liquibase-plugin:0.5.1‘
        } 
}

 

Merely applying the liquibase plugin with no other preparation wouldn’t work, since Gradle doesn’t natively know a plugin by that name. To tell Gradle where to find it, we have to declare it as a dependency. The dependencies declaration points to a Maven-style vector which happens to resolve in Maven Central. Note that the dependency is as- signed to the classpath configuration; all dependencies de- clared in the buildscript block should go here. This indicates that they will be made available in the classpath of the build script itself.

The rest of the build file is now free to make use of the Liquibase plugin as it sees fit. This may include using any number of new tasks, task types, project properties, and do- main objects introduced by the plugin. The full details of the

Liquibase plugin are beyond our present scope, but you can follow the project on GitHub for more documentation as it becomes available.

The Plugin API

For all the power Gradle plugins bring us, creating one is sur- prisingly easy. We’ll study the code of the Liquibase plugin as an example. The Liquibase plugin is written as a standalone project of its own, which builds to its own JAR and is deployed under its own artifact ID to Maven Central. This is the preferred way to write and distribute open source plugins, as it makes them very easy to incorporate into new builds, and gives them an independent existence as projects of their own. They can be modified, built, tested, and released just like any other piece of software. Remember, in Gradle’s opinion, the build is code, so it should come as no surprise that we would engineer build extensions just like we would the code the extensions are building.

However, the Liquibase plugin didn’t begin its life this way. It started as code inside an existing build.gradle file in the project whose needs gave rise to the plugin. The examples will show independent source files with a build process of their own, but an amazing feature of Gradle plugins is that they can be written directly inside Gradle build scripts. All of the classes we’re about to see could be – and originally were – coded directly into a Gradle build file and used from there. You should never let a plugin stay mixed together with a build in that manner, but beginning plugin development alongside a build is a perfectly appropriate, low-ceremony path to learning the API and discovering the requirements of your build extension.

Applying Yourself

The minimum subset Gradle plugin is a class which imple- ments the org.gradle.api.Plugin<Project> interface. The Plugin interface defines a single method: void apply(Project project). The Liquibase plugin’s apply() method looks something like in listing 2. Please note that the actual Liquibase plugin code has been modified to be more concise.

Listing 2

 

class LiquibasePlugin implements Plugin<Project> 
        { void apply(Project project) {
                // Create a domain object container for databases
                def databases = project.container(Database)
                // Create a domain object container for changeLogs
                def changelogs = project.container(ChangeLog)
                // Add the Liquibase convention to the project
                project.convention.plugins.liquibase =
                        new LiquibaseDatabaseConvention(databases, changelogs)
                applyTasks(project) 
        }
}

 

Inside this method, we have access to the Project object of the build, which is the best backstage pass a buildmaster could ever hope to lay hands on. Here we can add tasks and domain objects to the build, and also interact with existing domain objects and project properties. If we want our plugin to create an additional SourceSet, this is the place to do it. If we want to add an after-evaluation hook to ensure that all task names contain valid Esperanto phrases in camel case, we can do that here. If we want to ensure that the generated API documentation for a Java project will say “Abandon all hope, ye who enter here,” plugins make it possible.

These examples are lighthearted, but the point is quite useful: plugins can be used as a means to impose control over how individual developer may extend builds. Gradle normally sells itself as a very fluid, extensible build system that puts the build master back in control, but for enterprise environments in which an application architect must maintain strict controls over a large portfolio of applications, imposing control through plugins is a welcome feature.

More commonly, plugins add tasks, task types, and new conventions to a build. Task and task types are already familiar to experienced Gradle users. Conventions are new concept, simple in execution but powerful in implication. They are objects that get “mixed in” to the Gradle build, adding properties and methods to the build according to the needs of the plugin. We’ll look at each of these through the lens of the Liquibase plugin.

  

Tasks

The apply() method shown above calls a method called applyTasks(). Let’s take a look at what this method does (listing 3). This example creates three new build tasks: update, up- dateSQL, and dbDoc (In Liquibase, update pushes new schema changes into a database, updateSQL performs a dry run of an update, and dbDoc builds JavaDoc-like documentation of a schema.). Since the Liquibase plugin is written in Groovy, we’re able to use a very Gradle-like syntax to declare new tasks; indeed, the code shown here would work verbatim inside a build file, apart from any plugin definition. Build masters don’t have to write plugins in Groovy, but it’s a rewarding choice due to its similarity to the Gradle build file syntax and its other advantages over Java as a language.

Listing 3

 

void applyTasks(Project project) {        [ 'update', 'updateSQL' ].each { taskName ->                 project.task(taskName, type: LiquibaseBaseTask) {                       group = 'Liquibase' 
                   command = taskName              }       }       project.task('dbDoc', type: LiquibaseDbDocTask) {               command = 'dbDoc' 
             group = 'Liquibase' 
           docDir = project.file("${project.buildDir}/database/docs")      }  }

 

Note in the dbDoc definition that a task property is set to a value derived from an existing Gradle Project property. This underscores the fact that the plugin is interacting with the build not just by adding functionality to it, but by examining its state as well. There is no practical limitation to the ways in which a plugin might interact with the project object.

All three of the newly created tasks have custom task types. For the sake of brevity, we’ll examine just one of these: LibasebaseBaseTask. A custom task type is simply a class that implements the org.gradle.api.Task interface, or more commonly extends the org.gradle.api.DefaultTask base class. The LiquibaseBaseTask provides a basic interface between the rest of the build and the core action of executing the Liquibase command-line driver through which all Liquibase operations are normally accessed. The properties of the LiquibaseBaseTask will become task configuration parameters when the plugin tasks are used in an actual build later on. To keep individual task configuration short and simple, the task code looks to the project object for configuration settings a build master might set globally (these are the workingDatabase and changelogs.main parameters). The code is shown in listing 4.

Listing 4

 

import org.gradle.api.DefaultTask  class LiquibaseBaseTask extends DefaultTask {          Database database 
     def changeLogFile 
     def command     def options = []        @TaskAction 
   def liquibaseAction() {                 // Default configuration from a convention variable             if(database == null) { 
                        database = project.workingDatabase              }               // Default configuration from a convention domain object collection             if(changeLogFile == null) { 
                   changeLogFile = project.changelogs.main.file            }               // process configuration inputs                 ...             // invoke Liquibase command-line driver                 ...     } }

 

All of this functionality can be incorporated into a build merely by applying the plugin. The tasks appear in the build for free, potentially with no further coding or configuration at all.

Conventions

Plugin conventions have a less-than-intuitive name, but are a very simple concept. In short, a convention is a class whose properties and methods are mixed in to the Project object of the build to which the plugin is applied. The class is usually a POJO, or in the case of our Groovy-authored plugin, a POGO. If you want a plugin to create a new domain object collection (like SourceSets, as described above) or a new project property (like groupId, defined by the Maven plugin), conventions are they way to go. You’d simply create a convention object – a POJO or POGO – that has those properties, and the Gradle mixes them into the project when the plugin is applied. You can also add globally available methods to the Project object in this same way. A simplified version of the Liquibase convention object is shown in listing 5.

Listing 5

 

class LiquibaseDatabaseConvention {
        final NamedDomainObjectContainer<Database> databases 
        final NamedDomainObjectContainer<ChangeLog> changelogs 
        Database workingDatabase

        def databases(Closure closure) { 
                databases.configure(closure)

        }
        def changelogs(Closure closure) {
                 changelogs.configure(closure)
        }
}

Based on what we’ve learned about conventions already, the purpose of the workingDatabase property is clear enough: it will become a top-level project property to be set in a build file outside of a task definition. We’ll see an example of this shortly. But first, we have to explore the NamedDomainObjectContainer class used as the type of the databases and changelogs properties. These also become top-level project properties to be set in the plugin users’s build file, but their structure is somewhat richer than a simple string. In fact, it is at this point that we first begin to see the real power of plugins as a means of extending the Gradle DSL to encompass entities in our domain that were not – and could not be – forseen by the developers of Gradle.

Since Liquibase is a database refactoring tool, it must connect to a database to do its work. As a feature of its implementation, it stores an ordered collection of database refactorings in a file called a ChangeLog. A build that wants to use Liquibase to manage database change might have a collection of databases it manages (e.g., development, staging, QA, production, etc.), and it might choose to organize its database changes into several ChangeLog files. Hence the build model must expand to describe these objects, and it must describe them in a way that makes sense in the context of the expanded build domain. It must model a collection of database and a collection of ChangeLogs. The NamedDomainObjectContainer class lets us do just that.

The best-known example of a domain object collection in Liquibase is the SourceSet.

As a plugin author, you can create collections of whatever domain objects exist in the scope of the plugin’s work. This is no mere feature of the plugin API. The ability to introduce new domain objects and their aggregations is essential to the spirit of Gradle as a tookit for creating build standards. Gra- dle itself knows nothing about databases, or how they might change, or how you might prefer to manage their change. Gradle with the Liquibase plugin has a robust definition of all three concepts.

  

Using Our Plugin

Having introduced that convention object to the plugin (and having “registered” it in the project as shown above), let’s finally take a look at what it would look like to configure a build that uses the Liquibase plugin. This build file defines a single database ChangeLog, plus a single sandbox data- base using the H2 embedded database. It then sets a default workingDatabase, so that all of the tasks introduced by the Liquibase plugin will have a default database to connect to without being explicitly configured (listing 6).

Listing 6

changelogs { 
        main {
                file = file('changelog.groovy') 
        }
}

databases { 
        sandbox {
                url = 'jdbc:h2:db/local_sandbox;FILE_LOCK=NO' 
                username = 'sa'
                password = "
        } 
}
workingDatabase = databases.sandbox

A real-world build file might need to be more complex than this, but it’s possible for it to be this simple and to remain useful. Hundreds of lines of build code are completely hidden inside the plugin, and advanced functionality introduced to the build tool as a native part of its vocabulary. And that vocabulary itself is expanded to encompass a new domain that isn’t a part of the core build tool itself. This, ultimately, is what plugins are for.

Conclusion

Plugins are an essential part of the real-world use of Gradle. We use the core plugins for common purposes like building Java code and deploying to repositories. We can easily access plugins built by the community, adding functionality and new configuration options to our builds with just a few lines of code. Most importantly, advanced build masters can create their own plugins to perform arbitrary work and extend the language of the Gradle DSL in unique ways. All of this can be done from within a Gradle build file or in a standalone project with its own release history, tests, and deployment automation. With Gradle, developing the build is just like developing software. Plugins deliver on Gradle’s promises of easy extensibility and bringing the full scope of contemporary software development practice back into the build process – where it has belonged all along.

 

This article first appeared in Java Tech Journal – Gradle from December 2011. To read more of that issue, download it here.

Author
Tim is a full-stack generalist and passionate teacher who loves coding, presenting, and working with people. He is founder and principal software developer at the August Technology Group, a technology consulting firm focused on the JVM. He is a speaker internationally and on the No Fluff Just Stuff tour in the United States, and is co-president of the Denver Open Source User Group. He has recently been exploring non-relational data stores, continuous deployment, and how software architecture should resemble an ant colony. He lives in Littleton with the wife of his youth and their three children.
Comments
comments powered by Disqus