DIY Gradle Plugin Development

Plugging into Gradle Plugins

TimBerglund
gradle

Tim Berglund explores build automation tool Gradle and how it can be enhanced to manage changes in relational databases, via the use of Gradle plugins

Gradle uses a Groovy-based DSL for its build configuration
files. Since Gradle build files are also Groovy scripts, build
masters have essentially unlimited flexibility to extend Gradle in
whatever ways they want, just by writing Groovy code in the build
file. While this kind of power is attractive, and while every
Gradle user is likely to sketch some Groovy code into a build at
some point, this isn’t a sustainable way to grow the functionality
of the build. The only thing worse than a build tool that resists
your customizations is one that gives you a blank slate for as much
ill-considered, poorly-factored, and untested code as you can
write. Gradle has a better way.

Much of the out-of-the-box functionality provided by
Gradle is given through the means of Gradle

plugins. Building Java code,
packaging executable applications, or running code quality analysis
through Sonar is as easy as applying a core plugin with a single
line of code. Knowing these plugins is an essential part of using
Gradle for ordinary builds, and they are well-documented in the
Gradle User Guide that comes with every release.

But what happens when your needs outgrow the functionality
of the core plugins? What happens when you, like any other build
master building real applications, need to write code to do things
the creators of Gradle couldn’t anticipate? When you get to this
point, it’s time to write a plugin.

In this article, we’ll enhance Gradle with the ability to
manage changes in a relational database. Our goals will be to
ex
tend the Gradle DSL with a small new syntax, while keeping
imperative Groovy code entirely out of the build.

  

The Liquibase Plugin

Liquibase is an open-source
database refactoring tool. It provides a means for developers to
collaborate on the design of a relational database schema, and to
publish a time-ordered stream of schema changes to multiple
database instances. It’s very good at keeping multiple developer
sandboxes in sync, and for providing an automated mechanism for
tracking and deploying database change to shared staging, QA, and
production servers. A tool like this is virtually begging to be
scripted into a well-constructed build. Liquibase is a Java-based
tool with a convenient command-line wrapper built in, but even
“convenient” command-line Java is not all that pleasant to run.
Liquibase needs a Gradle plugin.

Applying a core plugin is easy: you simply include the statement
apply plugin: ‘plugin-id’ in your build script. Because core
plugins are a part of the Gradle release, their IDs are reg-
istered and their classes are available to the Gradle classload-
er. Applying an external plugin is very similar, but we have to
take one more step to tell Gradle where to find the plugin.

The buildscript method lets us
add jars to the classpath of the build itself, and even declare the
repositories Gradle should use to fetch those jars. To include the
liquibase plugin in a build, we’d do the following (listing
1).

Listing 1: Applying the Liquibase plugin using the
+buildscript+ method.

 

apply plugin: ‚liquibase‘
buildscript { 
        repositories {
                mavenCentral() 
        }
        dependencies {
                classpath ‚com.augusttechgroup:gradle-liquibase-plugin:0.5.1‘
        } 
}

 

Merely applying the liquibase plugin with no other preparation
wouldn’t work, since Gradle doesn’t natively know a plugin by that
name. To tell Gradle where to find it, we have to declare it as a
dependency. The dependencies declaration points to a Maven-style
vector which happens to resolve in Maven Central. Note that the
dependency is as- signed to the classpath configuration; all
dependencies de- clared in the buildscript block should go here.
This indicates that they will be made available in the classpath of
the build script itself.

The rest of the build file is now free to make use of the
Liquibase plugin as it sees fit. This may include using any number
of new tasks, task types, project properties, and do- main objects
introduced by the plugin. The full details of the

Liquibase plugin are beyond our present scope, but you can
follow the project on GitHub for more documentation as it becomes
available.

The Plugin API

For all the power Gradle plugins bring us, creating one is
sur- prisingly easy. We’ll study the code of the Liquibase plugin
as an example. The Liquibase plugin is written as a standalone
project of its own, which builds to its own JAR and is deployed
under its own artifact ID to Maven Central. This is the preferred
way to write and distribute open source plugins, as it makes them
very easy to incorporate into new builds, and gives them an
independent existence as projects of their own. They can be
modified, built, tested, and released just like any other piece of
software. Remember, in Gradle’s opinion, the build is code, so it
should come as no surprise that we would engineer build extensions
just like we would the code the extensions are building.

However, the Liquibase plugin didn’t begin its life this
way. It started as code inside an existing

build.gradle file in the project whose needs
gave rise to the plugin. The examples will show independent source
files with a build process of their own, but an amazing feature of
Gradle plugins is that
they can be written directly
inside Gradle build scripts.
All of the classes we’re
about to see could be – and originally were – coded directly into a
Gradle build file and used from there. You should never let a
plugin stay mixed together with a build in that manner, but
beginning plugin development alongside a build is a perfectly
appropriate, low-ceremony path to learning the API and discovering
the requirements of your build extension.

Applying Yourself

The minimum subset Gradle plugin is a class which imple-
ments the
org.gradle.api.Plugin<Project>
interface. The Plugin interface
defines a single method:
void apply(Project
project).
The Liquibase plugin’s
apply() method looks something like in listing
2. Please note that the actual Liquibase plugin code has been
modified to be more concise.

Listing 2

 

class LiquibasePlugin implements Plugin<Project> 
        { void apply(Project project) {
                // Create a domain object container for databases
                def databases = project.container(Database)
                // Create a domain object container for changeLogs
                def changelogs = project.container(ChangeLog)
                // Add the Liquibase convention to the project
                project.convention.plugins.liquibase =
                        new LiquibaseDatabaseConvention(databases, changelogs)
                applyTasks(project) 
        }
}

 

Inside this method, we have access to the
Project object of the build, which is the best
backstage pass a buildmaster could ever hope to lay hands on. Here
we can add tasks and domain objects to the build, and also interact
with existing domain objects and project properties. If we want our
plugin to create an additional
SourceSet,
this is the place to do it. If we want to add an
after-evaluation hook to ensure that all task names contain valid
Esperanto phrases in camel case, we can do that here. If we want to
ensure that the generated API documentation for a Java project will
say “Abandon all hope, ye who enter here,” plugins make it
possible.

These examples are lighthearted, but the point is quite
useful: plugins can be used as a means to impose control over how
individual developer may extend builds. Gradle normally sells
itself as a very fluid, extensible build system that puts the build
master back in control, but for enterprise environments in which an
application architect must maintain strict controls over a large
portfolio of applications, imposing control through plugins is a
welcome feature.

More commonly, plugins add tasks, task types, and new
conventions to a build. Task and task types are already familiar to
experienced Gradle users. Conventions are new concept, simple in
execution but powerful in implication. They are objects that get
“mixed in” to the Gradle build, adding properties and methods to
the build according to the needs of the plugin. We’ll look at each
of these through the lens of the Liquibase plugin.

  

Tasks

The apply() method shown above
calls a method called
applyTasks(). Let’s
take a look at what this method does (listing 3). This example
creates three new build tasks:
update, up-
dateSQL,
and dbDoc (In
Liquibase,
update pushes new schema
changes into a database,
updateSQL
performs a dry run of an update, and
dbDoc builds JavaDoc-like documentation of a
schema.). Since the Liquibase plugin is written in Groovy, we’re
able to use a very Gradle-like syntax to declare new tasks; indeed,
the code shown here would work verbatim inside a build file, apart
from any plugin definition. Build masters don’t have to write
plugins in Groovy, but it’s a rewarding choice due to its
similarity to the Gradle build 
file syntax and its
other advantages over Java as a language.

Listing 3

 

void applyTasks(Project project) {        [ 'update', 'updateSQL' ].each { taskName ->                 project.task(taskName, type: LiquibaseBaseTask) {                       group = 'Liquibase' 
                   command = taskName              }       }       project.task('dbDoc', type: LiquibaseDbDocTask) {               command = 'dbDoc' 
             group = 'Liquibase' 
           docDir = project.file("${project.buildDir}/database/docs")      }  }

 

Note in the dbDoc definition that a task property is set to a
value derived from an existing Gradle Project property. This
underscores the fact that the plugin is interacting with the build
not just by adding functionality to it, but by examining its state
as well. There is no practical limitation to the ways in which
a plugin might interact with the project object.

All three of the newly created tasks have custom task
types. For the sake of brevity, we’ll examine just one of
these:
LibasebaseBaseTask. A custom task
type is simply a class that implements the

org.gradle.api.Task interface, or more commonly
extends the
org.gradle.api.DefaultTask
base class. The LiquibaseBaseTask
provides a basic interface between the rest of the build and
the core action of executing the Liquibase command-line driver
through which all Liquibase operations are normally accessed. The
properties of the
LiquibaseBaseTask will
become task configuration parameters when the plugin tasks are used
in an actual build later on. To keep individual task configuration
short and simple, the task code looks to the

project object for configuration settings a
build master might set globally (these are the

workingDatabase and
changelogs.main parameters). The code is shown in
listing 4.

Listing 4

 

import org.gradle.api.DefaultTask  class LiquibaseBaseTask extends DefaultTask {          Database database 
     def changeLogFile 
     def command     def options = []        @TaskAction 
   def liquibaseAction() {                 // Default configuration from a convention variable             if(database == null) { 
                        database = project.workingDatabase              }               // Default configuration from a convention domain object collection             if(changeLogFile == null) { 
                   changeLogFile = project.changelogs.main.file            }               // process configuration inputs                 ...             // invoke Liquibase command-line driver                 ...     } }

 

All of this functionality can be incorporated into a build
merely by applying the plugin. The tasks appear in the build for
free, potentially with no further coding or configuration at
all.

Conventions

Plugin conventions have a less-than-intuitive name, but
are a very simple concept. In short, a convention is a class whose
properties and methods are mixed in to the

Project object of the build to which the plugin
is applied. The class is usually a POJO, or in the case of our
Groovy-authored plugin, a POGO. If you want a plugin to create a
new domain object collection (like
SourceSets,
as described above) or a new project property (like
groupId, defined by the Maven plugin),
conventions are they way to go. You’d simply create a convention
object – a POJO or POGO – that has those properties, and the Gradle
mixes them into the project when the plugin is applied. You can
also add globally available methods to the

Project object in this same way. A simplified
version of the Liquibase convention object is shown in listing
5.

Listing 5

 

class LiquibaseDatabaseConvention {
        final NamedDomainObjectContainer<Database> databases 
        final NamedDomainObjectContainer<ChangeLog> changelogs 
        Database workingDatabase

        def databases(Closure closure) { 
                databases.configure(closure)

        }
        def changelogs(Closure closure) {
                 changelogs.configure(closure)
        }
}

Based on what we’ve learned about conventions already, the
purpose of the
workingDatabase property
is clear enough: it will become a top-level project property to be
set in a build file outside of a task definition. We’ll see an
example of this 
shortly. But first, we have to explore
the NamedDomainObjectContainer class used as the type of the
databases and changelogs properties. These also become top-level
project properties to be set in the plugin users’s build file, but
their structure is somewhat richer than a simple string. In fact,
it is at this point that we first begin to see the real power of
plugins as a means of extending the Gradle DSL to encompass
entities in our domain that were not – and could not be – forseen
by the developers of Gradle.

Since Liquibase is a database refactoring tool, it must
connect to a database to do its work. As a feature of its
implementation, it stores an ordered collection of database
refactorings in a file called a
ChangeLog.
A build that wants to use Liquibase to manage database change
might have a collection of databases it manages (e.g., development,
staging, QA, production, etc.), and it might choose to organize its
database changes into several ChangeLog files. Hence the build
model must expand to describe these objects, and it must describe
them in a way that makes sense in the context of the expanded build
domain. It must model a collection of database and a collection of
ChangeLogs. The
NamedDomainObjectContainer
class lets us do just that.

The best-known example of a domain object collection in
Liquibase is the
SourceSet.

As a plugin author, you can create collections of whatever
domain objects exist in the scope of the plugin’s work. This is no
mere feature of the plugin API. The ability to introduce new domain
objects and their aggregations is essential to the spirit of Gradle
as a tookit for creating build standards. Gra- dle itself knows
nothing about databases, or how they might change, or how you might
prefer to manage their change. Gradle with the Liquibase plugin has
a robust definition of all three concepts.

  

Using Our Plugin

Having introduced that convention object to the plugin
(and having “registered” it in the project as shown above), let’s
finally take a look at what it would look like to configure a build
that uses the Liquibase plugin. This build file defines a single
database ChangeLog, plus a single sandbox data- base using the H2
embedded database. It then sets a default

workingDatabase, so that all of the tasks
introduced by the Liquibase plugin will have a default database to
connect to without being explicitly configured (listing
6).

Listing 6

changelogs { 
        main {
                file = file('changelog.groovy') 
        }
}

databases { 
        sandbox {
                url = 'jdbc:h2:db/local_sandbox;FILE_LOCK=NO' 
                username = 'sa'
                password = "
        } 
}
workingDatabase = databases.sandbox

A real-world build file might need to be more complex than
this, but it’s possible for it to be this simple and to remain
useful. Hundreds of lines of build code are completely hidden
inside the plugin, and advanced functionality introduced to the
build tool as a native part of its vocabulary. And that vocabulary
itself is expanded to encompass a new domain that isn’t a part of
the core build tool itself. This, ultimately, is what plugins are
for.

Conclusion

Plugins are an essential part of the real-world use of
Gradle. We use the core plugins for common purposes like building
Java code and deploying to repositories. We can easily access
plugins built by the community, adding functionality and new
configuration options to our builds with just a few lines of code.
Most importantly, advanced build masters can create their own
plugins to perform arbitrary work and extend the language of the
Gradle DSL in unique ways. All of this can be done from within a
Gradle build file or in a standalone project with its own release
history, tests, and deployment automation. With Gradle, developing
the build is just like developing software. Plugins deliver on
Gradle’s promises of easy extensibility and bringing the full scope
of contemporary software development practice back into the build
process – where it has belonged all along.

 

This article first appeared in Java Tech Journal – Gradle
from December 2011. To read more of that issue, download it
here.

Author
Tim is a full-stack generalist and passionate teacher who loves coding, presenting, and working with people. He is founder and principal software developer at the August Technology Group, a technology consulting firm focused on the JVM. He is a speaker internationally and on the No Fluff Just Stuff tour in the United States, and is co-president of the Denver Open Source User Group. He has recently been exploring non-relational data stores, continuous deployment, and how software architecture should resemble an ant colony. He lives in Littleton with the wife of his youth and their three children.
Comments
comments powered by Disqus