Rewriting the rulebook

Eclipse Intent: “Being Agile does not mean being short-sighted”

Diana Kupfer
eclipse-intent1

JAXenter speaks to Intent project lead Alex Lagarde to discuss the importance of documentation.

Eclipse
Intent
is a subproject of Mylyn that promises to make it easier
to keep documentation up-to-date, with the bother of development
artifacts.

JAXenter caught up with Project
Lead
Alex
Lagarde
 to discuss the importance of
documentation and how Intent can help.

Documentation is surely
not the most popular domain in programming. From your point of
view, what are the most common reasons – or excuses – for not
writing doc?

We could compare the current reluctance to
document software with the massive

rejection of unit testing in the 90’s:
people think it’s useless because it takes so much time and does
not bring any direct value to the end-user.

I think that many people consider that
documentation is useless because of the Manifesto for Agile
Software Development, which promotes the value of “Working software
over comprehensive documentation”. But in my opinion this sentence
is often misunderstood: it just means that the best documentation
in the world is no excuse if your project fails to deliver a
working software.

Moreover, the lack of tooling to browse
through/update documentation makes documentation-related tasks long
and painful. Any developer who had to maintain documentation
up-to-date with the changes made on their software will agree:
documentation update is quite a burden. When you make modifications
on the code, how many of you take the time to browse through the
hundreds of pages of documentation just to find where to document
these changes, and check the whole documentation consistency
afterwards? No one, and thus documentation becomes useless, as it
does not reflect the recent changes you made on your
software.

Sooner or later, no one will trust and hence
read it any more. Consequently, the design choices your team made
will vanish, although you spent a lot of time trying to document
these choices.This is the main reason for which documentation is
not popular: although it is time-consuming, you are never sure that
it is accurate and useful.

How would you try to
convince a developer to write doc, especially if there are severe
time constraints on a project?

Let’s continue with the testing
example: what convinced people that writing test was as important
as writing code?

First of all, the context matters: people
should be aware that documentation is important and understand the
issues caused by missing or unsynchronized doc. Fortunately, we are
not the only ones claiming that documentation is important, even in
Agile processes: Kurt Solarte (IBM) talks about the importance of
Agile Documentation and the “maintenance documentation gap”, Scott
Ambler talks about “Agile/Lean Documentation “ and defines a set of
best practices to update the doc, Andreas Rüping wrote a book about
“Agile documentation”…

So people are starting to get interested in
the improvements we can make (in process, in tooling) to make
documentation update more efficient. I will just add that being
Agile does not mean being short-sighted: remember that if your
team’s primary goal is indeed to develop software, its secondary
goal is to enable your next effort, “preparing for the next game”
as Cockburn would say. Yes, documentation can preserve knowledge
within a team, and prevent developers from re-inventing things when
team members leave and new people join.

If you think about your activity as a
developer, you will realize that you spend most of your time trying
to understand other people’s code. Don’t you think you would gain
time by having a clear and up-to-date documentation about the code
you are studying? Yes? Why don’t you write it for your own software
then?

Thinking that doc is a waste of time is
exactly like thinking tests are a waste of time: it takes effort to
write, but it allows to save time when your software
evolves.

But if we refer to the testing example, what
really convinced people was not the theoretical talk: it was the
emergence of tooling allowing to quickly write tests (such as
Junit). That’s where Intent comes into play.

How does Intent lower the
barrier for writing good and useful
documentation?

Intent is a documentation environment
integrated to the Eclipse IDE. So first of all you will be able to
edit your documentation directly inside your IDE, without having to
switch to another tool. Moreover, you will be able to open an
Intent editor on any sub-part of the documentation, no matter how
small. The idea is to follow the “loading off your brain” mantra
shared by all the Mylyn tools: we only show to the user what he
currently needs to see, and not a hundred of pages
document.

As explained previously, there is an
inconvenient truth about documentation: although we’re in 2012,
performing a task as simple as updating a documentation according
to changes made on the code is totally manual, long and painful.
That’s exactly why we decided to create the Mylyn Intent project:
to provide tooling for keeping your documentation synchronized with
any technical artifact (Java code, models, C++ code, plugin
dependencies, a code sample on the internet…).

How does this tooling
work?

To do so, we simply maintain formal links
between each documentation part and the software artifacts it is
describing. Whenever you modify your code, Intent will use these
formal links to determine which parts of the documentation should
be updated, avoiding you a painful manual research. Intent will
place error markers on all the outdated documentation parts, that
you will be able to see and open in one click in the Eclipse
Problem View. Then you can graphically visualize the differences
between what’s specified in the Intent document and the
corresponding technical artifact (e.g. “The class has been
renamed”), and decide to update the documentation or the
code.

So when do I update the
outdated parts?

You are free to update the outdated
documentation parts whenever you want. It’s up to you: just choose
the most efficient way of updating documentation according to your
process. The Disciplined Agile Delivery describes several processes
for updating your documentation: you may want to update it right
away (“document continuously”), wait for the code to stabilise
(e.g. the issue is closed), update the documentation just before a
release (“document late”) or when people complain (“document only
when it hurts”)…

So Intent is basically a
synchronization tool?

Intent is much more as it is a bridge
between technical artifacts and high-level concerns; it is a
perfect medium to define high-level constraints on your software.
As a matter of fact, I’m sure that you already defined such
constraints: your team shares the same dev guidelines, processes
and good practices.

But most of the time these constraints are
not expressed formally: best case scenario, they are expressed in
natural language inside a documentation file. Consequently,
reviewing the code to check those is a very hard and painful job,
and can lead you to endorse the bad role when having to explain to
your developers that they do not code like you want them to.
Moreover, as these constraints are not expressed formally, you are
never sure that you fulfilled all them.

What does Intent do more
specifically?

Intent provides tooling for automatically
checking these constraints and guidelines. For example, you can
plug inside Intent a rule that says that “Any bug should not be
closed if the corresponding test has not been written”, “Any
feature displayed to the end-user should be presented inside the
User Guide with a screenshot”… As soon as such a rule is broken,
Intent displays validation issues in the Eclipse Problem View,
pointing out the elements that broke it. Consequently, as a
developer you can know before committing that you broke some of the
dev guidelines, and fix the issue right away.

To sum-up, Intent will help you write an
up-to-date documentation (by maintaining formal links between
documentation & code) and automatically checks high-level
constraints. In result, you will be able to rely on the
documentation to store your team knowledge. You will gain time when
searching for information (no need to ask other team members), and
in code review (as Intent checks a lot of dev guidelines
automatically).

Intent is a subproject under
Mylyn Docs. How does it tie into the general purpose of
Mylyn/Tasks?

The Mylyn top-level project describes itself as
an Application Lifecycle Management (ALM) framework for Eclipse.
Each Mylyn project brings tooling for answering ALM
concerns.

As a Mylyn citizen, Intent brings additional
value to the ALM prime objectives:

  • “Increases productivity, as the team shares best practices for
    development”: Intent helps the team sharing the same best practices
    and automatically checks them.
  • “Developers need focus only on current business requirements”:
    Intent allows to consider only the parts of the documentation that
    are relevant to your current task (“loading off your brain”). Only
    the relevant sub-parts of the documentation will be embedded inside
    the Context of a Mylyn Task. Moreover, it is way easier to retrieve
    all the requirements related to the code you are currently
    interested in.
  • “Breaks boundaries through collaboration and smooth information
    flow“: as explained later in this article, Intent allows to share
    documentation and ideas in real-time.
  • “Cuts maintenance time by synchronizing application and
    design”: that’s exactly what Intent aims to do.

Moreover, documentation is an important aspect
of ALM. That’s why the “Mylyn docs” component was created, holding
sub-projects such as Wikitext or Intent.

A central premise or motto
of the Intent project is what Donald Knuth stated in his
book
Literate
Programming
:
Instead of imagining that our main task is to instruct a computer
what to do, let us concentrate rather on explaining to human beings
what we want a computer to do.” The book was written back in 1984 –
why do you think this statement is still valid?

Donald E. Knuth has brought most of the
concepts used in Intent. Literate Programming is a methodology that
combines a programming language with a documentation language
inside a same document, the Literate Program. Indeed, the main idea
is to treat a program as a piece of literature, addressed to human
beings rather than to a computer.

Of course, this still makes sense, even more
than in 1984: if code was supposed to be read only by computers, we
would write it using assembler languages. But we learned that
coding is a social task: your code is meant to be read by several
other people (including the future you), and you should focus on
being as readable as you can when writing code. According to Knuth,
keeping the intents behind the software, the reasons that lead you
to make a design choice or write a piece of code, is as important
as the code itself. Such intents cannot be properly captured inside
the code (e.g. through javadoc), because the code structure is too
narrowed to present high-level specifications.

But that idea isn’t
implemented directly in Intent, or is it?

We have three main differences with the
Literate programming approach:

Literate Programming was about mixing
documentation and C code inside a same document. With Intent, you
can mix documentation with any kind of technical artifact (models,
Java code, C++ code, workspace configuration…). For the Juno
release, we focused on what could be called the Literate Modeling
approach, mixing documentation and any kind of EMF models. We
already have business opportunities in this field, where Intent is
used to document Enterprise Architectures, System Engineering
models (using SysML)…

With Literate Programming, you could not
directly modify the code: you had to update your Literate Program,
and then re-generate the code. With Intent, we consider that code
should still be edited through with its dedicated tooling, models
should still be edited using modelers… Just use the right tool
for the right task.

With Literate Programming, you had to make your
code and your documentation evolve in the same time. In Intent, we
consider that documentation and code have different lifecycles. It
is acceptable to have a documentation that does not reflect the
changes made a few days ago. So when the documentation or the code
evolves, we just put synchronization issues on the outdated parts,
letting you choose when it is appropriate to change it.

How does Intent facilitate
document sharing?

We believe that documenting is a social task.
Sharing documentation in real-time is not just a nice-to-have
feature, but a real need, that allows real-time reviews and
feedbacks on your specifications and ideas. Sharing documents
through e-mails or VCS is not appropriated for this real-time
collaboration need, so we designed Intent to allow live
collaboration around docs.

Intent relies on a client/server architecture:
we have a central Repository, in which we store the Intent document
as a model, and many other useful information (like all the
validation and synchronization issues, traceability
information…). Each feature of Intent is managed by a client of
this Repository: the Eclipse editor is just one among others. Each
client communicates with others through a notification mechanism
ensured by the Repository.

This Repository concept is totally abstract,
and can be implemented with many concrete back-ends. For the
Eclipse Kepler release, we are working on a CDO-based Intent
Repository. If you are not familiar with
CDO, you should
definitely give it a try, it is a great piece of technology. In a
nutshell, CDO allows live sharing of models, with real-time updates
and fine-grained lock management to avoid conflicts.

With a CDO-based Repository, many users can
read and edit the same Intent document. As soon as a user saves his
document, the changes are propagated through the Repository, and
other users see those changes in real-time.

We are currently deploying this technology on
the
Obeo SmartEA tool, by
connecting Intent to a CDO server storing Enterprise Architecture
models. It is a very good use case for Intent, as EA models contain
complex and business-critical information that needs to be
rigorously documented. Intent represents a nice solution allowing
to easily keep the EA models and the documentation up-to-date,
 and to ease the impact analysis when simulating migration of
the Enterprise Architecture. The next step will be to work on
performances, to be able to handle huge models and Intent
documents.

Can you explain what
“Ariadne” is about?

Ariadne is
an open-source tool allowing to represent many technical artifacts
(Java code, GIT repositories and commits, bugzilla issues, plugin
dependencies, checkstyle configuration…) as Models. It also
features visualization mechanisms providing you a bird-eye view of
your projects.

By integrating Ariadne with Intent, we will be
able to link pieces of an Intent document with these technical
artifacts. For example, if you change the location of the git
Repository in which your example projects are stored, Intent will
raise synchronization issues in the part of the documentation that
explains where to check-out the example projects.

The most powerful feature of Ariadne is that,
by linking all these technical artifacts together, you can write
complex constraints on your project: for example, you can make sure
that any developer works with the expected target platform,
checkstyle rules, mylyn connectors…You can also check that any
git commit references a bugzilla issue, automatically generate API
changes…

Your session at EclipseCon
Europe 2012 was packed, and surely not only because your session
had a catchy title… Do you think there is a growing awareness of
the importance of documentation?

Definitely. As I explained earlier, more and
more influential people are stressing out how important
documentation is and try to make developers aware of it.

Especially in open-source projects, in which
transparency and external contributions are major concerns, people
should take care of their documentation. And I think that is why
more and more people are getting interested in Intent.

The Eclipse Foundation has recently launched an
ongoing effort to improve its processes and technologies, in order
to lower the barrier for external contributions. In that context,
we should define tools and processes to help people write
meaningful contribution guides. I really think that  Intent
& Ariadne can help lowering the barrier of
contribution.

Can you give us a short
preview of Intent in 2013?

Our prime objective is to make developers
consider documentation as something useful and easy to maintain,
that they can use to formalize their best practices and share their
knowledge. Hence the roadmap for the Eclipse Kepler release is
straightforward (see

“Mylyn Intent: the Shape of things to
come”
 for more details):

  • Improve the tooling, for example dropping a Java Class inside
    the documentation will automatically create a link between the doc
    and the Class. We also plan to provide new quick-fixes for updating
    the documentation, create Eclipse Views allowing to see what parts
    of your project are documented, provide new export formats (PDF,
    HTML Bootstrap)
  • Work on the CDO support (see question 6) to provide the best
    collaborative experience possible.
  • Work on the integration with Ariadne, as we know that code
    synchronization capabilities is the most wanted  feature by
    the eclipse community.

Although we have a strong vision of what we
want, Intent is an open-source project, and we are willing to
postpone any feature to answer to the community needs.  Do not
hesitate to spam us on the
Intent
forum
 with questions, ideas,
feedbacks…

There is still a lot of work to do on this
awesome topic, and we will gladly welcome external contributions
or
committers. I’m convinced that Intent
is the future tooling for managing our documentation.

Author
Diana Kupfer
Working at S&S Media since 2011, Diana Kupfer is an editor at Eclipse Magazine, Java Magazin and JAXenter.de.
Comments
comments powered by Disqus