TCK access controversy – chat with JPA 2.1 Expert Group member Oliver Gierke
Following debate over TCKs in the JPA 2.1 Expert Group, we talked to Oliver Gierke about the situation in JSRs and what needs to be done about it.
Last month, the JPA 2.1 Expert Group sparked an interesting
debate about the accessibility of the Technology Compatibility Kit
(TCK) for JSRs, which are used to verify implementations of
specifications for accuracy. VMware’s Oliver Gierke criticises the
current practice – we spoke to him about the current situation and
why he believes a transparent, open TCK process is the
JAXenter: Hi Oliver. Currently there is a controversial
discussion in the JPA 2.1 Expert Group on the availability of TCKs.
What role do these TCKs actually play?
Oliver Gierke: Before I joined the JPA expert group, I expected
the expert groups to define the TCKs alongside the specification,
to make sure both of them actually align with each other and that
they really implement what was specified.
Working on Spring Data JPA, I repeatedly stumbled over issues
in various persistence providers that indicated core methods of the
API were not being covered by the TCK or the tests were not strict
I asked for ways of how to improve the TCK but didn’t receive an
That was the reason I intensified my attempts to join the expert
group. After joining the group, I was caught by surprise that being
part of it does not mean you get access to the TCK either. All we
can do is formulate a mail to point to problems and hope they get
JAX: So TCKs are thus solely controlled and developed by
Exactly. The expert group does not have access to the TCK until
the final release of the spec. After that only licensees actually
get access. At this point, the JSR has been concluded already and
issues with the TCK are hard to identify. There’s only a vague
backchannel for feedback: an email to the specification lead.
JAX: Can you maybe give an example to explain what is
problematic about the current situation?
Let’s assume the following scenario: we’ve got a team of ten
parties defining requirements for a piece of software. But the team
does so without defining test cases. Besides that the development
teams start implementing code based on the written specification.
They can discuss issues with the spec team but not regarding the
test suite they have to adhere to eventually. They’re essentially
shooting into the dark entirely. In parallel to these activities, a
third stakeholder implements test cases for the spec – completely
isolated, without any feedback at all.
Now the specification gets approved, which means it’s set in
stone from this point on. Exactly at that point in time, the
implementors get access to the test cases and their primary task
right now is to get their implementation running against the
current TCK. At no point in time are there actual checks in how far
the TCK adheres to what is defined in the spec, whether there
untested parts of the spec and so on.
Would we run software projects this way nowadays? I am sure
projects like this exist in the real world. Still, this is not the
way a software project should be run, I think. Especially if it’s
about a Java Standard that developers are going to code against for
years – if not even decades.
To come back to the JSRs, we face the additional problem that
implementations will usually be certified. If we discover glitches
violating the specification which lead back to a glitch in TCK –
what do we do? De-certify the implementation?
There’s no process for how to deal with issues in the TCK, or
missing tests in it, and a way to correct the certifications. One
more reason for why it’s crucial to come to a solid TCK in the
So we potentially – but in my point of view also practically –
face quite a gap between what was specified and what the
implementors are required to adhere to passing the TCK. This is a
problem especially for the portability of applications between
persistence providers, ultimately ending in the question whether it
makes sense to code against a method that is vaguely specified but
is implemented with completely different semantics
throughout all persistence providers effectively. This is the point
where the value of a specification is undermined
And I think it’s fair to raise the question why one would want
to work on a specification if the only authoritative instance for
implementors – and developers using the API – is defined completely
separately from the spec and there’s now way to control how they
relate to each other.
JAX: Are these issues you describe specifically relevant
for JPA? Or put in another way: Are there similar discussions going
on in other expert groups?
The core aspect here is that every JSR is run based on a version
of the Java Community Process (JCP) that defines the process of how
to run a JSR, what needs to be produced at which point in time, and
under which license they have to be published. Some specs (e.g.
JPA) are very close to these rules. But I know about other
specifications (e.g. CDI) that are working more openly. The TCK of
the CDI spec has been publicly available under the Apache license
through the course of the entire specification process. This avoids
the gap between spec and TCK and integrates the community at the
same time. That does of course not mean, that ambiguities in the
spec can be avoided entirely but they are much less likely.
JAX: What do you suggest to improve the current
I am shooting for two things. First of all, the spec and the TCK
should be developed alongside each other, e.g. we could insist on
no API being able to be specified without dedicated test cases
being provided as well. This of course requires access for expert
group members to the TCK. Implementors would especially benefit
from that, as they would not have to implement for more than two
years without being able to run tests against their code.
The second thing I’d welcome warmly is a process of how to deal
with TCK glitches. There has to be a way to continuously improve
the test suite.
JAX: Would you like to see the TCKs made completely open
source and freely available to the community?
Exactly. The already mentioned CDI specification shows that this
is possible. The TCK is available on GitHub - ‘forkable’ and
‘improveable’ for everyone. Why shouldn’t this work for other
JAX: Oracle’s Linda DeMichiel wrote on the mailing list
recently that there is no special team for the JPA TCKs at Oracle.
She mentions that TCKs for other JSRs have been completed earlier.
Were other expert groups served much earlier? It sounds as if
the TCK team at Oracle is not really big, so that the provision
just takes a lot of time.
That’s a very good point. It’s hard to find out details about
who is working on the TCK for JPA 2.1 to what extend. Linda also
mentioned that so far no one has really shown interest in helping
out with the TCK. It’s important to note that I don’t want to
finger point at someone for doing bad work.
My main point is that in my opinion the current process is
flawed and significantly impacts the quality of the standard in a
negative way. I know that a better TCK means more work in the first
place and I am definitely willing to help out. Perhaps we may even
find more volunteers via the Adopt-a-JSR-program. But I do think
that it’s hard to attract people if it’s effectively impossible to
help if the process is not open. Beyond that I think most
developers using standards like JPA have not been aware of the gap
and why the current state is actually problematic at all.
If you look at the situation realistically, do you think
the current situation will change soon? Is Oracle ready for
I already mentioned that it’s not easy to get access to
information – even as a member of the JPA expert group. On the
other hand I see it working for other specifications. In this
respect, I consider this discussion as opportunity to create
transparency and hopefully create interest for participation in the
community at the same time. Ultimately it’s about a well-designed
and usable API. There are way too many APIs out there that
developers go berserk about. So every tiny improvement is an
So you would like to broaden the discussion on this
issue – our readers can comment here and share their opinion. What
else can they do to get involved in the discussion?