Fickle friends

Tutorial: Smarter search with FIQL and Apache CXF

Feed Item Query Language (FIQL) was originally specified by Mark Nottingham as a language for querying Atom feeds. It appears the combination of FIQL and Atom has not become well referenced in the community.

However the simplicity of FIQL and its capability to express complex queries in a compact and HTTP URI-friendly way makes it a good candidate for becoming a generic query language for searching REST endpoints.

FIQL Overview

FIQL introduces simple and composite operators which can be used to build basic and complex queries. The following table lists basic operators:

Basic Operator

Description

==

Equal To

!=

Not Equal To

=gt=

Greater Than

=ge=

Greater Or Equal To

=lt=

Less Than

le=

Less Or Equal To

 These six operators can be used to do all sort of simple queries, for example:

  • "name==Barry": find all people whose name is Barry

  • "street!=central": find all people who do not live at Central

  • "age=gt=10": find all people older than 10 (exclusive)

  • "age=ge=10": find all people older than 10 (inclusive)

  • "children=lt=3": find all people who have less than 3 children

  • "children=le=3": find all people who have less than or 3 children

The following table lists two joining operators:

Composite Operator

Description

;

AND

,

OR

These two operators can be used to join the simple queries and build more involved queries which can be as complex as required. Here are some examples:

  • "age=gt=10;age=lt=20": find all people older than 10 and younger than 20

  • "age=lt=5,age=gt=30": find all people younger than 5 or older than 30

  • "age=gt=10;age=lt=20;(str=park,str=central)": find all people older than 10 and younger than 20 and living either at Park or Central Street.

Note that while the complexity of the queries can grow, the complete expression still remains in a form which is easy to understand and quite compact. The latter property becomes very useful when considering how to embed FIQL queries into HTTP URIs.

FIQL draft also introduces Atom extensions for describing query interfaces and which items are available for queries. The draft is specific to working with Atom, but it is a good idea in general and can be implemented by non-Atom endpoints too, as it can help the consumers with typing the correct queries.

FIQL in HTTP URI

FIQL draft implies that FIQL expressions can be used in different URI parts and it is up to developers to decide what makes most sense with respect to making it easier to document and for users to type such queries (example, from consoles) when needed.

For example:

  • "/search?s=age=lt=5;age=gt=30"

  • "/search?age=lt=5;age=gt=30"

  • "/search/age=lt=5;age=gt=30"

The first two HTTP queries have FIQL expressions embedded within the URI Query component, with the second query even omitting the actual query parameter name such as “s” used in the first one. The last HTTP query actually embeds the expression within the last URI Path segment – some care is needed in the last case when processing it to make sure FIQL 'OR' operator (“;”) is not treated as HTTP matrix parameter.

The flexibility of HTTP URI and simplicity and compactness of FIQL makes it a very interesting combination.

FIQL support in Apache CXF

Apache CXF is a well-known Java-based framework for developing production-quality WS (SOAP) and HTTP (REST) applications. The development of REST applications is supported by JAX-RS 1.1 implementation front-end, with JAX-RS 2.0 being currently implemented.

Search support is the fundamental feature of most WEB applications and thus Apache CXF chose to support FIQL to make it easier for developers to get their applications offer more advanced search capabilities, in addition to those which can be supported by simple HTTP queries, with the minimum additional complexity overhead.

Processing of FIQL expressions is supported as follows. Initially, the expression is captured into a “search condition” which can then be applied to the data already available in memory or converted with the help of custom visitors to another typed or untyped expression which can be used to query the actual data store supporting a given REST endpoint.

Here is the example code showing how a query can be applied to the in-memory data:

package my.company.search;

import javax.ws.rs.Path;
import javax.ws.rs.GET;
import javax.ws.rs.core.Context;
import org.apache.cxf.jaxrs.ext.search.SearchContext;
import org.apache.cxf.jaxrs.ext.search.SearchCondition;

@Path("search")            
public class SearchResource {
      // list of books, the way this list is populated is not shown for brevity
      private List<Book> theBooks;

      @Path("book")
      @GET
      public List<Book> findBooks(@Context SearchContext searchContext) {
            SearchCondition<Book> condition = searchContext.getCondition(Book.class);
            return condition.findAll(theBooks);
      }
}

 The original search expression is converted into a SearchCondition typed by Book class (the latter is a typical bean with properties like 'title', 'author', etc). This offers a way to validate the actual properties used in the query by having Book implementation itself validating the properties or using the bean validation framework. SearchCondition can represent a primitive query such as “author==nhornby” or much more involved query involving composite 'AND' or 'OR' operators.

Finally, this condition is used to filter the in-memory data and return a list of matching Book instances.

Very often the actual HTTP queries are converted to the query language such as SQL to get the query run against the data store. This approach is supported with the help of custom FIQL converters implementing the well-known Visitor pattern. Apache CXF ships the utility converters for SQL and LDAP, JPA2 TypedQuery and Lucene Query and documents how other custom converters can be easily implemented.

Here is the example of using an SQL converter:

package my.company.search;

import javax.ws.rs.Path;
import javax.ws.rs.GET;
import javax.ws.rs.core.Context;
import org.apache.cxf.jaxrs.ext.search.SearchContext;
import org.apache.cxf.jaxrs.ext.search.SearchCondition;

@Path(“search”)            
public class SearchResource {
      // list of books, the way this list is populated is not shown for brewity
      private List<Book> theBooks;

      @Path(“book”)
      @GET
      public List<Book> findBooks(@Context SearchContext searchContext) {
            SearchCondition<Book> condition = searchContext.getCondition(Book.class);
            SearchConditionVisitor<Book, String> visitor = new SQLPrinterVisitor<Book>();
            visitor.visit(condition);
            String sqlQuery = visitor.getQuery();
          // use sqlQuery to query the data and return the list of books
      }
}

 Please check a demo shipped with the Talend ESB distribution for a more complex example (remove a parent pom section from the demo pom to get the build done fast). This demo shows how FIQL queries can be transparently converted to JPA2 TypedQuery or CriteriaQuery typed queries easily.

It is also worth looking at the Apache CXF wiki page [5] for more information, such as how to map search and bean properties, as well as decoupling the properties used in the search interface from the actual bean properties, on CXF specific extensions and other recommendations.

When to use FIQL?

 Most of the time using traditional HTTP query name and value pairs is very effective. However this approach has its limitations: query parameters have to be 'invented' in order to represent comparison operators other than “equals” and it can be tedious to get a similar query processing code ported across many similar data applications.

In some cases, Google-style search offers a viable solution. However, it is not necessarily optimal for simple to medium complexity Web applications, with the number and properties of data items known and where the search experience can be made much more user-centric with the options provided to do a fine-grained search around the data items.

Apache CXF with its FIQL support offers one way to generalize the search processing code and unify the search experience with the front-end FIQL queries being transparently converted internally to other query languages.

Alternatives

As already mentioned, using plain HTTP name and value queries is one approach which works well for simple applications. OData Protocol introduces a powerful OData query language with OData4J project [8] implementing it in Java. Apache SOLR [9] provides a search engine on top of Apache Lucene.

Conclusion

This article has introduced FIQL, a URI friendly, compact and flexible query language, which was originally specified for Atom feeds but can also be used with non-Atom Web applications.

Using FIQL can help with generalizing and simplifying the search processing for developers and offer a unified search experience to the users. Apache CXF offers a search extension which supports FIQL.

Sergey Beryozkin
Sergey Beryozkin

What do you think?

JAX Magazine - 2014 - 05 Exclucively for iPad users JAX Magazine on Android

Comments

Latest opinions