Tricky business

Just add mobility: JSF and JSP with ICEmobile

Developing mobile UIs is a tricky business, especially with the market changing by the minute. ICEsoft chief software architect Ted Goddard explains how to get the most out of the medium, in this JAX Magazine article

The future of the web is mobile. This is clear from the increasing use of both phones and tablets. Further, mobile devices often have a simple and reliable software update process which provides them with modern, updated, HTML5-compliant browsers. That makes it great to develop for.

Modern mobile devices have an impressive range of features that have the potential to create a very rich user interface: CSS3 and HTML5, GPS and compass, front and rear video cameras, accelerometers and gyroscopes, and 3D accelerated high resolution displays. In a typical legacy desktop environment, it's very easy to find machines that have nowhere near these capabilities.

Keep it simple and efficient

There are just two problems: mobile devices are small and they're mobile.

Small devices have tiny touchscreen controls which are difficult to hit. This is complicated further by the user's finger blocking their view, so we have to use larger UI targets, compounding the limited number of controls that can be used on a small display. Text entry is fairly inefficient on such devices. Of course, tablets are not as constrained, although they do tend to have smaller screens than current desktops.

Mobility itself raises two problem areas: network and power. Mobile network connections can be high latency, low bandwidth, or just not there at all. Mobile devices run on limited battery power most of the time, so we must be mindful not to waste this power through excessive UI features, network traffic or other client-side CPU use.

OK, there’s clearly more than two issues, but the next biggest problem is psychological; why is it necessary to have such elaborately styled mobile UIs? It has become fashionable to apply the greatest ornamentation and animation features to the smallest screens we have, often at the expense of functionality and performance. Remember the blink tag? It's more important for the user to be productive than entertained. (This concludes the rant; we now return you to the main article. Please continue to enjoy the simulated rich Corinthian leather of your mobile UI.)

Augmented Reality

Now that we've established that the mobile web should be based on simplicity and power-efficiency, let's look at a user interface capable of draining the device battery in just a few minutes: augmented reality.

Figure 1: Augmented Reality device

Modern mobile phones and tablets have unique sensors and displays that allow them to do things that are simply not possible with a desktop computer. The differences won't stop here however – future devices will be head-mounted with a display that is directly overlaid on the user's vision; whether this is useful or annoying will remain to be seen.

The main hardware components we need are: a rear-facing camera for visual input from the "real world", a compass and accelerometer to find the actual device orientation relative to the earth, a gyroscope to reliably track orientation changes (the compass is too easily influenced by nearby magnetic fields) and a front 3D accelerated display for viewing the application.

With these capabilities, we can create two very interesting Augmented Reality variants: virtual objects that appear to be located in real space, and virtual objects that are attached to real-world objects (sophisticated image recognition algorithms are required here in addition to the hardware capabilities).

Does this sound like a web application? It is; here's the JSF markup for a simple demo that lets you choose which puzzle blocks to display attached to augmented reality "markers":

Listing 1

 <mobi:fieldsetGroup> 

            <mobi:fieldsetRow > 
                <p>Select your desired 3D models:</p> 
            </mobi:fieldsetRow> 

            <mobi:fieldsetRow styleClass="mobi-center"> 
                <h:selectOneMenu value="#{realityBean.selectedModel1}"> 
                    <f:selectItem itemLabel="Puzzle Block 1" 
                            itemValue="puz1"/> 
                    <f:selectItem itemLabel="Puzzle Block 2" 
                            itemValue="puz2"/> 
                </h:selectOneMenu> 
                 <h:selectOneMenu value="#{realityBean.selectedModel2}"> 
                    <f:selectItem itemLabel="Puzzle Block 1" 
                            itemValue="puz1"/> 
                    <f:selectItem itemLabel="Puzzle Block 2" 
                            itemValue="puz2"/> 
                </h:selectOneMenu> 
           </mobi:fieldsetRow> 

            <mobi:fieldsetRow styleClass="mobi-center"> 
                <mobi:augmentedReality 
                    value="#{realityBean.selection}" > 
                    <mobi:augmentedRealityMarkers 
                        value="#{realityBean.markers}" var="marker" 
                        markerLabel="#{marker.label}" 
                        markerModel="#{marker.model}" /> 
                </mobi:augmentedReality> 
            </mobi:fieldsetRow> 

        </mobi:fieldsetGroup>

That's all there is to it: just add the mobi:augmentedReality component to the page and make the 3D.obj model files available. At the time of writing this article, the models are simply displayed attached to the markers. From the component attributes, you can see that the next step is to implement selection functionality, allowing the user to "click" on one of the models. Over time, more interactivity with the 3D models can be added, eventually bringing the entire web into an augmented reality view.

But what is this for? It allows you to interact with objects in a natural context: imagine a parts catalog that allowed you to look at the new part alongside the failed part and see instructions for repair and installation. The possibilities are endless.

However, you may have noticed that your web browser doesn't appear to support Augmented Reality. How can we get the software we need onto the device, but keep the benefits of web deployment? There is one dominant feature of the mobile landscape that is not imposed on us by physics: the (insert platform-specific application store name here).

These stores are a boon to application developers: they allows users to safely find, purchase, and install software and provide a reliable way to deliver payment for that software to developers. They are also a bane to application developers. If an application is not approved, there is no legitimate alternative means for distribution. Furthermore, sandboxing requirements imposed by store distribution conflict with a wide variety of applications. On top of that, the store owners take a significant cut of the revenue. You will observe, however, that the complaints about "store cut" are invariably made by competing application stores. This indicates that the distribution benefit provided to individual application developers vastly outweighs the revenue loss from the store cut.

ICEmobile-SX

As web developers, we are accustomed to easily updating applications under our organization's control, not some other entity, so we need to be able to update our applications without a round trip through the store review process. An important goal of ICEmobile is to allow us to keep the web deployment model, yet take advantage of rich mobile capabilities. The application store landscape still exerts a powerful effect, however, and the outcome of this was ICEmobile-SX.

Figure 2: ICEmobile-SX

Originally, ICEmobile was intended to function via a "web container", which is a device native application containing a web view extended with a JavaScript API for device capabilities. This is similar to PhoneGap, but different in the goal to support a server-side web application rather than a standalone JavaScript application.

This was not accepted by all of the application stores likely because of its inherent flexibility: with the ability to download arbitrary web resources and scripts, the ICEmobile Container could easily diverge substantially from what was reviewed at application submission time. The container variant is available in the open source bundle, however, and you can freely use it in managed enterprise deployments. Also, application review policies are always changing, allowing more dynamic scripting capabilities in applications.

To solve this, the native features of ICEmobile have been factored out into an app that does not contain a web view or JavaScript interpreter. This is called ICEmobile-SX for "Surf Expander" since it "expands" the capabilities of an existing web browser.

The benefits of this are twofold: ICEmobile-SX can actually be distributed by all application stores for the various platforms, and it allows the web application to run on the stock browser which is often significantly better maintained that the API-accessible web view.

Cloud Push

Let's wrap up with another mobile-specific feature, but one that has immediate use in today's enterprise applications: Cloud Push.

Cloud Push enhances Ajax Push by adding support for the notification clouds of various devices, as well as other common infrastructures, such as SMS or email.

Wondering what Ajax Push is? Closely related to Comet, Ajax Push adds "push" or application-initiated update capability to ajax. It makes web pages collaborative and interactive rather than acting like printouts on the screen. When two users are working in the same web application, changes made by one user can be pushed to the other user, allowing them to work more efficiently without reloading pages or being blocked by other users' changes. The main difference between Ajax Push and Comet is that Ajax Push is purely notification-based and Comet is message-based. Comet is used to transport data, whereas Ajax Push sends just a notification that prompts the application to exchange data in its typical fashion (such as an Ajax page update).

To understand Ajax Push better, consider the simplest possible collaborative application: a page that shares a text field between multiple users (actually, sharing a single bit via a checkbox would be even simpler in collaborative terms, but the code is the same, and the demo would be boring, so we'll use a text field).

What follows is complete working code; it's really very simple to develop. As a JSF application we have our page:

Listing 2

<!DOCTYPE html > 
<html xmlns="http://www.w3.org/1999/xhtml" 
      xmlns:h="http://java.sun.com/jsf/html" 
      xmlns:f="http://java.sun.com/jsf/core" > 
<h:head> 
    <title>Ajax Push Example</title> 
</h:head> 
<h:body> 

  <h:form> 
    <h:inputText value="#{bean.text}" /> 
  </h:form> 
        
</h:body> 
</html>

 

The ICEfaces rendering process automatically makes this an Ajax application (no need for region-based events and rendering via (f:ajax>). Our bean is just as simple and includes the Ajax Push code to add all users into the "jax" group and then render all users in the "jax" group when the text value is updated.  

Listing 3

 

import javax.faces.bean.ManagedBean; 
import javax.faces.bean.ApplicationScoped; 
import org.icefaces.application.PushRenderer; 

@ManagedBean 
@ApplicationScoped 
public class Bean  { 

  String text = "blank"; 
  public String getText()  { 
    PushRenderer.addCurrentSession("jax"); 
    return text; 
  } 

  public void setText(String text)  { 
    PushRenderer.render("jax"); 
    this.text = text; 
  } 

}

 

When a user changes the text value via the bean setter, all sessions in the "jax" group are notified, resulting in page updates being requested by the browsers and converted to minimal page update instructions automatically by ICEfaces. When the push occurs, all users will see a common value in the text field.

Now let's consider the mobile environment. Push updates all work fine as long as the user stays in their web browser, but with a small screen running on battery power, it's not unusual for the user to switch to a different application or to put the device to sleep. Cloud Push allows the Ajax Push notification to still be delivered: when the Ajax Push connection to the browser is severed (potentially even due to lost connectivity), Cloud Push is engaged with a device-specific notification cloud. On iOS, this means Apple Push Notification Service, on Android, this means Google Cloud Messaging, and for BlackBerry, this means BlackBerry Push Notification service. Or, the user may have configured a preference for email or SMS, causing an email or SMS message to be sent. From the developer's point of view, handling this is trivial.

Starting from the ICEfaces Ajax Push code already in the application:

PushRenderer.render("jax");

This is changed to: 

PushRenderer.render("jax", new PushMessage("ICEmobile Notification", text););

where the title and text are the contents of the notification to be displayed on the device. Once again, the key difference from Comet is that only notification is sent, no data (the text values are not data for the application, they are displayed to the user in the notification). This allows the user to accept the notification on their home screen at any time, and once accepted, be taken to the browser which is refreshed via an Ajax update. If the push mechanism transported data (like Comet does), data would have been lost in this round trip through the home screen. In this way, Ajax Push is well suited to mobile infrastructure.

ICEmobile is open sourced under the Apache license and has a growing community based on the existing ICEfaces users (but you don't need to use JSF since ICEmobile also now supports JSP).

The goal is to both push the boundaries of what can be done with the mobile web (such as with Augmented Reality) and to provide a generally useful framework for building mobile enterprise web applications (such as with Cloud Push). With the introduction of the ICEmobile JSP tags, all features are supported for JSF, Spring, and JSP developers. We invite you to try it out and respond with your feedback.

Author Bio: Ted Goddard is the chief software architect at ICEsoft Technologies, and the technical lead for the ICEfaces, ICEpush, and ICEmobile frameworks. Following a PhD in mathematics from Emory University that answered open problems in complexity theory and infinite colorings for ordered sets, he proceeded with post-doctoral research in component and Web-based collaborative technologies. Goddard has held positions at Sun Microsystems, AudeSi Technologies, and Wind River Systems. He currently participates in the Java Servlet and JavaServer Faces expert groups.

This article appeared in JAX Magazine: Reality Check in March. For that issue and others click here.

Ted Goddard
Ted Goddard

What do you think?

JAX Magazine - 2014 - 06 Exclucively for iPad users JAX Magazine on Android

Comments

Latest opinions