Tricky business

Just add mobility: JSF and JSP with ICEmobile

TedGoddard
icemobile1

Developing mobile UIs is a tricky business, especially with the market changing by the minute. ICEsoft chief software architect Ted Goddard explains how to get the most out of the medium.

Developing mobile UIs is a tricky business, especially with
the market changing by the minute. ICEsoft chief software architect Ted
Goddard explains how to get the most out of the medium, in this
JAX Magazine
article

The future of the web is mobile. This is clear
from the increasing use of both phones and tablets. Further, mobile
devices often have a simple and reliable software update process
which provides them with modern, updated, HTML5-compliant browsers.
That makes it great to develop for.

Modern mobile devices have an impressive range
of features that have the potential to create a very rich user
interface: CSS3 and HTML5, GPS and compass, front and rear video
cameras, accelerometers and gyroscopes, and 3D accelerated high
resolution displays. In a typical legacy desktop environment, it’s
very easy to find machines that have nowhere near these
capabilities.

Keep it simple and efficient

There are just two problems: mobile devices are
small and they’re mobile.

Small devices have tiny touchscreen controls
which are difficult to hit. This is complicated further by the
user’s finger blocking their view, so we have to use larger UI
targets, compounding the limited number of controls that can be
used on a small display. Text entry is fairly inefficient on such
devices. Of course, tablets are not as constrained, although they
do tend to have smaller screens than current desktops.

Mobility itself raises two problem areas:
network and power. Mobile network
connections can be high latency, low bandwidth, or just not there
at all. Mobile devices run on limited battery power most of the
time, so we must be mindful not to waste this power through
excessive UI features, network traffic or other client-side CPU
use.

OK, there’s clearly more than
two issues, but the next biggest problem is psychological; why is
it necessary to have such elaborately styled mobile UIs? It has
become fashionable to apply the greatest ornamentation and
animation features to the smallest screens we have, often at the
expense of functionality and performance. Remember the blink tag?
It’s more important for the user to be productive than entertained.
(
This concludes the rant; we now
return you to the main article. Please continue to enjoy the
simulated rich Corinthian leather of your mobile
UI.)

Augmented Reality

Now that we’ve established that the mobile web
should be based on simplicity and power-efficiency, let’s look at a
user interface capable of draining the device battery in just a few
minutes: augmented reality.

Figure 1: Augmented
Reality device

Modern mobile phones and tablets have unique
sensors and displays that allow them to do things that are simply
not possible with a desktop computer. The differences won’t stop
here however – future devices will be head-mounted with a display
that is directly overlaid on the user’s vision; whether this is
useful or annoying will remain to be seen.

The main hardware components
we need are: a rear-facing camera for visual input from the “real
world”, a compass and accelerometer to find the actual device
orientation relative to the earth, a gyroscope to reliably track
orientation changes (the compass is too easily influenced by nearby
magnetic fields) and a front 3D accelerated display for viewing the
application.

With these capabilities, we can create two very
interesting Augmented Reality variants: virtual objects that appear
to be located in real space, and virtual objects that are attached
to real-world objects (sophisticated image recognition algorithms
are required here in addition to the hardware capabilities).

Does this sound like a web application? It is;
here’s the JSF markup for a simple demo that lets you choose which
puzzle blocks to display attached to augmented reality
“markers”:

Listing 1

 <mobi:fieldsetGroup> 

            <mobi:fieldsetRow > 
                <p>Select your desired 3D models:</p> 
            </mobi:fieldsetRow> 

            <mobi:fieldsetRow styleClass="mobi-center"> 
                <h:selectOneMenu value="#{realityBean.selectedModel1}"> 
                    <f:selectItem itemLabel="Puzzle Block 1" 
                            itemValue="puz1"/> 
                    <f:selectItem itemLabel="Puzzle Block 2" 
                            itemValue="puz2"/> 
                </h:selectOneMenu> 
                 <h:selectOneMenu value="#{realityBean.selectedModel2}"> 
                    <f:selectItem itemLabel="Puzzle Block 1" 
                            itemValue="puz1"/> 
                    <f:selectItem itemLabel="Puzzle Block 2" 
                            itemValue="puz2"/> 
                </h:selectOneMenu> 
           </mobi:fieldsetRow> 

            <mobi:fieldsetRow styleClass="mobi-center"> 
                <mobi:augmentedReality 
                    value="#{realityBean.selection}" > 
                    <mobi:augmentedRealityMarkers 
                        value="#{realityBean.markers}" var="marker" 
                        markerLabel="#{marker.label}" 
                        markerModel="#{marker.model}" /> 
                </mobi:augmentedReality> 
            </mobi:fieldsetRow> 

        </mobi:fieldsetGroup>

That’s all there is to it: just add the mobi:augmentedReality
component to the page and make the 3D.obj model files available. At
the time of writing this article, the models are simply displayed
attached to the markers. From the component attributes, you can see
that the next step is to implement selection functionality,
allowing the user to “click” on one of the models. Over time, more
interactivity with the 3D models can be added, eventually bringing
the entire web into an augmented reality view.

But what is this for? It allows you to interact with objects in
a natural context: imagine a parts catalog that allowed you to look
at the new part alongside the failed part and see instructions for
repair and installation. The possibilities are endless.

However, you may have noticed that your web browser doesn’t
appear to support Augmented Reality. How can we get the software we
need onto the device, but keep the benefits of web deployment?
There is one dominant feature of the mobile landscape that is not
imposed on us by physics: the (insert platform-specific application
store name here).

These stores are a boon to application developers: they allows
users to safely find, purchase, and install software and provide a
reliable way to deliver payment for that software to developers.
They are also a bane to application developers. If an application
is not approved, there is no legitimate alternative means for
distribution. Furthermore, sandboxing requirements imposed by store
distribution conflict with a wide variety of applications. On top
of that, the store owners take a significant cut of the revenue.
You will observe, however, that the complaints about “store cut”
are invariably made by competing application stores. This indicates
that the distribution benefit provided to individual application
developers vastly outweighs the revenue loss from the store
cut.

ICEmobile-SX

As web developers, we are accustomed to easily updating
applications under our organization’s control, not some other
entity, so we need to be able to update our applications without a
round trip through the store review process. An important goal of
ICEmobile is to allow us to keep the web deployment model, yet take
advantage of rich mobile capabilities. The application store
landscape still exerts a powerful effect, however, and the outcome
of this was ICEmobile-SX.

Figure 2:
ICEmobile-SX

Originally, ICEmobile was intended to function via a “web
container”, which is a device native application containing a web
view extended with a JavaScript API for device capabilities. This
is similar to PhoneGap, but different in the goal to support a
server-side web application rather than a standalone JavaScript
application.

This was not accepted by all of the application stores likely
because of its inherent flexibility: with the ability to download
arbitrary web resources and scripts, the ICEmobile Container could
easily diverge substantially from what was reviewed at application
submission time. The container variant is available in the open
source bundle, however, and you can freely use it in managed
enterprise deployments. Also, application review policies are
always changing, allowing more dynamic scripting capabilities in
applications.

To solve this, the native features of ICEmobile have been
factored out into an app that does not contain a web view or
JavaScript interpreter. This is called ICEmobile-SX for “Surf
Expander” since it “expands” the capabilities of an existing web
browser.

The benefits of this are twofold: ICEmobile-SX can actually be
distributed by all application stores for the various platforms,
and it allows the web application to run on the stock browser which
is often significantly better maintained that the API-accessible
web view.

Cloud Push

Let’s wrap up with another mobile-specific feature, but one that
has immediate use in today’s enterprise applications: Cloud
Push
.

Cloud Push enhances Ajax Push by adding support for the
notification clouds of various devices, as well as other common
infrastructures, such as SMS or email.

Wondering what Ajax Push is? Closely related to Comet, Ajax Push
adds “push” or application-initiated update capability to ajax. It
makes web pages collaborative and interactive rather than acting
like printouts on the screen. When two users are working in the
same web application, changes made by one user can be pushed to the
other user, allowing them to work more efficiently without
reloading pages or being blocked by other users’ changes. The main
difference between Ajax Push and Comet is that Ajax Push is purely
notification-based and Comet is message-based. Comet is used to
transport data, whereas Ajax Push sends just a notification that
prompts the application to exchange data in its typical fashion
(such as an Ajax page update).

To understand Ajax Push better, consider the simplest possible
collaborative application: a page that shares a text field between
multiple users (actually, sharing a single bit via a checkbox would
be even simpler in collaborative terms, but the code is the same,
and the demo would be boring, so we’ll use a text field).

What follows is complete working code; it’s really very simple
to develop. As a JSF application we have our page:

Listing 2

<!DOCTYPE html > 
<html xmlns="http://www.w3.org/1999/xhtml" 
      xmlns:h="http://java.sun.com/jsf/html" 
      xmlns:f="http://java.sun.com/jsf/core" > 
<h:head> 
    <title>Ajax Push Example</title> 
</h:head> 
<h:body> 

  <h:form> 
    <h:inputText value="#{bean.text}" /> 
  </h:form> 
        
</h:body> 
</html>

 

The ICEfaces rendering process automatically makes this an Ajax
application (no need for region-based events and rendering via
(f:ajax>). Our bean is just as simple and includes the Ajax Push
code to add all users into the “jax” group and then render all
users in the “jax” group when the text value is updated.  

Listing 3

 

import javax.faces.bean.ManagedBean; 
import javax.faces.bean.ApplicationScoped; 
import org.icefaces.application.PushRenderer; 

@ManagedBean 
@ApplicationScoped 
public class Bean  { 

  String text = "blank"; 
  public String getText()  { 
    PushRenderer.addCurrentSession("jax"); 
    return text; 
  } 

  public void setText(String text)  { 
    PushRenderer.render("jax"); 
    this.text = text; 
  } 

}

 

When a user changes the text value via the bean setter, all
sessions in the “jax” group are notified, resulting in page updates
being requested by the browsers and converted to minimal page
update instructions automatically by ICEfaces. When the push occurs, all
users will see a common value in the text field.

Now let’s consider the mobile environment. Push updates all work
fine as long as the user stays in their web browser, but with a
small screen running on battery power, it’s not unusual for the
user to switch to a different application or to put the device to
sleep. Cloud Push allows the Ajax Push notification to still be
delivered: when the Ajax Push connection to the browser is severed
(potentially even due to lost connectivity), Cloud Push is engaged
with a device-specific notification cloud. On iOS, this means Apple
Push Notification Service, on Android, this means Google Cloud
Messaging, and for BlackBerry, this means BlackBerry Push
Notification service. Or, the user may have configured a preference
for email or SMS, causing an email or SMS message to be sent. From
the developer’s point of view, handling this is trivial.

Starting from the ICEfaces Ajax Push code already in the
application:

PushRenderer.render(“jax”);

This is changed to: 

PushRenderer.render(“jax”,
new PushMessage(“ICEmobile Notification”, text););

where the title and text are the contents of the notification to
be displayed on the device. Once again, the key difference from
Comet is that only notification is sent, no data (the text values
are not data for the application, they are displayed to the user in
the notification). This allows the user to accept the notification
on their home screen at any time, and once accepted, be taken to
the browser which is refreshed via an Ajax update. If the push
mechanism transported data (like Comet does), data would have been
lost in this round trip through the home screen. In this way, Ajax
Push is well suited to mobile infrastructure.

ICEmobile is open sourced under the Apache license and has a
growing community based on the existing ICEfaces users (but you
don’t need to use JSF since ICEmobile also now supports JSP).

The goal is to both push the boundaries of what can be done with
the mobile web
(such as with Augmented Reality) and to provide a generally useful
framework for building mobile enterprise web applications (such as
with Cloud Push). With the introduction of the ICEmobile JSP tags,
all features are supported for JSF, Spring, and JSP developers.
We invite you to try it
out
and respond with your feedback.

Author Bio: Ted Goddard is the chief
software architect at ICEsoft Technologies, and the technical lead
for the ICEfaces, ICEpush, and ICEmobile frameworks. Following a
PhD in mathematics from Emory University that answered open
problems in complexity theory and infinite colorings for ordered
sets, he proceeded with post-doctoral research in component and
Web-based collaborative technologies. Goddard has held positions at
Sun Microsystems, AudeSi Technologies, and Wind River Systems. He
currently participates in the Java Servlet and JavaServer Faces
expert groups.

This article appeared in JAX Magazine: Reality Check in
March. For that issue and others click here.

Author
TedGoddard
Ted Goddard is the chief software architect at ICEsoft Technologies, and the technical lead for the ICEfaces, ICEpush, and ICEmobile frameworks. Following a PhD in mathematics from Emory University that answered open problems in complexity theory and infinite colorings for ordered sets, he proceeded with post-doctoral research in component and Web-based collaborative technologies. Goddard has held positions at Sun Microsystems, AudeSi Technologies, and Wind River Systems. He currently participates in the Java Servlet and JavaServer Faces expert groups.
Comments
comments powered by Disqus