ONJava.com    
 Published on ONJava.com (http://www.onjava.com/)
 See this if you're having trouble printing code examples


Building Wireless Web Clients, Part 2

by Kim Topley
05/15/2002

The first part of this article showed how to use the HTTP support in a MIDP device to connect to Amazon's online bookstore and fetch details for a book, given its ISBN. The choice of ISBN as the key was based on the fact that the Amazon Web server provides a query that returns the details for a book given its ISBN, whereas searching by other means (such as author name or book title) sometimes results in an intermediate page being delivered, buried within which is a link to the page that the application needs.

From the user's point of view, however, ISBN is about the worst choice that could have been made. How many ISBNs do you carry around in your head? Even the most zealous of authors, I'm prepared to bet, don't memorize the ISBNs of all (or even any) of their books. To make the user's life easier, the second part of this article creates a local bookstore for the user. The local bookstore retains the details of all books previously obtained with the ISBN application, so the user can locally look up books by title. The application then gets the ISBN from the stored book details and uses the ISBN query to download the latest information from Amazon. In addition, the bookstore retains the sales ranking and the number of reviews, so that they can be presented from the local cache without having to make a network connection. You can also compare the old (local) figures with the latest data fetched from Amazon.

The Bookstore Web Client

The bookstore is implemented as a new MIDlet, called PersistentRankingMIDlet, which is in the same MIDlet suite as the MIDlet shown in the first part of this article. When this MIDlet executes for the first time, it prompts you to enter an ISBN and then fetches its title, sales ranking, and number of reviews, just as the first MIDlet did. It then stores these details on the device so that the next time you start the MIDlet, you will see a list of the books that you have already made queries for, as shown in the leftmost screenshot of Figure 1. This list is presented in alphabetical order, sorted by the book's title.

Screen shot.
Figure 1. List of books previously queried.

When you select a book from the list, the most recently obtained details are displayed, along with a button called Menu that allows you to access a menu of possible operations, shown in Figure 2.

Screen shot.
Figure 2. Menu options.

In this menu, the New operation lets you enter a new ISBN, and is therefore equivalent to the functionality provided by the RankingMIDlet developed in the first part of this article. The Delete option removes a book from the bookstore. Finally, the Details menu item causes the application to connect to Amazon.com, refresh the book details, and present a screen showing the difference between the new and old sales rankings and number of reviews. Figure 3 shows that the book has a ranking of 7,303, up 528 from the last time the device queried for information on the book.

Screen shot.
Figure 3. Display of current data from Amazon.com.

Storing Book Details

The extra facilities provided by this MIDlet require the ability to store book details on the MIDP device. All MIDP devices will provide some kind of long-term storage that is guaranteed to be preserved, at least while the device has some kind of power applied to it, which in the worst case means as long as the battery powering the device is not allowed to become completely discharged. Different device types provide different types of storage, so the MIDP profile defines a device-independent programming interface that allows a MIDlet to use whatever storage is available without needing to be aware of how it is actually implemented on the device. The API for this record management system is provided in the javax.microedition.rms package, a complete description of which, along with annotated reference information, can be found in O'Reilly's J2ME in a Nutshell.

J2ME in a Nutshell

Related Reading

J2ME in a Nutshell
By Kim Topley

What is the Bookstore?

The MIDP record-management APIs are based around the RecordStore class, which represents a collection of related records. Each RecordStore on a device is created and managed independently and belongs exclusively to the MIDlet suite containing the MIDlet that created it. All MIDlets in this suite can read and write records in any of their shared RecordStores and can delete the individual RecordStores if required. On the other hand, for security reasons, MIDlets cannot access or even find out about the existence of RecordStores belonging to other MIDlet suites. The tight binding between a MIDlet suite and its RecordStore is also apparent in a couple of other ways:

For the purposes of the bookstore client, we create a class called BookStore that provides a higher-level interface, which allows the application to work with the data for an individual book rather than the records that the RecordStore class deals with. As you'll see later, those records are a very primitive concept. The BookStore class maps directly to a RecordStore called, appropriately enough, BookStore, which is automatically created or opened as required.

The static openRecordStore() method opens a RecordStore given its name:

public static RecordStore openRecordStore(String name, boolean create)

If a RecordStore with the given name exists, it is opened, and an appropriate RecordStore object is returned. If it does not exist and the create argument is true, then an empty store is created. If create is false, this method throws a RecordStoreNotFoundException, one of several exceptions derived from the base class RecordStoreException that can be thrown by methods in the classes of the javax.microedition.rms.

The BookStore class opens or creates its associated RecordStore when its constructor is executed:

public class BookStore implements RecordComparator, RecordFilter {

   // The name of the record store used to hold books
   private static final String STORE_NAME = "BookStore";

   // The record store itself
   private RecordStore store;

   // Creates a bookstore and opens it
   public BookStore() {
  try {
 store = RecordStore.openRecordStore(STORE_NAME, true);
  } catch (RecordStoreException ex) {
 // Error handling not shown
  }
   }

A RecordStore can be opened several times by a single MIDlet and can also be open for access simultaneously by more than one MIDlet (in the same suite, of course). The RecordStore implementation keeps track of the number of times that a given store has been opened. This count is decremented when the RecordStore's closeRecordStore() method is called; the underlying storage is closed (if this concept exists) only when the RecordStore has been closed as many times as it was opened. This means that each invocation of openRecordStore() must be balanced by a corresponding call to closeRecordStore().

The RecordStore class has several global operations that operate at the record store level, including:

public static String[] listRecordStores()
Returns an array containing the names of the available RecordStores. For security reasons, only the names of those RecordStores created by the MIDlet suite containing the calling MIDlet appear in this array.

public static void deleteRecordStore(String name)
Deletes the record store with the given name. The name is, of course, interpreted relative to the record stores owned by the calling MIDlet's suite, so that storage belonging to other suites cannot be deleted. A RecordStore cannot be deleted while it is open.

public void addRecordListener(RecordListener l)
Registers a listener to be notified when the content of the RecordStore changes. The RecordListener interface defines methods that are called when records in the RecordStore are added, removed, or updated.

public void removeRecordListener(RecordListener l)
Removes a listener previously registered using addRecordListener().

public int getSize()
Returns the number of bytes of storage occupied by the RecordStore, which includes any private storage management information required by the implementation.

public int getSizeAvailable()
Returns the number of bytes by which the RecordStore can grow. Since the implementation requires some space for private management information, the number of bytes available for actual MIDlet data will usually be less than the value returned by this method.

public long getLastModified()
Returns the time at which the RecordStore was last changed, in the same form as the value returned by the System method currentTimeMillis().

public int getVersion()
Gets the version number of the RecordStore. This value is changed whenever the content of the RecordStore is changed in any way. Checking this value is a quick way to determine whether the store has been modified.

public int getNumRecords()
Gets the number of records in the RecordStore.

The BookStore class provides a simpler interface that allows applications to work in terms of the BookStore itself, instead of dealing with the underlying RecordStore. Here, for example, are the methods that get the number of books in the BookStore and allow the BookStore to be closed. As you can see, they both delegate directly to the corresponding RecordStore methods:

// Closes the bookstore
public void close() throws RecordStoreException {
    if (store != null) {
       store.closeRecordStore();
    }
}

// Gets the number of books in the bookstore
public int getBookCount() throws RecordStoreException {
   if (store != null) {
        return store.getNumRecords();
    }
    return 0;
}

What is Stored in the Bookstore?

In the first part of this article, we used HTTP to get the details for a book and convert them into an instance of the BookInfo class. This class, which we didn't show in the first part of the article, has the following instance variables:

public class BookInfo {

    int    id;          // Used when persisting
    String isbn;        // The book ISBN
    String title;       // The book title
    int    reviews;     // Number of reviews
    int    ranking;     // Current ranking
    int    lastReviews; // Last review count
    int    lastRanking;  // Last ranking

}

The isbn variable is the book ISBN obtained from the user. The values of all of the other variable, with the exception of id (which will be described below), are extracted from the HTML. The reviews and ranking fields hold the current values, while lastReviews and lastRanking are the values that were obtained on the previous query. Each time the book's Web page is fetched from the server, the value of reviews and ranking are copied to lastReviews and lastRanking, respectively, and the values extracted from the Web page are written to reviews and ranking.

The BookInfo class is designed to hold all of the information relating to a book in the BookStore. The intent is that each BookInfo instance will be mapped to a record in the underlying RecordStore. In terms of the RecordStore APIs, a record is simply a contiguous sequence of bytes, the meaning of which is opaque to the platform and to the classes in the javax.microedition.rms package. The RecordStore class provides several methods that operate on records:

public int addRecord(byte[] data, int offset, int length)
Creates a new record initialized with the specified portion of an array of bytes, returning a unique integer identifier known as recordId.

public void setRecord(int recordId, byte[] data, int offset, int length)
Replaces the content of the record identified by recordId with the given portion of an array of bytes. This may cause the record to grow or to become smaller.

public byte[] getRecord(int recordId)
Returns the content of the record with a given recordId.

public int getRecord(int recordId, byte[] buffer, int offset)
Reads the given record into the supplied array, starting at the given offset, and returns the number of bytes that were read.

public void deleteRecord(int recordId)
Deletes the record with the given recordId.

In order to add a record for a new book or replace the data for an existing book, we need to create an array of bytes that represents its BookInfo instance and use either the addRecord() or setRecord() methods to write the bytes to the RecordStore. If you were using J2SE, you might take advantage of object serialization to create a serialized version of each instance of BookInfo for storage, or use the Java Beans persistance mechanism introduced in Java 2 version 1.4 to flatten the object into an XML representation. Unfortunately, neither of these is available to MIDlets. The most convenient way for a MIDlet to convert a class instance into a record is to use a DataOutputStream in conjunction with a ByteArrayOutputStream. Here, for example, is how the BookStore class converts a BookInfo object into a byte array:

    // Writes a record into a byte array.
private byte[] toByteArray(BookInfo bookInfo) throws IOException {
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    DataOutputStream os = new DataOutputStream(baos);

    os.writeUTF(bookInfo.isbn);
    os.writeUTF(bookInfo.title == null ? "" : bookInfo.title);
    os.writeInt(bookInfo.ranking);
    os.writeInt(bookInfo.reviews);
    os.writeInt(bookInfo.lastRanking);
    os.writeInt(bookInfo.lastReviews);

    return baos.toByteArray();
}

The function of the ByteArrayOutputStream is to write everything that it receives into a buffer that can be extracted in the form of a byte array, while the DataOutputStream provides convenience methods that write Java primitives, types, and strings in a platform-independent way to an output stream.

You'll notice that the toByteArray method does not write out the id field from the BookInfo class. This is because each record is implicitly associated with its own recordId and therefore it does not need to be stored in the record content. The recordId is a unique positive integer value assigned when the record is created. Each time a record is written, it is assigned the next recordId, starting from 1. The software guarantees that recordIds are not reused. In particular, if you delete a record, its recordId is not reassigned at any time in the future. The getNextRecordID() method can be used to find out the recordId for the next record to be written to the RecordStore.

Reading the content of a record and converting it to a BookInfo object is a simple matter of reversing the steps shown above, using a ByteArrayInputStream and a DataInputStream to map the byte array returned from the RecordStore method getRecord() to the original Java types:

public BookInfo getBookInfo(int id) throws RecordStoreException,
                                                IOException {
    byte[] bytes = store.getRecord(id);
    DataInputStream is = new DataInputStream(
        new ByteArrayInputStream(bytes));

    String isbn = is.readUTF();
    BookInfo info = new BookInfo(isbn);
    info.id = id;
    info.title = is.readUTF();
    info.ranking = is.readInt();
    info.reviews = is.readInt();
    info.lastRanking = is.readInt();
    info.lastReviews = is.readInt();

    return info;
}

This method assumes that we already have the recordId for the book whose details we require. So how do we get this value?

Getting the List of Books in the Bookstore

In the PersistentRankingMIDlet, we need to get the set of BookInfo objects for all of the books that the user has stored in the RecordStore when the MIDlet starts up. If there are no records available (as will be the case when it is executed for the first time following installation or if all previously stored records have been deleted), we need to display the screen that allows the user to enter an ISBN. Otherwise, the list of books should be displayed so that the user can select one to view.

The only way to retrieve a record from the RecordStore is to call the getRecord() method, passing the required recordId. The problem is, there is no way to know which recordId values correspond to valid records. One way to tackle this problem would be to use code like this:

for (int i = 1, count = store.getNextRecordID(); i < count; i++) {
    try {
    byte[] bytes = store.getrecord(i);

    
    // do something with the record
    } catch (invalidrecordidexception ex) {
    // no record for this record id
    }
}

The trouble with this code is that it is going to be very slow after the RecordStore has been in use for some time. As long as records have not been deleted, this code will behave properly. Consider, however, the extreme case in which 100 records have been added to the RecordStore over time, but only the first and the most recent remain. This means that record IDs 1 and 100 are currently valid, but 2 through 99 are not. In order to discover this, the above loop iterates over all 100 possible IDs, successfully reading the first, but then encountering 98 successive exceptions for invalid record IDs, before reading record 100. Not only is this a waste of 98 iterations of the loop, it is also going to be slow, because it is very time-consuming to construct and throw an exception. Incidentally, it is possible to create this situation with as little as two records in the RecordStore at any given time because record IDs are never reused. Here is how that might happen:

  1. Enter the first record, which gets record ID 1.
  2. Enter the second record, which gets record ID 2.
  3. Delete the second record, making record ID 2 invalid.
  4. Enter the third record, which gets record ID 3.
  5. Delete the third record, making record ID 3 invalid.
  6. Enter the fourth record, which gets record ID 4.
  7. and so on...

Using a RecordEnumeration

Fortunately, there is a much more efficient way to tackle this problem. Instead of walking through all of the record IDs that have ever been assigned, you can get a RecordEnumeration that contains any subset of the content of a RecordStore. A RecordEnumeration is like a java.util.Enumeration, in that it allows you to iterate over a collection of objects. It is, however, more powerful than Enumeration because you can traverse the collection either forward or backward, and you can change direction at any time. For further details, refer to the API reference or Chapter 6 of J2ME in a Nutshell.

In order to get a RecordEnumeration, use the following RecordStore method:

public RecordEnumeration enumerateRecords(RecordFilter filter,
  RecordComparator comparator,
  boolean keepUpdated)

RecordFilter and RecordComparator are interfaces that define methods that allow you to exclude records from the enumeration and determine the order in which the records are returned, respectively. If the filter argument is null, then all records are included, while a null comparator causes the order of the records to be undefined. The following call, therefore, returns a RecordEnumeration containing all of the records of the RecordStore in no particular order:

RecordEnumeration enum = recordStore.enumerateRecords(null, null, false);

The keepUpdated argument determines whether the enumeration is static or dynamic. If this argument is false, the enumeration represents a snapshot of the state of the RecordStore when enumerateRecords() is called. If keepUpdated is true, however, changes in the content of the RecordStore will be visible through the enumeration (unless the changes involve records that are excluded by the filter). It is more efficient to create an enumeration with keepUpdated set to false, because it can be expensive to keep the enumeration in step with the underlying RecordStore. An alternative way to react to changes is to register a RecordListener and handle them in your own code.

The RecordFilter Interface

The RecordFilter interface defines a single method:

public boolean matches(byte[] data)

This method should be implemented to return true if the record whose content is passed to it meets the filter criterion and false if it does not. Only those records for which true is returned will be included in the RecordEnumeration. The BookStore class contains an example of a RecordEnumeration used with a RecordFilter in the saveBookInfo method:

// Adds an entry to the store or modifies the existing
// entry if a matching ISBN exists.
    public void saveBookInfo(BookInfo bookInfo)
                                throws IOException, RecordStoreException {
    if (store != null) {
      <b>searchISBN = bookInfo.getIsbn();
      RecordEnumeration enum = store.enumerateRecords(
                    this, null, false);
      if (enum.numRecords() > 0) {</b>
        // A matching record exists. Set the id
        // of the BookInfo to match the existing record
        bookInfo.id = enum.nextRecordId();
        byte[] bytes = toByteArray(bookInfo);
        store.setRecord(bookInfo.id, bytes, 0, bytes.length);
      } else {
        // Create a new record
        bookInfo.id = store.getNextRecordID();
        byte[] bytes = toByteArray(bookInfo);
        store.addRecord(bytes, 0, bytes.length);
      }

      // Finally, destroy the RecordEnumeration
      enum.destroy();
    }
}

This method stores the content of a BookInfo record in the BookStore. It needs to create a new record by calling addRecord() if the book does not already have an entry, or update the existing entry using setRecord() if it does. In order to do this, it needs to search the RecordStore for a record with the same ISBN as the one in the BookInfo object. As before, it would be inefficient to use a loop over all of the possible record IDs to find a record with a given ISBN. Instead, this code uses the enumerateRecords() method with a filter that returns true only for a record with the matching ISBN, but with no comparator. The enumeration will consist of either one record or no records.

In this case, the BookStore class itself implements the RecordFilter interface, so the filter reference is passed as this and the ISBN to be searched for is saved in the searchISBN instance variable. Here is the implementation of the filter:

// RecordFilter implementation
public boolean matches(byte[] book) {
    if (searchISBN != null) {
      try {
        DataInputStream stream =
          new DataInputStream(new ByteArrayInputStream(book));

        // Match based on the ISBN.
        return searchISBN.equals(stream.readUTF());
      } catch (IOException ex) {
        System.err.println(ex);
      }
    }

    // Default is not to match
    return false;
}

This method receives the content of a record in the form of a byte array, then wraps a ByteArrayInputStream and a DataInputStream around it so that the fields within the record can be obtained. As shown in the implementation of the toByteArray() method above, the ISBN is the first field written to the record, so it can easily be retrieved by calling the readUTF() method of DataInputStream and comparing it to the ISBN being searched for.

The RecordComparator Interface

Like RecordFilter, RecordComparator defines only one method:

public int compare(byte[] rec1, byte[] rec2)

This method is passed the content of two records and is expected to compare them to determine their relative ordering. The return value should be RecordComparator.PRECEDES if rec1 comes before rec2, RecordComparator.FOLLOWS if rec1 comes after rec2, and RecordComparator.EQUIVALENT if they are equivalent (have the same ISBN). The BookStore class uses a comparator when retrieving the list of all books for which there is stored information, from which the list initially displayed to the user is constructed:

// Gets a sorted list of all of the books in
// the store.
public RecordEnumeration getBooks() throws RecordStoreException {
    if (store != null) {
        return store.enumerateRecords(null, this, false);
    }
    return null;
}

Here, a RecordEnumeration that contains all of the records in the RecordStore is created, sorted according to the comparator, which is implemented by the BookStore class itself:

  // RecordComparator implementation
  public int compare(byte[] book1, byte[] book2) {
    try {
      DataInputStream stream1 =
        new DataInputStream(new ByteArrayInputStream(book1));
      DataInputStream stream2 =
        new DataInputStream(new ByteArrayInputStream(book2));

      // Match based on the ISBN, but sort based on the title.
      String isbn1 = stream1.readUTF();
      String isbn2 = stream2.readUTF();
      if (isbn1.equals(isbn2)) {
        return RecordComparator.EQUIVALENT;
      }
      String title1 = stream1.readUTF();
      String title2 = stream2.readUTF();
      int result = title1.compareTo(title2);
      if (result == 0) {
        return RecordComparator.EQUIVALENT;
      }
      return result < 0 ? recordcomparator.precedes :
                recordcomparator.follows;
    } catch (ioexception ex) {
      return recordcomparator.equivalent;
    }
  }

This method is really much simpler than it looks. All it is doing is wrapping both records with streams that allow the primitives to be extracted, and then extracting both the ISBN and title fields from both. The records are considered to be equivalent if they have the same ISBN. Otherwise, the ordering is determined by an alphabetical comparison of the books' titles.

Deleting a Book's Entry from the Bookstore

The last operation we need to be able to perform is to remove the entry for a book when the user selects the delete option from the menu shown in Figure 1. This is very simple to implement, using the deleteRecord() of the RecordStore class. The only attribute we need to perform this operation is the record ID of the book's record which, of course, we keep in the BookInfo object for this very purpose:

// Deletes the entry for a book from the store
public void deleteBook(BookInfo bookInfo) throws RecordStoreException {
    if (store != null) {
        store.deleteRecord(bookInfo.id);
    }
}

Summary

The second part of this article has shown you how to use the RecordStore class in the javax.microedition.rms package to provide persistent storage for the data managed by a MIDlet. Although the facilities provided are somewhat primitive, you have seen that it is quite straightforward to implement a wrapper class that can offer higher-level operations to application code.

A couple of final observations on this application and the current state of the storage APIs for J2ME itself:

First, the javax.microedition.rms package only supports access to data created by and belonging to the calling MIDlet suite. However, some developments that will change this situation are likely to arrive later this year. The next version of MIDP, for example, will extend the RecordStore class so that a MIDlet suite can allow other suites to access its data. More significantly, the PDA profile includes the ability for Java code to access some of the native personal information on the device that is currently available only to native applications. This might, for example, include an address book, a calendar, or a to-do list. None of these are currently accessible to MIDP applications and, unfortunately, it seems that there is no plan to incorporate this feature into MIDP version 2.0, with the result that only code running on the higher-end platforms will be able to access these databases.

Secondly, the technique used in this application to obtain information from the Amazon Web site is very brittle, because it relies heavily on the format of the Web page that is returned from the server. In fact, this format has changed a couple of times since I wrote the first version of this code, and the application code had to change both times. The reason for this is, of course, that the data returned from the server is really intended for a human user, not for an application. That said, there are many other applications that use the same "screen-scraping" technique, including most of the stock-price ticker applications and MIDlets that are commonly available. In reality, this type of application should be created using a business-to-business interface, encoded in XML, with a formal specification of the services available and how to invoke them. This specification then becomes a contract between the service provider and the client and is therefore not subject to the type of change that breaks the bookstore client. What I have just described is, of course, the classic definition of a Web service. Work is already under way to define the way in which J2ME clients will access Web services. When this work is complete, perhaps it will be possible to build a version of this application that uses a formal interface to the Amazon Web server -- or other booksellers' sites -- to obtain its data.

Kim Topley has more than 25 years experience as a software developer and was one of the first people in the world to obtain the Sun Certified Java Developer qualification.


Return to ONJava.com.

Copyright © 2009 O'Reilly Media, Inc.