ONJava.com    
 Published on ONJava.com (http://www.onjava.com/)
 See this if you're having trouble printing code examples


A Gentle Re-Introduction to QuickTime for Java

by Chris Adamson
05/14/2003

This article is going to be a bit of a restart, a "retro intro" if you will, to the topic of QuickTime for Java (QTJ). In two previous articles, one on using QTJ as a helper for the Java Media Framework and another on the inner workings of the QuickTime file format, I've covered essential concepts and important issues only as they related to other topics, perhaps burying them to a degree. Feedback suggests the time is right to go back and do a beginner-level introduction to QTJ.

It's not that we think O'Reilly readers skip the developer docs or ignore the demos that come with the SDK, but we just want to be sure we're good before we hit the hard stuff. After all, there's matrix math coming up when we get to sprite animation ...

I will be using this article in the future for reference purposes, in the sense of "If this is your first time writing a QTJ app, go back to this article and make sure you've got everything set up correctly." Between horrid CLASSPATH issues and the obtuse situation with QTJ and Java 1.4.1 on Mac OS X, it'll be nice to not have to reiterate three paragraphs of gotchas every time.

A Little Background ...

Apple's QuickTime for Java is a Java wrapper around much of the native QuickTime media API, which itself has been available and under ongoing development since 1991. It is available for the classic Mac OS, Mac OS X, and Windows. Note the obvious omission: it is not available for Linux or any other OS at this time; nor is there an all-Java version.

Related Reading

Mac OS X for Java Geeks
By Will Iverson

QuickTime is sometimes thought of as just a media format and player, a rival to Windows Media and Real Player, with support for a wide variety of media formats, including some that most people aren't aware of (such as Flash 5 and Photoshop). But that thinking overlooks the fact that it was designed as a media creation API, with tools to capture and edit media, apply effects, and to export to different formats. QuickTime also includes a legacy imaging API, sprites, and support for interactivity within a movie. QuickTime used to include a 3D API, QuickDraw 3D, but it was dropped in Mac OS X in favor of OpenGL. We won't discuss it again.

QuickTime had been around for years before the QTJ project started, and was designed for use with straight C — not necessarily C++ or Objective-C. While QTJ re-envisions QuickTime from a more object-oriented perspective, QTJ code still seems different than what you might be used to working with in JavaSoft's APIs. For example, while most classes in core Java and the standard extensions define constants in the classes where they are used (e.g., Label.LEFT, Integer.MAX_VALUE), QTJ defines massive lists of constants in StdQTConstants, StdQTConstants4, etc., which hints at their origin as members of big C header files like Movies.h.

By the way, Apple's sample code has a tendency to declare that it implements these no-method interfaces, which is an easy-to-write but hard-to-follow way of inheriting the constants. In my sample code, I will always explicitly reference the constant by class so you can find it again later.

When Not to Use QuickTime for Java

Let me start with the ultimate in simplicity — not using QTJ at all. Every once in a while, someone will post the to the quicktime-java list that they need to learn QTJ because they want to put a movie in an applet in a web page.

That is overkill. Since it requires the user to have Java, QuickTime, and QuickTime for Java all installed, the size of the potential audience is smaller than it would be without so many dependencies.

If all you want to do is put a QuickTime movie in a web page, just use a pair of simple HTML tags. Most browsers will pick up the <EMBED> tag, but since Internet Explorer is special, it needs an <OBJECT> tag. Here's a simple example of a suitable tag:

<object classid="clsid:02BF25D5-8C17-4B23-BC80-D3488ABDDC6B"
   width="160" height="136"
   codebase="http://www.apple.com/qtactivex/qtplugin.cab">
     <param name="src" VALUE="../media/keagy-ball-1.mov"/>
     <param name="autoplay" VALUE="false"/>

<embed src="../media/keagy-ball-1.mov"
width="160" height="136" autoplay="false"
pluginspage="http://www.apple.com/quicktime/download/"/>

</object>

Some of these values are boilerplate and never change, namely the <object>'s classid and codebase, and the <embed>'s pluginspage. Beyond that, the variables that you populate simply need to get repeated as both <param> children of the <object> tag and attributes of the <embed> tag. The only required values for the simplest case are:

I've added an entry, autoplay, as another attribute, mostly just to keep things interesting — it doesn't have a default value, since general autoplay behavior is determined by the user's QuickTime preferences. This and many more attributes are described in Apple's document "Embedding QuickTime for Web Delivery."

You could also drop the controller, instead controlling the movie through your own HTML buttons, links, or other page elements. You can do this with the QuickTime plug-in's JavaScript support. In this case, just add the attribute enablejavascript = true to the <embed> tag (it's not necessary for the <object> tag), give it a name attribute and parameter, and then refer to the movie by that name when you call methods such as Play() and Stop(), as in this example. More sophisticated methods are available for getting and setting movie properties like the playback rate and volume, examining the track structure of the movie, and getting limited access to sprite and QuickTime VR properties.

The downside of the JavaScript support is that it is only as good as the LiveConnect — or, in QuickTime 6, XPCOM — support in your browser. For QuickTime 6, that means that Mozilla and Internet Explorer work fine on Windows, but the situation is a disaster on the Mac, where only Mozilla supports JavaScript control of the QuickTime plug-in. IE is known not to work, and while Apple says that Mozilla-based browsers should be OK, Camino and OmniWeb don't work with my example, nor does Safari beta 2 (v.72).

One option to get a lot of mileage out of authoring (in lieu of coding) is to use the "Synchronized Multimedia Integration Language", or SMIL (pronounced "smile"). This XML markup, a WC3 standard, allows you to present audio, video, text, and other elements, from multiple sources, in one display, using the markup to arrange the elements temporally and spatially, indicating what goes where and when.

Here's an example QuickTime SMIL presentation. It shows some logos with background music, then offers links to multiple versions of a QuickTime movie:

<smil>
  <head>
    <layout>
      <root-layout id="root" width="320" height="240"
      background-color="white"/>
      <region id="main" width="320" height="240"
              z-index="1" fit="meet" />
      <region id="onjava-reg" width="294" height="82"
              z-index="1" left="13" top="79"/>
      <region id="macdevcenter-reg" width="202"
              height="88" z-index="1" left="59" top="76"/>
      <region id="and-reg" width="40" height="20"
              left="140" top="110"/>
      <region id="present-reg" width="80" height="20"
              left="120" top="110"/>
      <region id="link1-reg" width="80" height="60"
              left="240" top="60" z-index="2"/>
      <region id="link2-reg" width="80" height="60"
              left="240" top="135" z-index="2"/>
    </layout>
  </head>
  <body>
    <par>
    <!-- 15 seconds of audio parallel to a
         sequence of images -->
    <audio src="Worldwide-15.mp3"/>
    <seq>
      <img src="onjava_logo.jpg" region="onjava-reg"
       dur="03" />
      <img src="and.gif" region="and-reg" dur="01"/>
      <img src="macdevcenter_logo.gif" region="macdevcenter-reg"
       dur="3 sec" />
      <img src="present.gif" region="present-reg" dur="02"/>
      <!-- last thing in sequence is poster with
           link buttons! -->
      <par>
         <img src="keagy-closeup.jpeg" region="main"/>
         <a href="keagy-closeup-320-sv.mov" show="replace">
           <img src="link1.gif" region="link1-reg"/></a>
         <a href="keagy-closeup-160-sv.mov" show="new">
           <img src="link2.gif" region="link2-reg"/></a>
      </par>
    </seq>
    </par>
  </body>
</smil>

In the <head> tag, we define a number of spatial "regions". Our content will refer to these to indicate where it will be placed on the screen. You can have any number of regions, with several used at a time.

References to media go in the <body> section, grouped according to whether different items are "parallel" or in "sequence." Items in a <par> tag are shown or played at the same time, while those in a <seq> tag are shown or played one after another, either for their natural duration (if appropriate) or for the time specified in a "dur" attribute.

Our example pairs an audio clip (a 15-second soundtrack from FreePlay Music, whose royalty-free audio clips are available to .Mac users via the iDisk Software folder), with a sequence of elements. The first four members of the sequence are graphics: the ONJava logo, an "and" graphic, the MacDevCenter logo, and a "presents" graphic. Note that Apple's SMIL docs include a way to include text from a file or a data: URL, but I used graphics so I could control the font and color.

The last member of the sequence is another <par> tag, this time combining a poster frame, keagy-closeup.jpeg, with two image links to the movie. Notice that the link elements refer to two regions whose z-index is 2, while the main region has a z-index of 1. This causes the link graphics to be drawn on top of the poster image.

Among the many possible uses of SMIL, this illustrates a way to create "alternate movies" (movies that redirect to other QuickTime movies, perhaps to offer different sizes or bitrates). This is easier than the technique offered here. (If you don't scare easily, check out the low-level details of how this kind of real "alternate movie" works, along with an audacious QuickTime for Java implementation.)

Getting Started With QTJ

Let's assume that you really do need QuickTime for Java — you need to deliver as an application, you're writing a cool transcoding servlet, or whatever. Where should you begin?

First, check the system on your desk. Is it a Mac or a Windows box? Good. Sorry, no QuickTime on other operating systems, and thus no QTJ — and that includes CodeWeavers' CrossOver for Linux, which supports only the QuickTime browser plug-in, not QT in applications.

Next, you need to install QuickTime, which is available from Apple's site, if you don't already have it. I will assume from this point on that everyone is using QuickTime 6, which you need for some of the more interesting topics we'll be covering, such as MPEG-4. By the way, you don't need to purchase QuickTime Pro for QTJ development, though unlocking the editing and exporting features of Apple's player does make it easier to create test media for your development needs.

Mac OS X users can expect Java 1.3.1, QuickTime, and QuickTime for Java to be installed with the operating system. Windows users don't necessarily have any of these. Start at java.sun.com to get Java, then go get QuickTime from the link above. QuickTime for Java is not part of the "Recommended" QuickTime install, so you need to do a "Custom" install and specifically select QuickTime for Java.

QT installer (scaled)
Figure 1. Custom install of QuickTime on Windows

It's important to install Java first, since the QTJ installer needs to find Java installations on your system and install its QTJava.zip and QTJava.dll files where Java can find them.

There are QuickTime for Java SDKs available for download on Apple's site, but they are not strictly required for development, as all you really need to compile is the QTJava.zip file that is placed in your $JAVA_HOME/lib/ext directory. The SDK contains the javadocs and about 50 demo applications. While both are available on the web, it's well worth your time to get the SDK to have local copies.

Source Code

Download the source code for the simple player example.

Your First QTJ App

It's not quite HelloWorld, but a basic player is pretty much the first thing to create with a media API. Here's a simple player that asks the user to locate a QuickTime-playable file on the filesystem, opens the movie in a new frame, and plays it:

package com.mac.invalidname.simpleqtplayer;

import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import java.io.File;
import quicktime.*;
import quicktime.app.*;
import quicktime.app.players.*;
import quicktime.app.display.*;
import quicktime.io.*;
import quicktime.std.*;
import quicktime.std.movies.*;

public class SimpleQTPlayer extends Frame {

  Movie movie;

  public SimpleQTPlayer (String title) {
    super (title);
    try {
      QTSession.open();
      FileDialog fd = new FileDialog (this,
                                      "Select source movie",
                                      FileDialog.LOAD);
      fd.show();
      if (fd.getFile() == null)
          return;

      // get movie from file
      File f = new File (fd.getDirectory(), fd.getFile());
      OpenMovieFile omFile =
        OpenMovieFile.asRead (new QTFile (f));
      movie = Movie.fromFile (omFile);

      // get a Drawable for Movie, put in QTCanvas
      MoviePlayer player = new MoviePlayer (movie);
      QTCanvas canvas    = new QTCanvas();
      canvas.setClient (player, true);
      add (canvas);

      // windows-like close-to-quit
      addWindowListener (new WindowAdapter() {
        public void windowClosing (WindowEvent e) {
          QTSession.close();
          System.exit(0);
        }
      });
    } catch (Exception e) {
      e.printStackTrace();
    } 
  }

  public static void main (String[] args) {
    SimpleQTPlayer frame =
      new SimpleQTPlayer ("Simple QTJ Player");
    frame.pack();
    frame.setVisible(true);
    try {
      frame.movie.start();
    } catch (Exception e) {
      e.printStackTrace();
    }
  }
}

Here's what the output looks like — in my case, a home movie running in a simple AWT Frame:

QuickTime movie playing in our simple player
Figure 2. Simple QTJ Player

Try it with different kinds of media that QuickTime supports, such as .mov and .avi videos, still images in various formats, and .mp3 and .wav audio files. (You'll get a zero-size window if you play an audio-only file.) This simple app should play most, if not all, of the supported media types.

Before we fight with the compiler — which will be a hassle the first time you do it — let's examine what the code does. The essential QuickTime stuff is in the constructor:

There are some peculiarities here: the number of imports is probably surprising for such a small application, and you might wonder if there aren't some unnecessary extra steps here. In particular, why do we hand the Movie to this MoviePlayer object? Why is it then sent to the QTPlayer, not in a constructor, but in the curiously named setClient method?

Don't worry, all will be covered in due course.

Let's look at compiling first. The QTJava.zip file that contains the QTJ classes is put in your Java extensions directory by the installer. Strangely, it only gets picked up for running applications and applets, not for compiling. When you compile, you must explicitly put the file in your CLASSPATH. On the command line, use a javac or jikes argument like -classpath /System/Library/Java/Extensions/QTJava.zip (that's the Mac OS X path; the Windows path will depend on where you installed the JDK). In an IDE, it's usually a matter of dragging and dropping QTJava.zip into your current project:


Figure 3. Dragging QTJava.zip into Project Builder project on Mac OS X

It will be pretty easy to tell if you don't have the CLASSPATH correct, as all of those imports will fail, looking something like this:

SimpleQTJPlayer.java:5: package quicktime does not exist
SimpleQTJPlayer.java:6: package quicktime.app does not exist
SimpleQTJPlayer.java:7: package quicktime.app.players does not exist
SimpleQTJPlayer.java:8: package quicktime.app.display does not exist

Obviously, this is enough of a hassle that if you are not using an IDE, you may want to set up a Makefile or an Ant build.xml file to save you from ongoing CLASSPATH misery. The example code for these articles will continue to provide Ant files to build the code, and you're welcome to use them for your own QTJ development.

There's one additional compile-time hassle: in making huge changes to how Java works in Mac OS X, Apple's 1.4.1 implementation is not currently compatible with QuickTime for Java. The problem comes from Apple's move to using Cocoa for their AWT/Swing implementation in 1.4.1, instead of Carbon, which they used for 1.3.1 and QTJ. I wrote about this in an O'Reilly weblog last month, and the problem has not been resolved as of this writing.

The workaround on Mac OS X is to use Java 1.3.1 explicitly, which will reportedly still be in the next major release of Mac OS X. In fact, since 1.4.1 is currently an optional download, it's safer to code to 1.3.1 anyway. Unfortunately, 1.4.1 becomes the default when installed, so it's a hassle. When working from the command line, you can specifically point to the 1.3.1 javac and java by using the path:

/System/Library/Frameworks/JavaVM.Framework/Versions/1.3.1/Commands

I've included this path in the sample code's my.ant.properties.mac file, which you need to rename to my.ant.properties to get picked up by ant. If you're using an IDE, there's probably a way to set your compiler and interpreter version to 1.3 in the GUI — in Project Builder, it's under "Java Compiler Settings" in the Target Settings:

Setting Project Builder to use java 1.3.1 for QTJ on Mac
Figure 4. Setting Project Builder to use Java 1.3.1 for QTJ on Mac

By the way, people have been asking me if I think the 1.4.1 problem means that QTJ is doomed and if they should go looking for another media API. For what it's worth, I prefer to just wait and see what happens — Apple hasn't officially dropped or deprecated QuickTime for Java. In the past, they've been pretty clear about when a technology is toast. Furthermore, Amazon still has a listing for a new edition of a QTJ book, written by Apple insiders, due in September. The situation is unclear — maybe we'll get more information at WWDC. In the meantime, while I'd prefer not having to work around the 1.4.1 situation, it's not time to panic. Not yet, anyway.

Movie and Other Essential Classes

In our example, it was simple to create a Movie object from a file reference, by way of the OpenMovieFile object. Appropriately enough, Movie is the heart and soul of the QuickTime for Java API, offering method calls for most common tasks:

It's important to think of a Movie not just in terms of what you can call on it, but also what it represents. In QuickTime, a Movie represents an organization of different kinds of time-based data, but it does not represent the data itself. In the QuickTime way of thinking, Movies have Tracks that refer to Media, which represent the low-level data like media samples (for example, audio samples or video frames).

This division allows you to go as deeply into the details as you need. Suppose you have a Movie that's made up of successive video tracks of different sizes. (You'll be able to create such a thing by the end of part 2.) If you want to know the size it will take up on screen, perhaps to create a rectangle big enough to accommodate all of the tracks, then you could call Movie.getBox(). But if you need to know the size of just one of those tracks, you could iterate over the Track objects, find the one you want, and call its getSize().

Another key concept of Movies is that they have a time scale, an integer that represents the time-keeping system within the movie. This value defaults to 600, meaning the movie has 600 time units per second. Values returned by methods like getDuration() and getTime() use this time scale, so a 30-second movie with a time scale of 600 returns 18000 for getDuration(). More interesting, though, is the fact that the various Media inside this Movie can and will have totally different time scales — uncompressed CD-quality audio might have a time scale of 44100, since it's 44.1 kHz — but the Movie isolates you from such details, so you don't have to worry about it, unless you want to iterate over the Media objects and investigate them specifically.

The Movie object wraps a structure and many functions from the native QuickTime API. Other classes in QTJ are unique to the Java version. The QTCanvas is an AWT component that offers a bridge from the QuickTime world to the Java display. To use a QTCanvas, call the setClient() method to hook it up to a class implementing the Drawable interface. Drawable is another QTJ-only creation, which indicates the ability of certain QTJ classes to provide pixels and sizing information to a QTCanvas. MoviePlayer is used for the simplest playback needs — other special-purpose Drawables exist for playing movies with a controller (QTPlayer), showing output from a capture device (SGDrawer), rendering effects (QTEffectPresenter), and for stream-broadcasting (PresentationDrawer).

Notice that I described the QTCanvas as an AWT component. SimpleQTPlayer is written entirely in AWT, not Swing. That's because the QTCanvas is a native component, allowing QuickTime to use hardware-accelerated rendering of your movie. The disadvantage, of course, is the difficulty of mixing AWT and Swing components. Because heavyweight AWT components will appear on top of any Swing components, a QTCanvas will appear above Swing JMenus, blasting through JTabbedPanes, and generally making a nuisance of itself, if you're not careful.

If you must use Swing for a QTJ app, there are two options. First, you can use the JQTCanvas, a lightweight Swing component introduced in QuickTime 6 that behaves like other JComponents in honoring Swing z-axis ordering. Unfortunately, its performance is generally poor; the movie must be re-imaged in software to get it into Swing's graphic space so that it can be painted. The second alternative is to carefully design your Swing layout to accomodate the heavyweight QTCanvas, not using overlapping components. The most common problem people experience with this is JMenus disappearing under the QTCanvas. You can get around this by calling setLightweightPopupEnabled (false) on your JMenus.

Coming soon ...

Now we've got a basic player, but of course, that's what we got from the QuickTime plug-in when we started. In Part 2, we'll delve into what makes QuickTime unique by getting into the editing API, allowing us to write a basic video editor with just a handful of code.

Chris Adamson is an author, editor, and developer specializing in iPhone and Mac.


Return to ONJava.com.

Copyright © 2009 O'Reilly Media, Inc.