Making Media from Scratch, Part 1
Pages: 1, 2
CLAUDE I'd like to point out that this tape has not been tampered with or edited in any way. It even has a timecode on it, and those are very hard to fake. JUDGE For the benefit of the court, would you please explain "timecode"? CLAUDE Just because I don't know what it is ... doesn't mean I'm lying.from the movie Strange Brew
Download the source code for the
Actually, Claude, you are lying, and timecodes -- which are just a system for encoding the current time in a movie -- are very easy to fake. In fact, the next example will add a timecode to any QuickTime movie.
To do this, we'll add a text track with timecode-like
to the existing tracks in a movie:
Open a movie. Note that this has to be a real QuickTime movie, not just some other format that QuickTime can open, such as AVI or MPEG-4.
Add a text track for the timecode text, set to use the bottom center of the movie's display.
Write text samples every 1/30th of a second, in a typical timecode format (hh:mm:ss:ff).
Flatten the movie out to a new disk file.
We start off by getting a movie from a file rather than creating a new one. The movie already has some sizing information from its existing video track, which will help us later.
This time, I've used a time scale of
30 for the text media,
which will correspond with the idea of having timestamps every 1/30th of a
second. That means every sample will have a duration of 1. Of course, we
could have accomplished the same thing by using a time scale of 60 and samples
of duration 2, or a time scale of 600 and 20-unit samples, and so on.
What's interesting is what we don't have to do, namely care what the time scale or the frame rate of our video and audio is. Just as you can have audio and video at more or less arbitrary frame rates, freely changeable independent of one another, we can have 30 text samples a second regardless of the video's frame rate. Granted, this can't be truly accurate if the video's frame rate isn't 30 fps or something reasonably divisible, but that's not the point. The key is that for any given time in the movie, there is one appropriate sample in each track that QuickTime will retrieve for us, whether that's one of thousands of audio samples that fly by every second, one frame of video, or one of our text samples.
On the other hand, we do have to worry about where our text will be placed over the video. When the (0,0)-based coordinates of the text frames are mapped into the movie's display space, we get a timecode at (0,0), which is not what we want, as shown in Figure 3.
Figure 3. The added timecode
To place the caption box in a specific place relative to the other tracks in
the movie, we can use a transformation matrix. In QuickTime, this is
a 3x3 mathematical construct that maps points from one space into another. In
our case, we need to map from a rectangle whose upper left corner is at
(0,0) to a rectangle that is centered along the bottom of the
movie's space. We do this by calling
setMatrix() on our text
track, with a
Matrix object that describes the spatial
transformation we want QuickTime to perform.
The formula for matrix transformations is shown in Figure 4. Don't run away. It's not that scary, at least not in practice.
Figure 4. The formula for matrix transformations
The formula means that, given a point (x,y), we get the new coordinates (x',y') by applying matrix multiplication. The transformation can be expressed more simply as a pair of formulas:
x' = ax + cy + tx y' = bx + dy + ty
This buys us the ability to specify operations that move, rotate, and scale your source, all with one object. A full discussion of the possibilities is available on Apple's developer site.
For our purposes, we only need to specify a move to a pair of coordinates we
boxTop, which are then used
to create a
QDRect object called
toBox. We can then
Matrix that represents the moving of pixels from the
textBox, with an upper left corner of (0,0), to
toBox, with upper left corner of (
boxTop). Setting this as the text track's matrix causes QuickTime
to use the matrix when drawing the text frames at playback time:
Matrix transformMatrix = new Matrix(); QDRect toBox = new QDRect (boxLeft, boxTop, TEXT_TRACK_WIDTH, TEXT_TRACK_HEIGHT); transformMatrix.map (textBox, toBox); textTrack.setMatrix (transformMatrix);
If you read the docs, you'll notice that the
tx and ty values
are the only ones used for moving pixels; i.e., for translating between
coordinate spaces. So we could replace the
map() call with:
transformMatrix.setTx (boxLeft); transformMatrix.setTy (boxTop);
Either way, this puts the text box in its proper location relative to the rest of the movie, as seen in Figure 5.
Figure 5. A better timestamp location
Matrix class provides a several methods that allow you
to define matrices that can perform scaling and rotation operations, all
without you having to do your own trigonometry. For example, adding this
rather silly call rotates our timecode counter-clockwise by 45 degrees,
centered on the top left corner:
transformMatrix.rotate (315, boxLeft, boxTop);
The result looks amusing as a screenshot, but is more impressive (or just plain goofy) when played as an accurate, running timecode for the movie, as shown in Figure 6.
Figure 6. A rotated timestamp
One More Neat Trick
Overall, the code for this example is fairly similar to that of the first one. Again
we create a text
Track and accompanying
which we populate with samples. The
addTextSample has a few
differences to superimpose the text onto the video:
fontSizeis 14, for better readability.
QDConstants.bold, for better readability.
StdQTConstands.dfKeyedTextis given as the
This use of
dfKeyedText produces a chromakey effect, replacing the background color (
QDColor.black, in our case) with the pixels from the video underneath. So the black box surrounding the text becomes invisible, and we just see the text on top of the video.
As before, the resulting movie is
flatten()ed out to a file,
this time called timecoded.mov, which you can open in QuickTime
Having done this simple little timecode with a text track, it should be
noted that QuickTime offers a real "timecode track" as one of the
many track types it supports. It is much more involved than is necessary for this tutorial, but if you have professional needs, check out the
TimeCodeMedia classes in QTJ.
Now that we've done some simple text tracks, the next step is to get into the good stuff: writing out video tracks from scratch. In our next article, we'll do just that, borrowing an image-to-movie effect from our favorite Civil War documentarian.
Chris Adamson is an author, editor, and developer specializing in iPhone and Mac.
Return to ONJava.com.