ONLamp.com
oreilly.comSafari Books Online.Conferences.

advertisement


make: The Evolution and Alternatives

by Andy Oram, Robert Mecklenburg
11/18/2004

Editor's note: The following article is adapted from the foreword to the third edition of Managing Projects with GNU make.

The make utility is an enticing servant, always there and always accommodating. Like the indispensable sidekicks found in many novels and movies, make starts out as the underappreciated supplicant to whom you throw a few odd jobs, and then gradually takes over the entire enterprise.

I had reached the terminal stage of putting make at the center of every project I touched when Steve Talbott, my supervisor and the author of the original O'Reilly classic Managing Projects with make, noticed my obsession and asked me to write the second edition. It proved to be a key growth experience for me (as well as a pretty wild ride) and my entry into the wonderful world of O'Reilly, but we didn't really think about how long the result would stay on the market. Thirteen years for one edition?

Enthralled in the memories of those days long ago when I was a professional technical writer, I'll indulge myself with a bulleted list to summarize the evolution of make since the second edition of Managing Projects with make hit the stands:

  • The GNU version of make, already the choice of most serious coders when the second edition of the book came out, overran the industry and turned into the de facto standard.

  • The rise of GNU/Linux made the GNU compiler tool chain even more common, and that includes the GNU version of make. As just one example, the Linux kernel itself relies heavily on extensions provided by GNU make, as documented in Chapter 11 of the third edition of the book.

  • The adoption of a variant of BSD (Darwin) as the core of Mac OS X continues the trend toward the dominance of the GNU tool chain and GNU make.

  • More and more tricks are being discovered for using make in a robust, error-free, portable, and flexible way. Standard solutions to common problems on large projects have grown up in the programming community. It's time to move many of these solutions from the realm of folklore to the realm of documented practices, as the third edition of this book does.

  • In particular, new practices are required to adapt make to the C++ and Java languages, which did not exist when make was invented. To illustrate the shifting sands of time, the original make contained special features to support two variants of FORTRAN--of which vestiges remain!--and rather ineffective integration with SCCS.

  • Against all odds, make has remained a critical tool for nearly all computer development projects. None of make's many (and insightful) critics would have predicted this thirteen years ago. Over these years, replacements sprang up repeatedly, as if dragon's teeth had been sown. Each new tool was supposed to bypass the limitations in make's design, and most were indeed ingenious and admirable. Yet the simplicity of make has kept it supreme.

Related Reading

Managing Projects with GNU Make
By Robert Mecklenburg

As I watched these trends, it had been in the back of my mind for about a decade to write a new edition of Managing Projects with make. But I sensed that someone with a broader range of professional experience than mine was required. Finally, Robert Mecklenburg came along and wowed us all at O'Reilly with his expertise. I was happy to let him take over the book and to retire to the role of kibitzer, which earns me a mention on the copyright page of the book. (Incidentally, we put the book under the GNU Free Documentation License to mirror the GPL status of GNU make.)

Robert is too modest to tout his Ph.D., but the depth and precision of thinking he must have applied to that endeavor come through clearly in this book. Perhaps more important to the book is his focus on practicality. He's committed to making make work for you, and this commitment ranges from being alert about efficiency to being clever about making even typographical errors in makefiles self-documenting.

This is a great moment: the creation of a new edition of one of O'Reilly's earliest and most enduring books. Get yourself a copy and read about how an unassuming little tool at the background of almost every project embodies powers you never imagined. Don't settle for creaky and unsatisfying makefiles--expand your potential today.

Editor's note: The following is an adaptation of a portion of Managing Projects with GNU make, 3rd Edition written by author Robert Mecklenburg.

Alternatives to make

I've been programming in Java for about five years and I enjoy working in the language. There is, however, an annoying tendency in Java to reinvent the wheel. Of course, Java must have its own implementation of file access, socket communication, and database access. These are all done in the name of portability and seamless interaction with the rest of the language, and rightly so. But there are also Java implementations of the CVS protocol and LR(1) parser generators, and a Java version of make called Ant.

I recently finished writing the third edition of Managing Projects with GNU Make and made sure to include a chapter on Java. make is one of the major choices available for coordinating Java program compilations, along with various IDEs and Ant. Ant was created was because applying make to Java can be tricky. That's why Managing Projects with GNU Make contains a chapter on make and Java.

Let's see how Ant, IDEs, and make compare for managing the build process.

Ant

The Java community is very active, producing new tools and APIs at an impressive rate. One of these new tools is Ant, a build tool intended to replace make in the Java development process. Like make, Ant uses a description file to indicate the targets and prerequisites of a project. Unlike make, Ant is written in Java and Ant build files are written in XML.

To give you a feel for the XML build file, here is an excerpt from the Ant build file:

<target name="build"
        depends="prepare, check_for_optional_packages"
        description="--> compiles the source code">
  <mkdir dir="${build.dir}"/>
  <mkdir dir="${build.classes}"/>
  <mkdir dir="${build.lib}"/>

  <javac srcdir="${java.dir}"
         destdir="${build.classes}"
         debug="${debug}"
         deprecation="${deprecation}"
         target="${javac.target}"
         optimize="${optimize}" >
    <classpath refid="classpath"/>
  </javac>
  
  <copy todir="${build.classes}">
    <fileset dir="${java.dir}">
      <include name="**/*.properties"/>
      <include name="**/*.dtd"/>
    </fileset>
  </copy>
 </target>

As you can see, a target is introduced with an XML <target> tag. Each target has a name and dependency list specified with <name> and <depends> attributes, respectively. Actions are performed by Ant tasks. A task is written in Java and bound to an XML tag. For instance, the task of creating a directory is specified with the <mkdir> tag and triggers the execution of the Java method Mkdir.execute, which eventually calls File.mkdir. As far as possible, all tasks are implemented using the Java API.

An equivalent build file using make syntax would be:

# compiles the source code
build: $(all_javas) prepare check_for_optional_packages
        $(MKDIR) -p $(build.dir) $(build.classes) $(build.lib)
        $(JAVAC) -sourcepath $(java.dir) 			\
                 -d $(build.classes)     			\
                  $(debug)               			\
                  $(deprecation)         			\
                  -target $(javac.target)			\
                  $(optimize)            			\
                  -classpath $(classpath)                       \
                  @$<
        
        $(FIND) . \( -name '*.properties' -o -name '*.dtd' \) | \
        $(TAR) -c -f - -T - | $(TAR) -C $(build.classes) -x -f -

This snippet of make uses techniques that this book hasn't discussed yet. Suffice to say that the prerequisite <filename>all.javas</filename> contains a list of all java files to be compiled. The Ant tasks <mkdir>, <javac>, and <copy> also perform dependency checking. That is, if the directory already exists, mkdir is not executed. Likewise, if the Java class files are newer than the source files, the source files are not compiled. Nevertheless, the make command script performs essentially the same functions. Ant includes a generic task, called <exec>, to run a local program.

Ant is a clever and fresh approach to build tools; however, it presents some issues worth considering:

  • Although Ant has found wide acceptance in the Java community, it is still relatively unknown elsewhere. Also, it seems doubtful that its popularity will spread much beyond Java (for the reasons listed here). make, on the other hand, has consistently been applied to a broad range of fields, including software development, document processing and typesetting, and web site and workstation maintenance, to name a few. Understanding make is important for anyone that needs to work on a variety of software systems.

  • The choice of XML as the description language is appropriate for a Java-based tool. But XML is not particularly pleasant to write or to read (for many). Good XML editors can be difficult to find and often do not integrate well with existing tools (either my integrated development environment includes a good XML editor or I must leave my IDE and find a separate tool). As you can see from the previous example, XML and the Ant dialect in particular are verbose compared with make and shell syntax. And the XML is filled with its own idiosyncrasies.

  • When writing Ant build files you must contend with another layer of indirection. The Ant <mkdir> task does not invoke the underlying mkdir program for your system. Instead, it executes the Java mkdir() method of the java.io.File class. This may or may not do what you expect. Essentially, any knowledge a programmer brings to Ant about the behavior of common tools is suspect and must be checked against the Ant documentation, Java documentation, or the Ant source. In addition, to invoke the Java compiler, for instance, I may be forced to navigate through a dozen or more unfamiliar XML attributes, such as <srcdir>, <debug>, etc., that are not documented in the compiler manual. In contrast, the make script is completely transparent; that is, I can typically type the commands directly into a shell to see how they behave.

  • Although Ant is certainly portable, so is make. Writing portable makefiles, like writing portable Ant files, requires experience and knowledge. Programmers have been writing portable makefiles for two decades. Furthermore, the Ant documentation notes that there are portability issues with symbolic links on Unix and long file names on Windows, that Mac OS X is the only supported Apple operating system, and that support for other platforms is not guaranteed. Also, basic operations like setting the execution bit on a file cannot be performed from the Java API; an external program must be used. Portability is never easy or complete.

  • The Ant tool does not explain precisely what it is doing. Since Ant tasks are not generally implemented by executing shell commands, the Ant tool has a difficult time displaying its actions. Typically, the display consists of natural language prose from print statements added by the task author. These print statements cannot be executed by a user from a shell. In contrast, the lines echoed by make are usually command lines that a user can copy and paste into a shell for re-execution. This means the Ant build is less useful to developers trying to understand the build process and tools. Also, it is not possible for a developer to reuse parts of a task, impromptu, at the keyboard.

  • Last and most importantly, Ant shifts the build paradigm from a scripted to a non-scripted programming language. Ant tasks are written in Java. If a task does not exist or does not do what you want it to do, you must either write your own task in Java or use the <exec> task. (Of course, if you use the <exec> task often, you would do far better to simply use make with its macros, functions, and more compact syntax.)

Scripting languages, on the other hand, were invented and flourish precisely to address this type of issue. make has existed for nearly 30 years and can be used in the most complex situations without extending its implementation. Of course, there have been a handful of extensions in those 30 years. Many of them conceived and implemented in GNU make.

Ant is a marvelous tool. It has found wide acceptance in the Java community. However, before embarking on a new project, consider carefully whether Ant is appropriate for your development environment. This chapter will hopefully prove to you that make can powerfully meet your Java build needs.

IDEs

Many Java developers use Integrated Development Environments (IDEs) that bundle an editor, compiler, debugger, and code browser in a single (typically) graphical environment. Examples include the open source Eclipse (www.eclipse.org) and Emacs JDEE (jdee.sunsite.dk), and, from commercial vendors, Sun Java Studio (wwws.sun.com/software/sundev/jde) and JBuilder (www.borland.com/jbuilder). These environments typically have the notion of a project build process that compiles the necessary files and enables the application execution.

If the IDEs support all of this, why should we consider using make? The most obvious reason is portability. If there is ever a need to build the project on another platform, the build may fail when ported to the new target. Although Java itself is portable across platforms, the support tools are often not. For instance, if the configuration files for your project include Unix- or Windows-style paths, these may generate errors when the build is run on the other operating system. A second reason to use make is to support unattended builds. Some IDEs support batch building and some do not. The quality of support for this feature also varies. Finally, the build support included is often limited. If you hope to implement customized release directory structures, integrate help files from external applications, support automated testing, and handle branching and parallel lines of development, you may find the integrated build support inadequate.

In my own experience, I have found the IDEs to be fine for small-scale or localized development, but production builds require the more comprehensive support that make can provide. I typically use an IDE to write and debug code, and write a makefile for production builds and releases. During development, I use the IDE to compile the project to a state suitable for debugging. But if I change many files or modify files that are input to code generators, then I run the makefile. The IDEs I've used do not have good support for external source code generation tools. Usually, the result of an IDE build is not suitable for release to internal or external customers. For that task I use make.

Conclusion

IDEs provide workable build environments for many small projects, but if your project grows beyond a couple of people or a single .jar file, you will probably want to consider more powerful build tools. At that point, you'll most likely choose between Ant and make. If you want to work with a scripting language and have a build that is transparent to your developers (i.e., tells you precisely what it is doing at each step), choose make. If you like XML or work in a development team unfamiliar with Unix software tools, consider using Ant.

I have many colleagues that have used both make and Ant. In truth, those that have spent more time with Windows and IDEs than Unix and bash prefer Ant. I'll stick with make, but you knew that.

Andy Oram is an editor for O'Reilly Media, specializing in Linux and free software books, and a member of Computer Professionals for Social Responsibility. His web site is www.praxagora.com/andyo.

Robert Mecklenburg began using Unix as a student in 1977 and has been programming professionally for 23 years. He has worked in many fields ranging from mechanical CAD to bioinformatics, and he brings his extensive experience in C++, Java, and Lisp to bear on the problems of project management with make.


View catalog information for Managing Projects with GNU make, 3rd Edition

Return to ONLamp.com.



Sponsored by: