ONLamp.com    
 Published on ONLamp.com (http://www.onlamp.com/)
 See this if you're having trouble printing code examples


Design by Wiki

by Jason R. Briggs
01/20/2005

Enterprise Architecture on a Budget

Until recently, I've had a certain amount of difficulty latching onto a documentation strategy. In the dim, dark, distant past, I've worked on projects that destroyed small portions of forest producing thick binders full of technical documentation. I have also worked on projects that grew from basic prototypes, relying on verbal explanation by the team lead for the detail, with very little in the way of format documentation--not my favorite approach. Most common seems to be the myriad array of Word documents, stored in the version control system of choice; linked, referenced, cross-referenced by arbitrary, and sometimes arcane, naming schemes.

Now, I was never all that enamored of Word back when I was using Windows. With no offense intended toward the developers of Open Office, KOffice, et al, spending half my day trapped inside a word processor was never high on my list of priorities. To be honest, none of these approaches have particularly appealed.

Eventually I decided HTML was the obvious way to go and began writing specifications with carefully designed (standards compliant!) style sheets; with hrefs linking them together rather than oddball references in a Word doc. When a colleague installed MediaWiki for various other documentation requirements, it was a head-slapping, Homer Simpson-style Doh! moment.

Still, using a wiki for high-level architectural docs and then drilling down to a lower-level system design seems, to me at least, to be somewhat of a dark horse in the software development world. There is some evidence to show that projects, both in the open source and commercial worlds, use wikis to aid their development efforts, but either I'm exposing a severe lack of l337 g00gl3 sk1lz or there is very little in the way of formal literature regarding the use of wikis for enterprise architecture and/or technical design.

In this article I hope to demonstrate one approach to documenting an architectural design using a wiki and, with any luck, stimulate further discussion.

Why a Wiki Makes Sense for Collaborative Development

First of all, and in case you've had your head buried in the sand since '95, what is a wiki? Wikipedia is an excellent example, and it provides a useful definition:

A Wiki or wiki (pronounced "wicky" or "weekee") is a website or other hypertext document collection that allows a user to add content, as on an internet forum, but also allows that content to be edited by any other user.

From a collaborative point of view, a wiki can provide a central point of contact for a team, and indeed for all major stakeholders in a project, no matter how distributed--an end to the "check out doc, modify doc, check in doc, notify interested parties" cycle of document editing. It allows the tracking and monitoring of changes and control over edit access, and there is no need to enforce a standardized template or document structure because the wiki software itself very much dictates layout and structure.

The open nature of a wiki encourages discussion. For instance, I've worked on a project where the designer unnecessarily increased the complexity of a system, through a flawed of understanding of a fundamental technology, and then mysteriously forgot that people had brought this to his attention at the beginning. It's obvious that the ease with which even the lowliest developer can add comments to project documentation would have been beneficial--even if only from an auditing perspective.

In my experience at least, there's an additional psychological benefit to using a wiki. Web browsing has become such an ingrained experience, opening the browser is a throwaway action. Indeed, the use of a browser has become so deeply rooted in the psyche that the technology itself barely registers on the user experience. Editing a wiki page becomes a straightforward exercise, in a way that opening a word processor and switching your thought patterns to work within that environment most likely never can be.

A couple of years ago, I moved back to New Zealand from the United Kingdom, which highlighted for me once again the differences of scale between the two countries and between the much larger organizations there and the generally smaller companies here. Large companies have larger technical budgets, and established teams typically have already provisioned a selection of software for the various stages of system development. They also tend to be inflexible, at times, when it comes to the use of alternatives--even when requirements fall outside the capabilities of the existing software suites.

Smaller companies often have commensurately smaller budgets, and as a consequence they can be more flexible when it comes to software provisioning--a sweeping generalization, of course, but one that has generally held true in my experience. However, the general availability of high-quality wiki software, both open source and commercial, can meet the requirements of the most severe budget. Not only that, but there is also a huge freedom of choice, given the availability of wiki software for all the major, popular languages.

Related Reading

Learning Python
By Mark Lutz, David Ascher

An Enterprise Architecture Methodology

The Zachman Framework is a blueprint or structure for describing complex enterprise systems. Its characteristics include the production of a fairly comprehensive set of documentation, so I have become particularly interested in seeing how to use a wiki to develop tech docs, applying Zachman as its basis. Without working through an entire example, I hope to expose a few fundamental requirements for the successful use of a wiki. It seems to me that these fundamentals will therefore still be useful when applied to design methodologies, such as RUP and the more lightweight processes.

A starting point: wiki choice and the front page

I chose MoinMoin as the wiki with which to host my docs, mainly because I like working with Python, and it appears fairly easy to extend MoinMoin with plugins and macros. I haven't found the software as easy to use nor as elegant as perhaps MediaWiki, but it is certainly adequate for my purposes.

The Zachman Framework uses a 2-D matrix, columns being defined as the abstractions of Data, Function, Network, People, Time, and Motivation; and rows as the perspectives of Planner, Owner, Designer, Builder, Subcontractor, and the final working system. It makes sense that the front page should be a table of the components the matrix describes, with each component linking to subsequent pages--drilling down into the detail.

MoinMoin's editing syntax is straightforward, but defining a table with that many cells looks rather an exercise in the arcane. Case in point: this wiki markup for the front page. That said, the front page is a onetime exercise, and many of the subpages I'll link to will be considerably simpler in scope.

Figure 1 shows that the final result is a little more legible than you might expect upon seeing the markup. This gives me a structure for documenting my system(s) and gives the major stakeholders a top-level view into the doc hierarchy. Also, the nature of a wiki means that I don't have to create those subsequent pages immediately.

The new wiki front page
Figure 1. The new wiki front page

Drill Down

The first row of the framework defines various lists of information, which is extremely easy to implement in any wiki. For example, the page for operating locations (column 3, row 1 of the framework) defines a heading and then a bulleted list of links to each location:

== A list of operating locations ==

* [http://www.somebusiness.com/nz Auckland, NZ]

* [http://www.somebusiness.com/ca Toronto, Canada]

* [http://www.somebusiness.co.uk London, United Kingdom]

* [http://www.somebusiness.com/sg Singapore]

SubPageWithMoreDetail

The other cells of the framework are more complicated and detailed; many require diagrams to describe their specifics. For example, column 1, row 2 of the framework is the semantic model. From zifa03:

This is a model of the actual Enterprise things (objects, assets)... It would typically be represented as an "E/R"-type model ...

The ideal approach in these cases would be the ability to edit a diagram within the wiki, in a similar manner to editing normal wiki text, something you can do in many word processors. This would allow the wiki to track the iterations of any diagram through the life of the project. While these capabilities (as far as I'm aware) aren't available out of the box in most if not all wikis, several possibilities can at least approximate this idea:

Embedding SVG Diagrams, Step by Step

an SVG image embedded in a wiki page
Figure 2. The semantic model page, showing an extremely simple E/R diagram

One issue to note is that MoinMoin does not support the overwriting of attached files, so the macro must choose a new name each time. This makes the process a bit more clunky.

Some thoughts moving forward

Dia also supports export-to-SVG from the command line; for example, dia -e mydiagram.svg -t svg mydiagram.dia. Assuming Dia was installed on the server--which requires either Dia v0.94 or better or Xvfb if you don't have an X server--the macro could take a .dia file, convert it to SVG format, and then write the embed HTML tag for that file into the page.

Mozilla has an SVG project in progress. If or when it makes it into the main browser, the wiki can include the XML for the SVG inline, not as an attached file. It would seem easier then to automate the upload process--at the very least, not requiring a new filename with every diagram modification.

For those who can't wait that long, perhaps the SVGViewer at Apache's Batik project would be acceptable instead. Assuming that the SVGViewer could be converted to an applet, it could potentially read SVG data directly from the HTML page (using a parameter passed to the applet) and render it in a similar fashion to Adobe's viewer. Considering that Batik is open source, it perhaps offers considerably more flexibility than the Adobe product. Indeed, in about five minutes I put together a very quick test applet using Batik's JSVGCanvas to render an SVG document embedded as a parameter on a page.

Export

Another facility that any documentation repository inevitably requires is exporting to alternate formats. The tree-killing set, unable to read documentation without the mulching and bleaching of trees, no doubt feel the need for something substantial to stack on their desks. There is a LaTeX formatter for MoinMoin, which I haven't yet tested, that, coupled with a LaTeX-to-PDF converter, could probably work for this purpose.

However, considering the relatively simple to parse layout of the HTML created by MoinMoin (edited content always appears within in a <div id="content" ...> tag, for example), perhaps a custom screen scraper along with one of W3's HTML converters will provide a more flexible option.

This example pipes the output from a basic Python Wiki screen scraper to a modified sed script obtained from the W3 list:

# modified \\par to include a noindent
s?<[Pp]>?\\par \\noindent?g

# added img tag processing
s?<[Ii][Mm][Gg].*[Ss][Rr][Cc]=["|']\([^"']*\)["|'][^>]*>?\\includegraphics{\1}?g

The full command to produce the LaTeX output is:

./scrape.py myserver 80 testwiki/MyPage | sed -f html2latex.sed > output.tex

Figure 3 shows the final result of importing output.tex into LyX and then viewing it as a PDF.

viewing LaTeX output as a PDF in LyX
Figure 3. Viewing LaTeX output as a PDF in LyX

Conclusion

A full implementation of the Zachman Framework will require various other features. For example, row 5 of the framework specifies Detailed Representations: Data Definition, Network Architecture, and so on. In the case of Data Definition (column 1) and Rules Specification (column 6) in particular, it might be preferable to serve those documents from within the source control system for the project. Undoubtedly they will require stricter control than that provided by a wiki, perhaps needing tags/branches for releases and so on.

However, preferences and/or user requirements might dictate the inclusion of DDL or rules with the other documentation, therefore from within the wiki. In that case, you'll probably need another macro to embed DDL within the wiki page along with identifiers to make it easier to scrape (to pass to some kind of DDL-to-SQL translator). The embed.py macro should have demonstrated just how simple this is to accomplish.

While the use of a wiki is by no means a perfect solution to these kinds of tech doc problems, I find it suits my work habits. There are plenty of opportunities to automate tasks I would just find annoying in a word processor.

Better yet, it suits my budget.

References

zifa03. The Framework for Enterprise Architecture Cell Definitions. 2003, The Zachman Institute. www.zifa.com

Wikipedia. Definition of Wiki. www.wikipedia.org

Resources

Addendum

Since writing this article I have, of course, continued to use MoinMoin as a repository for anything and everything in the way of technical docs. A little further hacking has come up with a way to automatically handle the conversion of .dia to .svg within the wiki:

  1. Find the action /python/2005/01/20/examples/AttachFile.py. On my server this is in /usr/lib/python2.3/site-packages/MoinMoin/action. Then apply the patch attachfile_backup_with_dia.diff. This is a combination of a patch to allow attachment backups and some code to call dia to convert a .dia file to an .svg. patch AttachFile.py attachfile_backup_with_dia.diff should apply the changes, but note that I'm using MoinMoin 1.2.4. Your mileage may vary if you're running another version.
  2. Copy the macro file dia.py to the macro directory. On my server this is /usr/lib/python2.3/site-packages/MoinMoin/macro.
  3. Make sure that you have enabled file attachments. Look for the line:

    allowed_actions = ['DeletePage', 'AttachFile']

    probably commented out, in moin_config.py. On my server, this is in /usr/share/moin/cgi-bin.

With any luck you'll be able to upload a .dia file and then refer to it in the wiki page:

attachment:core_db.dia

[[dia(core_db.dia,800,500)]]

The first line gives a user access to download the .dia file directly. The second line embeds an HTML object tag in the page with a link to the converted SVG file, using the specified width and height.

Jason R. Briggs is an architect/developer working in the United Kingdom.


Return to the Python DevCenter.

Copyright © 2009 O'Reilly Media, Inc.