ONLamp.com    
 Published on ONLamp.com (http://www.onlamp.com/)
 See this if you're having trouble printing code examples


Customizing X Window: An Introduction

by Frank Pohlmann
11/06/2007

X Window, X11 or "X," as it is known for short, provides the programming framework and the underlying runtime system for most Unix and Linux-based network-transparent windowing implementations. It runs on a huge number of Linux and Unix flavors, including Mac OS X and with a bit of help, on several Windows varieties. Without X11 , there is no KDE, no GNOME, and no Linux-based window manager, unless one is prepared to accept an X replacement. They do exist, and many carry a proprietary license, while X comes with a GPL-compatible license.

Many Linux and Unix desktops and window managers should actually be called X Desktops, since they use the X Window framework to provide users with a full, bitmapped system GUI. Still, most Linux and Unix users would not actually see X Window directly, except when they run X applications, like the xterm and rxvt terminals or some fairly basic games. On its own, X Window requires, but does not provide software to manage and display flexible GUI elements, e.g., windows. It provides the primitives to display them, however, including the ability to draw graphical elements and, of course, strings of characters.

X.Org Rationales

These days, X Window comes in the so-called X.Org variety: the X.Org Foundation provides the canonical X Window open source implementation. XFree86 used to be very common, but the coming of 64-bit processors and various other issues obsoleted XFree86 fairly quickly.

Most of the time, it is possible to leave standard X Window configurations alone, but there are occasions when X looms very large in the lives of system administrators and power users: if there are major input and output device issues, or when system-based applications refuse to display window decorations or fonts properly, a fairly deep knowledge of X configuration files—and not GUIfied X11 configuration tools—is of the essence.

What Needs to Be Localized?

Localization results in culturally specific visible effects. It is perhaps surprising that almost no localization resources residing on or attached to a computer are changed or used in a productive manner. X Window and of course, some window managers and desktop shells, give you this opportunity. X Window deals with visual input and output devices, whether software or hardware, including most graphics cards and monitors. X Window can deal with multiple monitors and provides the libraries to help render 2D and 3D primitives to the screen. Compositing all those elements in layout, collating fonts or even full 3D environments is also a task where X Window, and in particular its configuration files play an important role.

What Needs to Be Done

First of all, input devices, i.e., keyboards and mice need to be configured. Keyboards conform to technical as well as linguistic specifications. The number of keys, the layout of the keys and the way keys are mapped to the underlying character set as well as the available fonts are determined by X Window and font configuration files. In a localization context, mouse configuration is of course not as important as keyboard configuration, but we should not forget that the way copying and pasting is handled can effect particular text layout environments: cut-and-paste works differently for top-down and right-to-left scripts, like classical Chinese text layout and traditional Mongolian scripts.

Input methods sometimes have to be heavily modified using special input software and some of it is available with X11. It is, however, language and sometimes, script-specific what needs to be done, so we will not talk about it here.

X11, Fonts and Locales

Secondly, X Window makes available, and partly renders, all fonts used by windowing applications. Conversely, X Window does not care much about fonts available at Runlevel 2, or rather the level of the command line user: X11 uses its own fonts. Of course, if there is no window manager or desktop shell, or indeed, no X Window installation, localization and fonts is purely a matter of system locales and system fonts. X Window does care about character sets very much, though and derives its information about systems locales from the system or (g)libc locales. Please remember that X determines its own keyboard mappings and uses its own fonts.

It is always useful to check which locales are available by typing

$ locale -a

at the command line.

System Locales

X11 needs system locale information as a localization starting point: Nothing is more embarrassing than having a fully localized terminal menu bar, while the shell running inside the terminal cannot display, say, French or Russian filenames. Here is, in short, what you need to do to make your system locale information available to X11:

First of all, you need to make sure that all possible character sets that could become part of your system locale are included by X11. There are three files you need to keep updated: the locale.alias, compose.dir, and the locale.dir files. You will find all of them in the /usr/X11R6/lib/X11/locale location. Your system might prefer another path, so mileage may vary.

Checking locale.alias

If you need a complete survey of the locales that have been tested under X11, you should go to http://webcvs.freedesktop.org/xorg/xc/nls/locale.alias and find the relevant locales. You can of course create your own locales if necessary, but this would go beyond the purview of this article.

A partial list for all varieties of German alphabets and its flavors would be (including some Austrian, Swiss, Luxemburgish, and Belgian flavors):

de:                    de_DE.ISO8859-1
de_AT:                      de_AT.ISO8859-1
de_AT@euro:                 de_AT.ISO8859-15
de_AT.iso88591:             de_AT.ISO8859-1
de_AT.ISO-8859-1:           de_AT.ISO8859-1
de_AT.ISO_8859-1:           de_AT.ISO8859-1
de_AT.iso885915:            de_AT.ISO8859-15
de_AT.ISO-8859-15:          de_AT.ISO8859-15
de_AT.ISO-8859-15@euro:     de_AT.ISO8859-1
de_AT.UTF-8@euro:           de_AT.UTF-8
de_AT.utf8:                 de_AT.UTF-8
de_BE:                      de_BE.ISO8859-1
de_BE@euro:                 de_BE.ISO8859-15
de_DE.ISO_8859-15:          de_DE.ISO8859-15
de_DE.8859-15:              de_DE.ISO8859-15
de_DE.8859-15@euro:         de_DE.ISO8859-15
de_DE.ISO-8859-15@euro:     de_DE.ISO8859-15
de_LU.ISO-8859-15:          de_LU.ISO8859-15
de_LU.ISO-8859-15@euro:     de_LU.ISO8859-15
de_LU.UTF-8@euro:           de_LU.UTF-8
de_LU.utf8:                 de_LU.UTF-8
GER_DE.8859:                de_DE.ISO8859-1
GER_DE.8859.in:             de_DE.ISO8859-1

This is necessary to make it possible for you to use short names for your system locales. Multiple permutations need to be reflected; you might realize that there are multiple spellings for each locale name. It might be a good idea to be careful about the placement of hyphens and underscores within each name.

Checking compose.dir

You also need to update the compose.dir file located in the same location to reflect the character sets you are actually using: The first name indicates the database file and the second name indicates the locale name used system wide. This is just a single example; you can add several locales, and if you are implementing several keyboard layouts for separate languages, you actually have to do so.

iso8859-1/Compose:        de_CH.ISO8859-1
Checking locale.dir

Finally, you need to update the locale.dir file in the same location indicate to X Window that it should use the locale the non-windowing systems use in the first place. The first name shows the local database filename name and the second entry, again, shows the full locale name.

iso8859-1/XLC_LOCALE:            de_CH.ISO8859-1

This makes sure that you can use the character set adapted to Swiss German.

Personal Preferences

In this vein, we should also mention the issue of personal preferences: fonts and layout issues are to some extent a matter of personal taste and as such often shaped by cultural forces.

Font choice is based on technological possibilities, although little can be more annoying than having anti-aliasing enabled in your font configuration file, and your users complain about headaches because individual glyphs seem to fade into the presumably darkened terminal window background. This is, for instance, an X Window issue, not a KDE or GNOME issue, although desktop shell utilities may be of help in this matter.

Keyboard Alternatives

Further, physical keyboards can be a matter of personal preference as well, when, for instance, the Dvorak keyboard layout can help avoid carpal-tunnel syndrome, since it stresses key press combination that are substantially different from the usual QWERTY keyboards.

The Dvorak layout has been adapted to other languages and scripts, but there are of course additional issues to be resolved, since the physical layout of the keyboard has to be changed in a different fashion if non-Latin scripts are involved. Still, X Window would need to be aware of both the keyboard layout (Dvorak) and the linguistic environment (say, English and Turkish) to avoid both health problems and enable keyboard switching while using the same keyboard hardware.

The main configuration files, xorg.conf and fonts.conf, are not very well documented. Both contain basically all information to configure and localize X Window as a whole. To perform some surgery on both needs some fairly detailed research, which this article will facilitate.

X 11 Architecture and Localization

X Window basically consists of a client-server core protocol for an X server and multiple X client the X server communicates with. X servers are written such that they can accommodate extensions. Clients can of course communicate with each other without taking X server into account to a great extent, but we do not need to deal with it here. We don't need to worry about protocol details, but we do need to pay attention to the way keyboards and fonts are managed.

Keyboards are managed by the Xorg server via the X keyboard extension, also known as xkb. The X Window core protocol contains keyboard control commands, but the xkb extension substantially improves upon X Window core-protocol based control of keyboards.

The keyboard extension is, strictly speaking, a part of the X Server, while font management, since the appearance of the first Xorg X Window implementation released in 2004, is very largely a client issue: a library called xft takes care of client-side font management. This is in contrast to XFree86, where font management was very much a server side issue and font servers were a separate entity exposed quite directly to users and system administrators.

It is possible to use the window manager to add and select existing fonts to be used system-wide, but please be aware that any window manager or desktop shell provides little more than a pretty interface to the client-side X Window xft library and the fontconfig utility to enable font management. KDE and GNOME provide some additional facilities, but neither goes much beyond X Window functionality. The Pango library is vital, however, for very sophisticated font drawing and layout management. X Window does not deal with text layout management at all, so additional help from other libraries is actually necessary.

Xorg.conf Surgery for Keyboards

First, you should make sure that you actually generated an xorg.conf file. Look in /etc/X11/xorg.conf or the directory in which your Linux distro or Unix flavor saves its xorg.conf. If it doesn't exist yet, run the following commmand, but make sure you run it as root or at least use sudo to run this command:

$Xorg -configure

This command will do a good job of finding all your hardware and provide a basic xorg.conf for you.

The xorg.conf file consists of at least eight sections and at most 12 sections. All sections begin with Section "section name" and end with EndSection. You have to enter all entries between the those lines.

Every entry has to stand alone in a line, and is made up of a keyword, followed by one or more arguments. You can add either an integer in decimal, hex, or octal formats; the latter is important for geometry specifications or floating point numbers. You can also add a string within double quotations marks, something which the Section "SectionName" line already demonstrated.

These days, keyboards and mice are defined in one xorg.conf section, but let us just look at the keyboard entry first.

Here we go:

Section "InputDevice"
    Identifier      "Generic Keyboard"
      Driver        "keyboard"
      Option        "CoreKeyboard"
      Option        "XkbRules"  "xorg"
      Option        "XkbModel"  "pc102"
      Option        "XkbLayout" "us"
EndSection

This won't do anything just yet, since this entry has to be referenced in another, namely the ServerLayout section, which we will come to in a minute.

You can also add the keyboard option to an X Window startup command line.

$xorg -keyboard "Generic Keyboard"

You can activate the keyboard in this fashion, provided there is no CoreKeyBoard option in your InputDevice section. Otherwise, no matter whether you specify this option on the command line or not, the Generic Keyboard entry will activate at Xorg server startup.

You need both the Identifier and the Driver name value sequences, since otherwise your entry will by replaced by X defaults. We have used a run-of-the-mill keyboard without a very exciting physical layout, so the identifier and the driver can remain the way they are. You might want to use kbd instead of keyboard, but this wouldn't change much either.

xkb riddles

The next three entries need more circumspect treatment; let us start with the last one:

      Option    "XkbLayout" "us"

xkb represents one of the more complex X Window libraries; it functions as an intermediary between the X server and most possible keyboards. It uses a complex rules language which is useful if you want to produce your own keyboard mapping. We already know that it has been implemented as an X server extension.

The Option entry included in the InputDevice section is necessary to decide which locale- or purpose-specific keyboard layout you would like to start with. I am choosing those words advisedly, since it is possible to switch keyboards, not just their layout. The mechanisms for doing so can be set in the same entry. But be aware that you might have to add keyboard mappings; this adds considerable amount of work to X11 localization.

So, if you would like to start with a US-style layout, you need to choose the "us" character-style combination. If you would prefer French-style keyboard mapping, choose "fr". German is represented by "de" etc. You can also choose the Dvorak layout, using the "dvorak" keyword. The latter would also apply if you actually changed the physical keyboard, not just the layout.

This is an example of a xkb rule and we will hear about them a little more.

The next option will be:

      Option    "XkbModel"  "pc102"

since we need to decide whether we need an international keyboard or just varieties of English. This entry refers to the keyboard geometry. The quick rule of thumb says we should use a bigger keyboard with more keys, particularly if we want to be able to switch keyboards while typing. So let's change it to:

      Option    "XkbModel"  "pc105"

105 keys sounds a bit more roomy and it will accommodate languages with additional letters in the alphabet as well as Umlauts and compound characters if required.

The last entry we have to talk about isn't that exciting, since we have only two options to worry about:

      Option    "XkbRules"  "xorg"

since we are using the Xorg X server, we don't need to worry about the xfree86 option. Xfree86 is going rapidly the way of outdated technology.

In other words, we have changed only one entry so far, namely XkbModel.

Layout Switching

More surgery is needed to enable the same keyboard to be the carrier of several keyboard layouts. Some keys will be mapped to glyphs different from standard QWERTY layouts, and on occasions key press combinations will result in characters composed of the character glyph with a diacritic sign on top, or other glyph combinations.

But first we have to make sure that keys that are not part of the original keyboard layout (e.g. "us") to become part of key press combinations. That is achieved by making sure that all keys are functional, even though some keys might only result in visible glyph combinations after they and another key have been pressed. So, ‡, ‰, and š will all be possible, once the following option has been enabled:

Option "XkbVariant"  "nodeadkeys'

If you would like to create your own keyboard layout, you can of course do that, too, but this requires in-depth knowledge of xkb functionality and rules syntax. The files live in /etc/X11/xkb/symbols/ or its subdirectories. This is a rather complex operation, which has been documented elsewhere.

Now we can finally switch between layouts by changing one and adding another option:

Option "XkbLayout"    "us,fr,de"
Option "XkbOptions"   "grp:alt_shift_toggle'

The commas between the keyboard layout names are essential, and there are no spaces between the layout types. We have enabled a standard US, a German, and a French keyboard layout. We are able to switch between keyboard layout by pressing the ALT and the SHIFT key together.

Finally, the complete InputDevice Section:

Section "InputDevice"
    Identifier    "Generic Keyboard"
      Driver    "keyboard"
      Option    "CoreKeyboard"
      Option    "XkbRules"        "xorg"
      Option    "XkbModel"        "pc105"
    Option    "XkbVariant"        "nodeadkeys'
      Option    "XkbLayout"         "us,fr,de"
    Option    "XkbOptions"       "grp:alt_shift_toggle'
EndSection

Font Basics

All text processing systems need to provide access to the software machinery that transforms keystrokes into glyphs. Multiple keystrokes, as we have seen, can result in a complex or rare glyph. This is often important for languages that do not rely on the Roman or Cyrillic alphabet, particularly, Arabic, and Indic scripts and their derivatives, as well as Chinese, Japanese, and Korean scripts and their historical predecessors.

Fonts come in a particular design, also known as a typeface. A particular design is prevalent within a group of fonts, making up a font family.

Digital as opposed to physical fonts have to provide the data to generate glyphs that adhere to a given typeface.

If the fonts do not have a fixed size, a scaling algorithm can be added to the font description, enabling the creation of glyphs of varying sizes. So-called Type 1, TrueType, and OpenType fonts are all scalable and available for X Window. In some cases and in particular with X Window, the data actually describe bitmaps and thus many fonts usable for X Window are collections of bitmapped data.

Ordering Font Names

Although fonts often seem to be designated by names that reach far back into human history, they are ordered according to very clear principles. Fonts have names, families, sometimes sizes, weights, and many other properties.

Digital glyphs are also indexed according to font file formats: TrueType fonts order glyphs by the integers given to them by a known encoding scheme, e.g., Unicode. Unicode itself maps character set members to a unique sequence number, and is used at times as a glyph encoding scheme.

Type 1 fonts give glyphs a glyph name, based on the font family, font name, and glyph position. Type 1 font description are usually not enough to create a font on the screen; they need to be supplemented by metric data, which is usually made available in separate files. The format and algorithms associated with Type 1 fonts have been open sourced quite a while ago and are freely available to anyone who cares to inspect them.

OpenType fonts can be considered supersets of most other fonts and should be installed if possible.

Font Paths

For X Window to find your fonts, you actually have to go back to xorg.conf first. The Xorg X server needs to know where the fonts are that it can use in any and all cases. This is largely due to the fact that there are still two fonts systems that can be accessed by X11. One group of fonts is known as the core fonts and the others as the xft fonts. The latter is a bit of a misnomer since it refers to a group of libraries necessary to maintain, add, and render new fonts in your X Window System;

The information necessary for X Window to find your core fonts is contained in the Files section of the xorg.conf file. Remember that these are fonts that should also be used by xterm, and indeed most term implementations. Most "pure" X Window applications will find their fonts here. Please be aware that "pure" X11 implementations do not actually exist: they are called X apps, since they don't need more than a minimal window manager, like twm, to function.

If those applications should support xft, we will go about things a bit differently and the server font path is actually not necessary, since fonts will be available only client-side, not server-side.

A possible font path for standard X Window implementations, including FreeBSD, Mac OS X and some Linux distributions would look like this:

Section "Files"
    FontPath "/usr/local/lib/X11/fonts/
    # further Section entries follow
EndSection

Normally, the core fonts should be installed when your system has been installed. It might be advisable not to add too many non-xft fonts, even if you are tempted, since far fewer applications are likely to use them these days.

X11 Font Management

You can add fonts usable by xft and the fontconfig system under the directory above by just copying the font file directly to the /fonts directly.

First of all, though, you should check on the command line, which fonts are available:

$ fc-list ''

should list all fonts available to X Window. If you would like to check which installed font can be used to display a particular language, you can use the following flags for the fc-list command:

$ fc-list :lang=de

the : flag means "all font faces with the following characteristics, if any." lang=de restricts the search to font faces compatible with the German language, while lang=fr would restrict the search to those compatible with French.

We need to make the fontconfig system aware of any font changes by typing

$ fc-cache -f

at the command line.

To make sure that you can render most fonts available for X Window, it would be a good idea to add the so-called Freetype module in the modules section of xorg.conf:

Section "Modules"
    Load 'FreeType'
# further Section entries follow
EndSection

You can of course add other modules in this fashion, if necessary, but Freetype covers most types of fonts and processes them.

You also have to add the FontPath line in the xorg.conf to make freetype fonts accessible:

Section "Files"
    FontPath "/usr/local/lib/X11/fonts/Truetype
    # further Section entries follow
EndSection

Font Configuration

Under X11, Fonts are managed with the fontconfig library.

The file fonts.conf, which sets X11-wide defaults, should not be changed by hand, unless really necessary. Any and all font configuration changes can also be entered in the local.conf file which should be available in the /etc/fonts/local.conf (Ubuntu) or similar locations.

Again, different Linux flavors and the BSDs have subtly different rules on this matter. The local.conf file is an xml file, which should start with the following header:

<?xml version="1.0"?> 
<!DOCTYPE fontconfig SYSTEM "fonts.dtd"> 
<fontconfig>

and end with the </fontconfig> tag.

Again,You could make your font path explicit in the local.conf file, but that is not always required. Still, to stay within the pattern on font paths we have established in the xorg.conf file, we are adding your font location or locations, this time just with XML tags. Please be aware that this is an arbitrary example, and you have to follow your system defaults to match your font locations:

<dir>/usr/local/share/fonts/</dir>

How to Make Them Look Pretty

Finally, you will have to make sure that all fonts, regardless of font face or language have to display properly. One aspect is anti-aliasing, which in some cases might not work particularly well with small Arabic or Chinese fonts. You can switch off antialising for all fonts smaller than size 13 by using the following rules:

<match target="font">
<test name="size" compare="less">
<double>13</double>
</test>
<edit name="antialias" mode="assign">
<bool>false</bool>
</edit>
</match>
<match target="font">
<test name="pixelsize" compare="less" qual="any">
<double>13</double>
</test>
<edit mode="assign" name="antialias">
<bool>false</bool>
</edit>
</match>

In essence, this XML fragment looks for any fonts and figures out whether there are any fonts with sizes of less than 13. It allows for size by abstract measurement ('13') or pixel size.

It is of course much more interesting to do it for specific fonts and switch off anti-aliasing for, say, an Arabic font:

<match target="font"> 
<test name="family"> 
<string>Lateef</string> 
</test> 
<edit name="antialias" mode="assign"> 
<bool>false</bool> 
</edit> 
</match>

Font configuration can also be personalized: You don't need to centralize all details in fonts.conf, since you can also copy a fonts.conf version to your home directory and fiddle with the details of the XML file until it comes out just right.

Frank Pohlmann is a full-time writer, technology analyst, and teacher. He runs Ostracode, a small consultancy company from Manila and the U.K., while thinking about logic, poetry, and computer security.


Return to ONLamp.

Copyright © 2009 O'Reilly Media, Inc.