As the use of client-server systems continues to grow, so does the need to serve ever-expanding numbers of users. This is complicated by the competing needs of maintaining the speed and productivity of the services and maintaining affordable costs for the service provider.

Many modern systems use a three-tiered architecture. This approach consists of the client, or user interface, also known as the presentation layer; the data storage on the server end, or the database layer; and a middle layer that performs the processing and business logic that are the more time-consuming and intensive operations required during client-server interaction. The architecture separates presentation, processing, and data into distinct entities. The middle tier will normally use well-defined protocols to interact with the client and data server ends.

The Scalability Bottleneck

For the purpose of this discussion, we will consider a multiuser three-tier system. This system consists of multiple clients performing computations and uploading them to a server that stores those results in a database. This type of architecture is common in GRID computing applications. For example, the middle layer, in this case, processes and validates all result data before inserting it into the database.

Despite the separation afforded by the middle tier, a three-tier system is limited by the flow of data from the client to the data store. The business logic allows some client data to be processed, while other data is being saved to the database. However, once the server reaches its maximum permitted number of upload connections, no other clients can connect to the server to upload data or obtain information from the server to continue their operation. This coupling between the client and the data server causes a difficult-to-resolve bottleneck. Data stores — such as traditional relational databases — scale with difficulty when configured to match some existing user load. Furthermore, even large-scale databases depend on the underlying hardware for additional space, and as such are not infinitely expandable.

The middle layer does not eliminate the linear relationship between data upload and confirmation of data storage. Uploaded data must be processed and saved, and the completion of data submission depends on the data being successfully stored in the database. In effect, the client cannot continue its work until it has received confirmation or new data from the database level. This inherent coupling between the client and the server unnecessarily increases the load on both ends, as the processing-intensive logic has finished and does not play a part in the actual transfer of data.

The solution is to create an easily scalable middle tier for data storage that will parallel the idea of abstracting data processing from data storage. This further subdivides the concept of data storage into two autonomous units: data submission from the client, and the saving of data in the database, with the appropriate confirmation to the client. This is an implementation of an architectural construct known as a ticketing system.

Basic Structure of a Ticketing System

The ticketing system for a client-server application is analogous to a restaurant's order system. The diners are the clients, the kitchen is the server, and the waiters writing down the orders and delivering these order tickets to the kitchen are the ticketing system. An actual ticketing system requires a data upload from the client and a server's confirmation ticket acknowledging the data receipt. The server will then process and store the data according to the internal schedule, storing the result in the server-side ticket. In the meantime, the client periodically checks the status of the ticket until it determines that the job has completed, whether it has succeeded or failed.

To extend the analogy further, imagine if you couldn't order the main course until you had selected, ordered, received, and eaten the appetizer. If restaurants operated in this manner, we would all become fabulous home cooks. Similarly, anyone running a client on a personal or work computer would not be inclined to wait throughout a long contiguous time block to upload or receive data from the server. Worse yet, accommodating a large number of lengthy client-server connections would become unprofitable for any system relying on a three-tiered architecture.

A Modular Approach

The ticketing system eliminates the linearity of the client-server connection by providing an intermediate data store. This additional tier may be located on the same hardware as the processing logic, or on different machines that are connected to the architecture's middle tier. Figure 1 shows the view of the network under this architectural implementation:

Click for larger view
Figure 1. Network architecture with a ticketing system (Click on the image to open a full-size view.)

As indicated on the diagram, the file cache is a data store located behind the server's gateway. It is transparent to clients; they do not know it exists. The entire ticketing system functions as a plug-in module that can be added, modified, replaced, or removed without adjusting the existing components of the network. It is "plug and play" in the true sense of the term.

The modular structure of a three-tiered architecture allows this seamless integration. The middle layer provides two APIs that establish a known protocol of communication with the client on one end and the data server on the other. Each part of the architecture is a self-contained module that only needs to adhere to the rules set out by the API to work within the whole system; we do not start using money differently if the bank restructures its internal operation. Similarly, the addition of a ticketing system will be entirely invisible to the user. From the business perspective, the database does not require any major restructuring. This allows for less expensive development. If the original design is not suitable, the entire ticketing module can be changed internally without modifying the rest of the system.

The ticketing module further subdivides into three sub-modules. The file cache, or the server cache, is the temporary data store used to save client data until it is processed and entered into the database. The ticket database is a server-side database, separate from the actual client-data database, which stores all the tickets applicable to uploaded data. Finally, a server daemon processes the cached data and updates the ticket database as needed. This daemon may perform the actual data processing required by the application or use a known API to forward the correct data to an already existing application responsible for verifying and analyzing the data and storing it in the database.

If we look more closely, the ticketing module itself features a three-tiered design. The database tier is the ticket database. The effective client in this case is the server cache, since the data it contains simulates a dynamic real-time client upload. Finally, the middle logic layer is the server daemon responsible for processing the contents of the cache and relaying the result of the processing to the relevant database ticket. Once again, the advantage of this design is the lack of coupling between the components. Whether the server cache is implemented as a group of flat files or as a small-scale relational database, the rest of the system will not be affected by the choice or change of storage medium. With a robust API in place, each component remains stable with respect to changes in other parts of the system.

The modular design allows scalability on different levels of the system. If the number of users doubles, the entire ticketing system can be scaled up without affecting the client or the database. Storing additional raw data can be accommodated by adding more machines, each with a local server cache, whereas adjusting a large-scale relational database is much more tricky and costly. Similarly, changing the nature of client data will require a new algorithm for the server daemon and any relevant processing applications, but these changes need not cascade to any other parts of the system.

Finally, an important aspect of the module-driven approach is optimal resource usage. Creating modules to perform various tasks in a client-server application helps to group resources within their native environment. Data storage is one such environment, and algorithms to process some logic in a set of data are another. The more comprehensive the resulting environment, without external influences, the more efficient its performance.

In the ticketing system, the server cache fulfills the sole function of accepting and storing data from a client via an Internet connection. Since it does not need to validate the data, analyze it for certain results, or insert it into database tables, the server cache module is constantly available to receive user data. The module's operation is not hindered by other tasks, which allows the module to be readily available to service any clients. In this manner, the module is used to its maximal potential, while other modules function in parallel within their own environment. The overall effect is obtaining top productivity from all modules concurrently. In other words, you get the most for your money.

Look Who's Talking

Having a ticketing system is great for your application, but how do you implement such a system without unnecessary complications? As it turns out, the key to a successful ticketing system is the API. If you know how the components communicate with each other, the rest will fall into place.

Figures 2 and 3 show one representation of the communication protocol in the ticketing system:

Click for larger view
Figure 2. The ticket architecture (Click on the image to open a full-size view.)


Click for larger view
Figure 3. The ticket architecture, page two (Click on the image to open a full-size view.)

This diagram accounts for the various interactions between the client and the database via the server CGI. Note that we make the assumption that the ticket database in this diagram includes a database-servicing application. (We will analyze this later). The flow of processing is:

  • The client sends some result data to the server.

  • One of the available server CGIs processes the data and stores it in the server cache. Note that each web server may have its own separate cache. The server will return an error to the client if it cannot store the data in the cache. Note how this error-handling does not make unnecessary use of database resources.

  • After storing the data in the cache, the server CGI initiates the creation of a ticket in the ticket database. This is invisible to the client.

  • If ticket creation fails, the server sends an explanatory message. Otherwise, it issues a receipt for the new ticket to the client. Storing the data and creating the ticket are very rapid operations, which minimize the idle connection time while the client waits for a reply.

  • The new ticket is stored in a central repository to avoid conflicts during ticket processing. The client retains a copy of the ticket receipt in order to check the status of the request in the future.

While the client is checking for ticket status, it is free to engage in other work. Similarly, the database has resources available for server-side processing. If, during a status check, the server detects that the ticket had been removed — i.e., it was not processed during the allowable amount of time — the client will simply upload the same data again. At the heart of this system of interactions lies the need the minimize client-server connection time and to ensure prompt error handling and recovery by all parts of the ticketing system.

Let's take a closer look at the implementation of the ticketing construct. We'll start with the client, as it is most important not to inconvenience the user while modifying server-side technology. The perfect scenario will leave the client software untouched. Since the software already uploads results to the server using some predefined protocol, the ticketing system can plug into this procedure. Any good server will make sure to send a confirmation to the client once the data has been processed, or provide an error message if something went wrong or the data contained errors. The server can substitute its usual reply with a ticket receipt issued by the ticketing system to the client.

If the client is smart enough, the transition is entirely unnoticeable. A less-savvy client will need minimal modifications. The client will then contact the server at regular intervals to find out if its ticket has yet been serviced. If so, the ticket will now contain the reply that the client originally would have received directly. The key difference is that the client is free to continue its routine work while waiting for its ticket to be serviced. The only caveat in this approach is when work progression depends exclusively on the service of a particular ticket. In this case, the client should simply idle, freeing its system resources to be used elsewhere.

As for the server side, it processes tickets according to a predetermined order. One way is to process the tickets is a first-come-first-served manner — for those of you familiar with queues, this is a FIFO queue. This manner will preserve the order in which clients uploaded their data. Other algorithms prefer to work with the largest or smallest sets of data first. The benefit of this is that the clients are not aware of or influenced by this processing order.

Figure 4 illustrates the activity of the server daemon:

Click for larger view
Figure 4. Server daemon architecture (Click on the image to open a full-size view.)

Note that each server has its own set of tickets that correspond to the data stored in the server's cache. As mentioned above, the processing algorithm determines the order in which the ticket database provides tickets to the daemon.

Data Integrity and Security

Users will inevitably bring up the issues of data integrity and security. How do you know the data you sent was not intercepted by malicious software? How do you ensure that you receive two different tickets for two consecutive data uploads?

Two implementation decisions address these concerns. First, each data set and corresponding ticket is uniquely identified as belonging to a particular client. Also, each data set has a single unique ticket. This three-way identification ensures that tickets and data are specific to the sender of the data and, thus, are secure from any outside influence. Second, there is a single, central ticket repository that every middle-level application queries. This avoids potential duplication or repeated processing of the same data.

There will usually be a small application servicing the ticket database, which is akin to a CGI in that it is always available to be queried. This application allocates the correct tickets to the appropriate server daemon and schedules the overall ticket processing. It can implement the logic to create, update, and delete tickets, as required by the database. It may also perform "garbage collection" of abandoned tickets. Tickets in the database are uniquely identified with the server containing their respective data in the server cache. This eliminates the possibility of multiple server daemons processing the same data set or attempting to process data that is not available on the local server cache.

The server-end processing of the data is hidden from the client and can be more flexible. The server daemon, making queries to the ticket database helper CGI, determines which data from the cache is to be sent to the processing application. Each server cache has its own daemon. Note that multiple processing applications can run simultaneously, since there can be multiple machines dedicated to analyzing the data. This significantly improves the system's throughput rate.

Summary of Benefits

Of course, implementing the ticketing system is not entirely trivial, but the effort required in creating such a system is well worth the improvements in client-server architecture. The additional constructs inherent in the ticketing system outlined above lay the foundation for easier and more efficient system modifications. The vagueness in the preceding discussion is intentional, as it is up to reader to determine how to implement a ticketing system to meet his or her specific needs and requirements.

The key gain of the ticketing system is reduced communication time between clients and servers. It subdivides one continuous connection into one atomic data upload followed by periodic checks on ticket status. This prevents connection timeouts, avoids overloading the servers with open connections, and reduces or eliminates data errors stemming from incomplete upload transactions. As ticket-related checking done by the client is no longer demanding on the server, it leaves the server available to process the data.

Once the ticketing system is in place, it allows for transparent scaling or modifications without disrupting the work of the client. Machines can be plugged into the data cache sub-module, or the ticket database may be moved to better hardware, without requiring modifications in any of the other modules. This is profitable not only in terms of time and labor, but also in terms of testing available scaffolding by simply plugging in a new module.

Finally, the productivity gain on both the client and the server ends is quite significant. On top of reduced connectivity errors, a client relying on a ticketing system is free to do work locally while waiting for its ticket to be serviced. The server, unencumbered by lengthy client connections, can optimize the data processing as desired, while only requiring small overhead for ticket maintenance. This allows the overall system to scale to a much larger number of users than would be otherwise possible.

Return to