ONJava.com    
 Published on ONJava.com (http://www.onjava.com/)
 See this if you're having trouble printing code examples


Using a Request Filter to Limit the Load on Web Applications

by Ivelin Ivanov and Kevin Chipalowsky
03/24/2004

How do you respond when a web application is running a little slow? If you are like me, you try to help it along by clicking on another link, refreshing the page, using the Back button, or otherwise sending more requests to the server. Although this may seem like an innocent way to get the site "unstuck," it really just makes the situation worse by increasing the load.

Consider reading the front page of an online newspaper. A headline at the top of the page catches your attention, so you click on the link. While waiting for that page to load, you also decide to click on a link for tomorrow's weather that you found halfway down the page. A split-second later, you spot what you were really looking for in the sports section at the bottom of the page, and click on that link. As an IT professional, you probably spend a lot of time reading web pages, and are efficient at skimming sites. So, as in this example, it is conceivable that you may click three links on a familiar page within the three or four seconds it may take for a server to start sending back a response from your first request.

In this article, we present the design of a filter that synchronizes client requests and restricts the load each user can put on your applications. The source is available as RequestControlFilter.java. Application designers can use our filter to prevent a downward performance spiral where well-intentioned users bring an already overloaded server to its knees.

In the example above, you changed your mind twice and sent three requests to the server. You only care about the last request. Our filter will help the server detect that you do not care about the others and prevent it from doing unnecessary work to process them.

Restricting Requests

To control the load and prevent unnecessary processing, we want to restrict the server so that it only processes one request at a time per session. This limits the load any individual user can put on the server, regardless of how aggressively they click through the application. We do this with a prioritized request queue. Impatient users will make multiple additional requests while waiting for a first one to finish, so we need intelligence in the queue to handle the user's requests without doing unnecessary work on the server. This is how the queue works:

Java Servlet & JSP Cookbook

Related Reading

Java Servlet & JSP Cookbook
By Bruce W. Perry

This queue models the way that browser software handles a user clicking through a web site. In the earlier example, your web browser will request the headline article, then the weather, and finally the sports article. However, it will only display the sports article from your last request, even if the server starts sending back the other pages first. Similarly, here is how the example plays out in with the filter installed on the server:

  1. The server receives the request for the headline article. The queue is empty, so the server starts processing this request immediately.
  2. The server receives the request for the weather, and since it is still processing the previous article, this second request is placed in the queue.
  3. The request for the sports article arrives, and it replaces the weather request in the queue.
  4. After the server finishes processing the headline article, it processes the request for the sports article.

With the queue design presented here, the web server detects that it does not need to waste CPU cycles processing the request for the weather.

This code fragment implements the queue. Requests are queued per session.

synchronized( getSynchronizationObject( session ))
{
  // if another request is being processed,
  // then wait
  if( isRequestInProcess( session ) )
  {
    // Put this request in the queue and wait
    enqueueRequest( httpRequest );
    if( !waitForRelease( httpRequest ) )
    {
      // this request was replaced in the
      // queue by another request so it need
      // not be processed
      return;
      }
    }

    // lock the session, so that no other
    // requests are processed until this
    // one finishes
    setRequestInProgress( httpRequest );
  }

  // process this request, and then release the
  // session lock regardless of any exceptions
  // thrown farther down the chain.
  try
  {
    chain.doFilter( request, response );
  }
  finally
  {
    releaseQueuedRequest( httpRequest );
  }

This implementation expects that the HttpSession is not clustered across multiple virtual machines. Although HTTP session clustering is supported by recent versions of the major servlet containers, the popular consensus is that it is rarely justified. It would be possible to build a filter that synchronizes requests spread across multiple servers within the same session, but the overhead may exceed the potential gain. In those environments, an easier solution to performance problems may be to increase the size of the server farm.

We implement the javax.servlet.Filter interface. This feature of the Servlet 2.3 API allows us to intercept requests before any servlet has a chance to process them. The application server takes care of setting up a chain, and calls each filter's doFilter() method. In ours, the implementation decides between continuing down the chain, and canceling the request.

To use the filter with an application, it needs to be registered in the filter section of the web.xml file, as in this example:

<filter>
  <filter-name>requestControlFilter</filter-name>
  <filter-class>RequestControlFilter</filter-class>
</filter>

This filter is useful not only for limiting the load on the application server, but also for simplifying the programming model. Since the server will only be processing one request per session at a time, there may not be a need to worry about synchronizing access to objects in the session context. Only objects that could be accessed from multiple sessions might need to be synchronized.

Not All Requests Should be Serialized

The request queue is usually valuable for business transactions, which acquire expensive application resources, such as database queries or communication with other remote applications. However, this request queue need not be used to handle all requests. The designs of some applications mean that users will make multiple concurrent requests. For example, if an application creates pop-up windows, the user's browser will request the contents of a pop-up window at the same time as the contents of a new main page, all within the same session. Similarly, applications that encourage users to open links in new windows or tabs will have this same problem.

In addition to those examples, you would not want this filter to interfere with:

We decided to implement selective request filtering by using regular expressions to match request paths. From our experience, we find that we want the filter to process most requests, so the patterns match paths that should be excluded from the queue. These requests are processed immediately, regardless of which other requests the server is handling.

Matching regular expressions to the context path means that the design of the context paths must consider the load control filter. For example, you might want all of your pop-up windows to load from a context path that starts with /popup/, so that the filter can exclude them. Likewise, if you are serving static images through your application server, you could make sure that their path starts with something like /images/ that can easily be matched with a regular expression.

The patterns to exclude are configured in the web.xml file as initialization parameters to the filter. Here is an example configuration that will prevent the filter from interfering with requests for images on a particular web site:

<filter>
  <filter-name>requestControlFilter</filter-name>
  <filter-class>RequestControlFilter</filter-class>
  <init-param>
    <param-name>excludePattern.1</param-name>
    <param-value>/images/.*</param-value>
  </init-param>
</filter>

Our filter reads this configuration on initialization, and pre-compiles all of the expressions. At the beginning of the doFilter() method, we quickly compare the request's context path to all of the patterns. If there is a match, the filter hands processing down the chain immediately instead of putting it in the queue.

There are other ways we could decide which requests to queue and which to exclude. For example, the application could keep state in the session to indicate whether the user will make multiple concurrent requests.

Relaxing the Filter to Only Queue Requests For a Few Seconds

Because this filter only allows the server to process one request at a time, users will have to wait for each request to complete before the server starts working on the next one. This can be frustrating to users who have changed their minds and requested another page. In the earlier example, the user needs to wait for the server to complete the headline article request before it processes the request for the sports article. If the online newspaper includes a healthy discussion board with many comments posted about the headline, then waiting potentially fifteen seconds for the server to finish rendering the page can be painful for the user.

We got around this problem by implementing a time limit for requests waiting in the queue. This means that if the server does not complete a request within a set amount of time, the filter allows the server to start working on the next request waiting in the queue. By default, the delay is five seconds.

We give up some of the benefits of synchronized requests by only holding the queue for a short time, and must once again worry about concurrent access to shared session objects. Since a new request could be processed every five seconds per session, users can still add to the load on a server, but the impact is not as high as if they could continuously create new requests. Also, pages that take more than five seconds to complete should rarely exist, if we expect the users to be happy.

The queue timeouts are configured in the web.xml file as parameters of the filter. You can specify a value (in seconds) for different context paths. Again, the paths are regular expressions as in this example, which limits wait time to four seconds for paths that start with /test/:

<filter>
  <filter-name>requestControlFilter</filter-name>
  <filter-class>RequestControlFilter</filter-class>
  <init-param>
    <param-name>excludePattern.1</param-name>
    <param-value>/images/.*</param-value>
  </init-param>
  <init-param>
    <param-name>maxWaitMilliseconds.4000.1</param-name>
    <param-value>/test/.*</param-value>
  </init-param>
</filter>

Much like the exclude pattern, these patterns are also loaded and pre-compiled when the filter is initialized. As requests are placed into the queue, the compiled patterns are matched against the requested path to determine the timeout. We use the Object.wait() and Object.notify() synchronization methods, which allow us to specify a timeout. This is the implementation of our waitForRelease() method, which handles the timeouts:

// wait for the currently running request to
// finish, or until this thread has waited the
// maximum amount of time
try
{
  getSynchronizationObject( session ).wait(
    getMaxWaitTime( request ) );
}
catch( InterruptedException ie )
{
  return false;
}

// This request can be processed now if it hasn't
// been replaced in the queue
return (request ==
    session.getAttribute(REQUEST_QUEUE));

Experience

Our experience with this filter has been a success in our environment, which handles a moderate size of user transactions and a high volume (10,000/minute) of machine-to-machine transactions. Of course, the design of our web application guarantees that the same server handles all requests within a session. We also do not worry about multi-threaded spiders, because our application is mostly dynamic and not meant to be indexed by search engines. Your configuration may be different, but we still believe that you will find this filter to be a valuable addition to your toolbox.

Ivelin Ivanov is a frequent contributor to the open source community and has recently participated in projects that include Apache Cocoon, JBoss, GNU XQuery and Jakarta Commons.

Kevin Chipalowsky is the principal software engineer and sole proprietor of Far West Software.


Return to ONJava.com.

Copyright © 2009 O'Reilly Media, Inc.