Using Squid on Intermittent Connectionsby Jennifer Vesperman
One of the more frequent requests on the Squid mailing lists is for help configuring Squid to operate well on dial-up or demand-dial networks. Offline mode will function for some of these networks, but is far from ideal. Unfortunately many of the features of Squid's offline mode appear to have largely vanished during the development of the Squid 2.x series. In the 2.3 STABLE 4 version, the offline mode has nearly no effect at all.
Squid can be patched to work well with dial-up and other intermittent connections. Having a cache on the intermittent side of the link can take some of the curse off these connections, providing access to cached information and reducing bandwidth use on the link. Unpatched, Squid can work reasonably well on dial-on-demand connections, but dialing in each time it needs to resolve a query can become expensive.
Squid is designed for permanent connections. Making Squid work on intermittent connections requires changing how Squid handles stale web pages and stale DNS lookups.
Pages are considered stale when their TTL (time to live) has expired. Many web pages have expiry data in the headers -- information on how often the page can be expected to be updated, and when to consider it "old." Squid has default expiry as well -- if there is no data in the headers, Squid sets its own. Stale pages are kept in the cache -- if the cache still has room for them.
If the cache runs out of space, stale pages are thrown out on an oldest-first basis. This algorithm is called LRU (least recently used). Other algorithms are available in later versions of Squid. The compile time option
--enable-heap-replacement allows you to choose other options.
If this doesn't clear enough space, pages that are still fresh get thrown out as well. This is bad for cache efficiency -- if this is happening and there is room on the disk, or you can squeeze more disk space and RAM from your boss, adjust your configuration.
Stale pages that remain in the cache are validated when a client requests the page. Squid sends a request, asking "Hey, has this page changed since [time]?", and the origin server responds with "Yeah, here's the new page," or "No, it's still good." The page is then considered fresh again, whether it's the new page or the old one. This is called an IMS request, shorthand for the term "if modified since." Since when? Since Squid last verified that it was fresh.
DNS lookups are cached as well, and have a configurable TTL. When a cached Fully Qualified Domain Name (FQDN) or IP DNS entry expires, it is removed from the cache.
So why not just use offline mode?
Offline mode is designed for complete disconnection from the network -- for reading only from what is already in the cache. It works well for demonstrations and presentations. If you were a guest on "60 Minutes" or were working offline from a trade show floor, you could display parts of your web site that are stored in the cache and they will appear with marketable speed. Just be sure to avoid anything that isn't in the cache!
How do you deal with dial-up connections? Where does Squid fit in?
Squid does this by switching off expiry of stale pages and stale lookups, and blocking off its attempts to retrieve web pages. If the web page is a "cache miss," it returns an error to the client rather than trying to retrieve the page.
Offline mode also never expires cached DNS information, and never refreshes stale web pages. This is usually not what dial-up users and others with intermittent connections want.
In the unmodified Squid configuration,
offline_mode on signals Squid to never attempt to validate, and
offline_mode off signals normal operation.
Note: In Squid 2.3 STABLE 4, the offline mode no longer has most of these functions. If you add the "intermittent connections" patch, this functionality will be restored and extended.
To work with intermittent connections, Squid must be patched. The patch must be able to tell whether or not the connection is up.
If it is connected, it must run as if in standard mode: expiring web pages; replacing stale web pages, if requested; and expiring and removing stale DNS entries.
If disconnected, it must run almost as if in offline mode -- except it must mark old pages and DNS entries as stale, so they can be removed or replaced when Squid is next connected.
Intermittent connections patch
A patch to enable intermittent connections is available from here.
The file name is
squid-tristate-offline-patch-1.0.txt. If any bugs (or even more miraculously, patches or bug fixes) turn up, please report them to email@example.com.
The patch is currently against Squid 2.3 STABLE 4, a widely deployed Squid release. Some manual changes are required to patch against later versions. To install the patch:
- Download the patch and the source code for Squid.
- Unpack your Squid source-tree, and go into its top directory.
patch -p1 --dry-run < ../squid-tristate-offline-patch-1.0.txt
- If no errors appear, remove the
--dry-runparameter and run it again.
- Compile, reinstall (
make install), and modify your
squid.conffile. Then restart Squid.
This patch has the following effects:
- In offline mode, no object is ever considered stale. Thus, no validation occurs.
- In offline mode, no
ipcacheobject is ever expired.
- In offline mode, no
fqdncacheobject is ever expired.
- If a DNS lookup takes place in offline mode, Squid inserts a dummy cache entry with an immediate expiry (that cannot be expired until Squid is no longer in offline mode).
- In offline mode, any attempt to go forward to query another server results in a "503 - Cannot Forward" error. This covers all
- Offline mode is now tri-state. It can be on, off, or dependent on the status of a network interface.
In a modified Squid, the
offline_mode directive works differently. Formerly it was a simple boolean type with the values
off. Now we use the values
# Normal online operation. offline_mode 0 # Offline operation. Use cached information only. # Do not talk to the world. offline_mode 2 # Conditionally offline. # Online if a monitored network interface # exists AND is up. If either condition is # false, Squid will act in offline mode. offline_mode 1
The new directive
network_interface takes a string parameter, which is the name of the network interface to check. If we are in mode 1, Squid will act in
online(0) mode depending on the status of this interface. Squid will test the status of this interface no more frequently than once per second.
# watch ppp0 network_interface ppp0
You can use a dummy interface to manually toggle Squid between online and offline modes.
At a prompt, run
% ifconfig dummy0 [up|down]
Because PPP connections take several seconds to properly establish after the interface is created, you may prefer to switch modes using a dummy interface and control that dummy interface from your
ip-down scripts. (See
man 8 pppd for more information.)
Caveats and gotchas
Offline mode may not work properly in unmodified versions of Squid.
As Squid is not designed for intermittent connections, none of the information in the "Further Reading" section is specific to intermittent connections. It may be useful when installing and configuring the patch.
Jennifer Vesperman is the author of Essential CVS. She writes for the O'Reilly Network, the Linux Documentation Project, and occasionally Linux.Com.
Return to the Linux DevCenter.