Visualizing Network Traffic with Netflow and FlowScanby Michael W. Lucas
My previous article explained how to gather session-level network information with Netflow. This article shows how to convert all that nifty data into pretty multicolored pictures on a web server. If you found this article by a web search, you really do need to read the previous article. I will discuss the details of
flowscan as appropriate for the setup discussed in that article; if you are not using
flow-tools to collect your flows, and if you don't have Cflow.pm properly installed, this may or may not work for you.
FlowScan is a Perl script that parses Netflow records and stores them in RRD database files. Many network monitoring tools use RRD to show recent activity in great detail but increasingly aggregate older records. This allows retention of long-term trends without using excessive disk space. Other programs generate reports with these RRD files. FlowScan includes hooks to allow third-party modules to take advantage of FlowScan's processes to generate custom reports. I will focus on a particular FlowScan module,
Start by installing the basic
flowscan program from /usr/ports/net-mgmt/flowscan. This installs several other Perl modules as dependencies, then installs the main FlowScan tools in /usr/local/var/db/flows/bin. FlowScan will not work out of the box, however, so don't start it yet!
First, you need to install an updated FlowScan module. The official FlowScan distribution hasn't had an official update for some time, and doesn't handle flow-capture records. The author has posted an updated flowscan.pm module, but hasn't integrated it into the distribution. Fetch the updated FlowScan.pm version 1.6. Copy it into /usr/local/var/db/flows/bin, overwriting version 1.5 of that module.
This same directory contains a sample FlowScan config file, flowscan.cf.sample. Copy this to flowscan.cf and edit the copy. First, tell FlowScan where to find the flow files. The sample assumes that you are logging to the FlowScan directory, whereas you probably have a directory elsewhere. FlowScan will try to process every file in that directory unless you give it a regular expression that renders correctly, including flow-capture's temporary file and the saved directory, unless you also give it a regex that describes the completed files. The following example assumes that you're logging to /var/log/netflows, and will capture any completed flow-capture files.
ReportClasses lists all of the report modules that you're using. FlowScan comes with two modules,
SubNetIO. While they were great when they came out, I find
CUFlow more useful than either on a day-to-day basis. List all the report classes you want to use.
WaitSeconds is the number of seconds between FlowScan's attempts to process the directory. Many add-on tools assume this five-minute rollover and will break if you choose another time. While you can customize those tools and tweak
flow-capture to match them, for right now just take the defaults.
Finally, verbose logging can be very useful during setup. You might choose to remove this once the whole system is up and running correctly.
FlowScan is configured, but you still need to configure the report module, so that FlowScan records information appropriately for that report.