New Technique for Filtering Trace Files

I have discovered a new way to load trace files so that as they are loaded they are filtered.  This is really useful when the filter is an advanced plug-in filter -especially, if the plug-in filters out packets from the file and generates its own packets that are inserted instead.   In my case, I was looking for a way to load a NetFlow file, and have the NetFlow plug-in generate fake packets, instead of loading the NetFlow packets. However, this technique will work for any filter or filter plug-in.  

The key to this technique is the SQLFilter plug-in, which can be downloaded from the MyPeek Community Portal (current maintenance required).   It loads packets from a file into a real-time capture window through the SQLFilter Plug-in, instead of directly into a file window.   By using a real-time capture window, there is a user defined ring buffer, instead of a static file buffer.    This makes it possible for the NetFlow Analyzer adapter (current maintenance required) to process and filter out NetFlow packets, and create and insert fake packets into the capture instead.

Here are the steps necessary to use this technique:

First, create a real-time capture using an adapter that won't actually provide packets when the capture is started. The Microsoft Loopback Adapter is a good example, but there are others.     This is important because certain features, like the PeerMap, won't do anything unless the capture has been started.  Choosing a "NULL" type adapter allows the capture to be started, but not actually capture any packets.

Once the capture window opens, create a new advanced filter on the filter or filter plug-in of your choice.   In the example below, the NetFlow Analyzer is selected.

Next, create a new SQLFilter database and add one or more files to it.   This will add the files to the database. The database file is a single sqlite database that will now contain all the packets for all the files you load into it. This has many advantages, but in this case, we are only doing it in order to load the trace file into the real-time capture window.  In the example below, a NetFlow file is loaded into the database.

Now start the capture.   

VRoom!  Do you hear the roar of the engine?  If so, you may have a problem.  You should probably clean out the fan on your computer, or call support. ;-)

Finally, double click on the packet file.  As the packets are being loaded from the database, a progress dialog will appear. This dialog has a Cancel button, so you can cancel the load at any time. On large files, this is a very nice feature.

And that's it. I found this to be very useful, and thought I would share.