January 2009 Archives

Synergy

| No Comments | No TrackBacks

Synergy: The working together of two or more things to produce an effect greater than the sum of their individual effects. 

So says the dictionary, but more to the point, synergy is what happens when you use multiple OmniPeek plug-ins together.   The effect is of course not limited to plug-ins, synergy is also found in the use of the many tightly integrated features like summary stats and graphs.   But plug-ins greatly increase the effect of built-in features as well increase the effect of each other.

For example, let's say that you want to capture the same packet from multiple locations on different segments of the network and compare certain things about them like the delta times, window sizes, and hop counts.   By using the Remote TCPDump Adapter Plug-in and the PeekPlayer Plug-in you can capture from multiple remote sources and aggregate all of them into a single capture window, in real-time.   Along the way, you may have used a C Decoder Plug-in, the WebStats Plug-in, or any number of other plug-ins that customize the workflow to the specific needs of your business.

Another example is using multiple RFGrabbers, where each one is capturing packets from a different channel, and using the PeekPlayer Plug-in to redirect all the packets into a single capture window.   In this way you can see the Signal and Channel graphs for all of the channels being monitored in a single capture window.   If you have not done this yet, give it a try.   The PeekPlayer makes it possible to redirect packets from one capture window to another.

Finally, take the first example and add the SQLFilter to the aggregating capture window so that as the packets are aggregated they are indexed into a single database for post capture data mining and forensics.  In this case, we are using OmniPeek, the Remote TCPDump Adapter, the PeekPlayer, and the SQLFilter.    You might also be using an expert logging plug-in, or any number of other plug-ins for filtering and processing of the packets in the final capture window.

One of the advantages in these scenarios is the aggregation of multiple streams into a single capture window in real-time.   This makes it easier to analyze the data and saves the user the time it would have taken to manually aggregate the packets through PeekCat.   But the other advantage is being able to do most of the configuration on the aggregated capture window instead of to all of them.  Again, this improves the workflow and as a result save the user time and money.

This is really all about improving workflow.   And since workflows are different from business to business and network to network, the ability to customize OmniPeek sets it apart from other products that are not quite so configurable.

 

Network Forensics

| No Comments | No TrackBacks

Introduction
Just as "Internet Search" was revolutionized by Google, so will "Packet Search" be revolutionized by the WildPackets SQLFilter Plugin.  While Google indexes content from websites, the SQLFilter indexes packet traffic to and from those websites. As more companies save large quantities of network traffic to disk, tools like the SQLFilter make it possible to search through packet data more efficiently. For network engineers, this significantly decreases the amount of time it takes to find the packets in question. Not only does the SQLFilter allow users to search for packets across 1000's of trace files using a simple UI, as well as arbitrarily complex SQL, it also loads the resulting packets directly into OmniPeek. This cuts out many of the steps usually involved in this process and dramatically shortens the period of time it takes to find packets.

The free version of the SQLFilter can be downloaded from the MyPeek Community Portal. Although it is easy to use (says me, the guy who developed it ;-)), you should download and read the documentation. It is short, but that way you will be able to make the most of the functionality offered by the plug-in.     For more advanced and scalable packet data mining and network forensics, WildPackets offers a support package through its' Professional Services Team.   This article discusses the features and components included in this support package as well as the different configurations made possible through the support of a client/server database system. 

At the core of this solution are WildPackets award winning products, the OmniPeek Console and the Omni Engine.  Built as an application on top of this rich, extensible platform are a variety of database tools.  One of these is the SQLFilter, a plugin  for searching through large numbers of trace files using arbitrarily complex SQL queries. The free version of the SQLFilter, which uses sqlite as the database back-end is extremely easy to install and use.   The UI is intuitive and the use of sqlite as the database allows for powerful and fast queries in a package that requires no database setup or maintenance.


The SQLFilter has also been enhanced to support the MySQL database.   This version is known as the "MySQLFilter".     This high-end versions of the SQLFilter has significant advantages over the sqlite version.  The MySQL version of the SQLFilter is also available for free to maintenance users.  

SQLFilter
All versions of the SQLFilter bring simple and extensible data mining to WildPackets network analyzers. The plugin lets you perform Structured Query Language (SQL) searches across very large, user-defined sets of packet files from within OmniPeek and EtherPeek. It also allows you to perform SQL, regular expression, or text string searches of the Packets view. Results of a query are packets displayed directly in the Packets view of the local Capture window.  These packets are processed by all of the available facilities like Node Stats, Protocol Stats, the Expert, and the Peermap.  The packets are also processed by plugins, including ones that you can write yourself.   The SQLFilter adds a new Tab to a Capture Window which is used to manage databases that the SQLFilter is aware of.


With the SQLFilter, SQL Queries are made through the use of a query line in the Capture Window where arbitrarily complex SQL can be entered.   The query line also knows about numerous fields like IP Address, Ethernet Address, Ports, and Dates.   These types can be entered into the query line and the SQLFilter Plugin will generate the correct SQL.   A Search Dialog is also provided which can be used to enter query parameters.

For specific instructions on how to use the WildPackets SQLFilter Plugin, please refer to the  SQLFilter PDF Manual at the bottom of the SQLFilter Page

MySQL
Of course, the manual does not reflect the newly added support being developed for MySQL.   With the MySQL support in the SQLFilter Plugin, and the ability to store the packets themselves in the MySQL database, numerous new features, more configurations, greater scalability, and better manageability become possible.    With MySQL, a single database can contain as much as 64 terabytes of data.   More specifications about MySQL can be found at http://www.mysql.com.

Because of the client/server nature of both MySQL and Omni, different configurations of the system are possible, and different choices can be made about where the data is and how much data there is in each database.   For example, in the simplest configuration, the Database and the OmniPeek Console can reside on the same machine.   This solution can be achieved with both the sqlite version of the SQLFilter, as well as the MySQL version.    The sqlite version is not client/server and each database is a single file.   In the MySQL version, the database is accessed through a MySQL service which can be local or remote on Windows or any other platform (eg Linux) that supports MySQL.

Remote Access
With the MySQL version of the SQLFilter, whole packets can be added to a database that can be queried by other users.   This is because once the packets are in the database, they can be retrieved through an SQL query.  This also makes it possible to perform string searches on the packet contents, not just the headers. 

Once the packets are in the database, multiple users can make queries against the same database simultaneously by using the SQLFilter Plugin. 


Web Browser Access
Also available from MyPeek is software that makes it possible to easily query packet databases through a web browser.   Shown below are screenshots of the SQLFilter Web Front-end.   After supplying a query, the web front-end can display a number of different views including Summary, Packets, Nodes, Pairs, Ports, and Files.   In Nodes, Pairs, and Ports views a row can be selected to display the packets that represent that item.  Finally, any packet can be selected, resulting in the display of the decode and hex for that packet.   The SQLFilter web decode uses the same decoder library and decoders used by Omni.  Like in Omni, this means you can write your own decodes.  The added benefit here is that others can see your decode through the web without having to download any dcd files.

Architecture
The SQLFilter Web Front-end software is made of standard components like a web server (eg Apache),  html, and PHP.   The result is a very flexible and extensible multi-tier system.   The diagram below shows some of the components that make up the architecture.