Category Archives: TimeLine Network Recorder

The Growth of Data on the Network and What You Should Do about It

More applications, more devices, and server virtualization adoption are all key contributors to the growth of data on networks. Recently, we came across an Infonetics research report that showed that the demand for higher-speed ports (10G, 40G, 100G) rose 62% from 2012. Not a surprise really for anyone in the networking industry.

Data on networks are colossal, and growth continues seemingly unabated.

So what does that mean for a network engineer in terms of monitoring and analyzing data? How should your habits and practices change?

Below we provide four key tactics that network engineers should abide by when handling increased network data.

Continuous Capture
With network backbones either at 10G or greater, it is essential to capture data 24/7. Traditional network analysis in the form of portable troubleshooting is no longer an option. By the time you dig out the network analyzer, find the right port(s) to monitor on the 10G switch, and get things fired up the problem is ancient history. And most laptops aren’t going to have a 10G card in them, and even if they do “standard” network interface cards (NIC) are not up to the task of lossless 10G packet capture. At 10G, you need dedicated hardware that can capture data 24/7 for easy troubleshooting the instant an issue occurs.

Check out this video for more details:

Adequate Storage
Network analysis at 10G requires not just new hardware and 24/7 monitoring, it also requires a different approach. Detailed, real-time analysis is just not practical at 10G – and it’s not required since the problem you’re looking for only encompasses a small subset of the data. What is required is ongoing recording of all network data (packets) so you can “rewind” to the timeframe of interest and perform a more targeted analysis of the specific problem. To do this, you need to store all of this packet data so it’s available when you begin your investigation. For example, if you’re recording at a full 10Gbps, and you have 32TB of disk space in your appliance, you can record about 7.0 hours of network data. Fortunately, even on a 10G network segment, you’re not going to find 10Gbps steady state on the line, so you should have enough storage space even If the problem occurs overnight. However, if you need storage for an entire weekend, you need to carefully plan your disk space against your expected aggregate traffic. One solution is to add an aggregation tap in your network infrastructure. This helps by sending packet data to multiple appliances and increases to overall storage available for heavily utilized high-speed networks.

Proper Capture Points
If you are monitoring a physical network connection, your capture points are pretty obvious, especially when dealing with a network backbone. However, with the increased volume of east-west traffic due to virtualization, you may need to adjust your monitoring points, or add some, to maintain full visibility. The best way to deal with this in a distributed virtual environment is to add a vSwitch into the architecture and use it as the connection point for your network analysis appliance. For more details on this tactic, check out our blog “Where to capture packets in high-speed and data center networks.”

Prioritizing the data you collect is key. Any amount of data that you can filter out increases the overall throughput of data you can monitor and extends the range of your available storage. For example, if you have a lot of web traffic on your network (and who doesn’t), and it’s all encrypted, why not slice all of the payloads off the data? This will significantly reduce the overall volume of data. Or perhaps backups are the biggest source of overnight network traffic. Again, you really don’t need the payloads of backup traffic; you really just want to know that it’s happening and perhaps log some metrics like the latency of the transfers. By leveraging what you know about your own network you can significantly reduce your network analysis needs, and either save money or extend the capabilities of your existing assets.

Why Customers Choose WildPackets

Customers come to us for a multitude of reasons. Some aren’t happy with their current network monitoring solutions; others are experiencing network glitches that they cannot solve; and some simply need a cohesive analysis solution. WildPackets offers a suite of products that bring customers to us from far and wide, many of whom need specific capabilities in their monitoring solution. Let’s take a look at just a few of the reasons WildPackets is the leading network analysis solution.

10G Analysis
WildPackets led the way in 10G analysis, being the first to introduce a network recorder to break the 10G barrier. When our TimeLine network recorder was introduced in 2010 it was the only network recorder to capture and store packet-level data, with no data loss whatsoever, at 11.7Gbps. Since then, WildPackets has continued to refine TimeLine, offering even more real-time statistics, increasing our overall data throughput, and adding support to capture directly from 40G network segments.

Network Forensics
Going hand-in-hand with network recording is network forensics. As you’re streaming packets to the network recorder perhaps you see a troubling trend in the real-time dashboard, or maybe a user enters a trouble ticket. Network forensics allows you to analyze a subset of your recorded data while the overall high-speed capture continues uninterrupted.

Often associated with security, network forensics goes well beyond security and also helps solve far more common issues on your network, like spikes in utilization, drops in VoIP call quality, and increased latency in both network and application performance. If a problem does occur, you no longer have to try to recreate the problem, which is typically the most time consuming task in any troubleshooting session. Instead, with TimeLine, you simply go back in time, find the problem on the dashboard, and solve it.

Remote Analysis
The days of using a laptop to perform portable analysis, especially on high-speed wired networks, are now extinct. Corporate networks are highly distributed, even for small to medium sized businesses. Even if your company operates from a single location, odds are you host some services remotely, and use some level of software-as-a-service (SaaS), making it difficult to always be where problems are occurring. WildPackets’ Omni Distributed Analysis Platform provides a wide range of options for remote network analysis, from “lightweight” software solutions like OmniPeek Remote Assistant and OmniEngine software probe, to high performance network recording appliances like TimeLine. With a WildPackets solution, network engineers can monitor and analyze highly distributed network architectures without ever leaving their desks.

Top-Down Approach to Network Monitoring
For an overall, top-down view of any network segment, customers find WildPackets flagship OmniPeek network analyzer most helpful, whether as a portable analyzer or as a console to any of our remote analysis solutions. OmniPeek provides complete visibility into your network – including Ethernet, Gigabit, 10G, 802.11a/b/g/n/ac, and VoIP and video. OmniPeek provides visual context into the network so that even junior IT staff can drill down into performance problems and solve performance issues across multiple network segments. This ensures maximum network uptime and user satisfaction.

The Full Suite of Network Monitoring and Analysis Products
And for a complete view across your entire network, WildPackets offers WatchPoint network monitor. This solution builds on our suite of distributed analysis products and provides a comprehensive graphical interface of overall network performance, including top talkers, top applications, overall utilization, VoIP performance, and detailed reporting of detected network and application problems (Experts). WatchPoint also provides a direct link for detailed, packet-level analysis to determine the root cause of any issue.

What is your favorite WildPackets product? Feel free to leave us a comment and share your thoughts.

Q&A: What you should know about managing high-speed networks

Thanks to Moore’s Law and the advances of IT, the growing appetite for network bandwidth is unabated. More applications, devices, and services are being added to the network causing greater complexity for network administration.

A lot of customers come to us with questions about how they should plan for the influx of data on their network, and how they should analyze this massive amount of data. Below, we’ve provided answers to some of the most common questions. If you have more, please leave us a comment and we’ll be happy to respond.

How should I monitor data on high-speed networks?
Due to the amount of data that is traversing the network, it is essential to identify the data you need to monitor for business purposes, especially if you plan to perform real-time analysis on the data.

We suggest dissecting your network into three areas – devices, network performance, and application performance. The reasoning here is that different solutions excel in each of these areas. Leveraging the best solution for each area will provide the maximum analysis capability, and spreading the monitoring and analysis load over several solutions will enable you to analyze even more data in real time.

Is a network recorder necessary at 10G?
Yes. At 10G, assuming you want to monitor the entire link, you need to capture and record data 24/7. It is extremely challenging, and time consuming, to recreate problems that you’ve missed at 10G. Trying to do this is like trying to find a needle in a haystack.

Use of a network recorder means you’ll be performing all of your analysis post-capture, since at 10G real-time analysis is essentially impossible. Most network recorders provide a very reasonable set of real-time stats to provide guidance when further analysis is required.

Is it better to have multiple 1G streams for network analysis, like five or six, rather than simply one network recorder monitoring at 10G?
Basically, this question boils down to “do you want to perform real-time analysis, or is post-capture analysis sufficient for your needs”? If you really want to do real-time analysis on a 10G link, use a smart tap to capture the 10G traffic and then break that traffic down into manageable chunks, typically 1Gbps or less, and then feed these slower speed streams into your real-time network analysis solution.

How does network analysis change at 10G?
The basic difference is moving from real-time analysis to post-capture analysis, or network forensics. The analysis itself is basically the same, except that in post-capture analysis you pick a small subset of the data, based on time, or an IP address range, or a certain protocol, and only do the detailed analysis on that specific slice of data. This approach allows a single appliance to be able to both capture and store all data on a 10G link and provide detailed analysis when required. As always, being judicious in your data analysis will enable you to focus on the specific problems and solve them most quickly.