Recently in Network Performance Management Category

They say the truth shall set you free...

The fact is that today's networks are actually faster than the available network analysis and troubleshooting solutions, resulting in greatly diminished network visibility. This is significant to network administrators because the 10G technology enterprises are actively deploying can be difficult to troubleshoot, and until now, no vendor has been able to capture at the half-duplex line-rate without packet loss. However, achieving both real-time visibility and historical network traffic storage for post-incident analysis is possible in 10G environments - you just have to add clarity to your analysis by being specific and a bit more selective.

Three tips for being more selective when analyzing 10G Ethernet traffic:

1. Understand the network and the data you need to collect

Do not try to blindly move forward and perform analysis without knowing what data matters most to your organization. It's important to know exactly what you want to capture and what information is going to be beneficial for your analysis. Your requirements will likely vary between each network segment and you are likely going to have to capture data at several locations. The key is to use post-capture analysis and only save the data to a disk in real-time. Trying to capture and analyze simultaneously, in real-time, on highly utilized network segments puts too much strain on the system.

2.  Capture only what you need

There is a great temptation to try to capture and analyze everything because enterprises fear that the source of the problem is not immediately known. When it comes to 10G Ethernet traffic, analyzing every bit of data is nearly impossible  due to the volume of data. However, if you know your network well enough, certain conditions can be immediately ruled out. By using these clues to limit the collection and analysis to only what is necessary, you can dramatically improve network analysis performance.

3. Limits are everything

Even after analysis has been streamlined to only essential areas of the network, data capture for network analysis on 10G networks generates a great deal of data quickly, and managing the data becomes a significant challenge. The data is typically stored for subsequent retrieval and post-capture analysis. The two most common formats are standard packet files and databases. In either case, two metrics to manage closely are file size and frequency of disk writes. If the files are too large they will be unworkable on the computer being used for analysis. Smaller files lead to more frequent disk writes, and this can rob the system of resources for performing the actual packet capture. Optimum performance is achieved with a balance of these two demands, and this is different depending on the hardware resources available. After a few captures, you can determine if either of these parameters can be better optimized for your system.

Until a solution is developed that doesn't just claim "line-rate" analysis but actually allows packets to be written to disk without data loss, these tips are helpful for keeping an organization up to speed with 10G traffic. In the end, paying careful attention to detail when configuring network management systems will reward you with the analysis and troubleshooting results you desire.

It's called World Cup fever for a reason. With millions of people watching and reading related news online, it undoubtedly increased Internet traffic. In fact, people apparently care more about soccer than they do the presidential election. The very first day of the tournament, traffic exceeded the previous record of 8.5 million views per minute (vpm), which occurred when Barack Obama was elected. According to measurements by Akamai, traffic for news sites on June 11 started to climb steadily at 6 am ET and peaked six hours later, reaching nearly 12.1 vpm.  And the fact that June 11 was a workday didn't stop people from watching. People turned to their office computers to follow the action. Being able to identify high talkers and effectively manage traffic is a must for organizations that want to stay productive and successful.

Below are some considerations for enterprises in regards to their networks during times of high traffic: 

Understand how your network performs normally

The only way a company can improve network performance is to know where they stand in terms of network demands. Then they can measure success against those baselines. By looking at Internet connections, WAN links, WLAN environments and the data center, enterprises can get a sense of how their network normally acts. A network analyzer can also help organize this information into a report that can be used to not only solve issues that currently exist, but also to allow organizations to go back in time to validate performance and bandwidth utilization as necessary.

Prune and clean WLAN traffic

Remove unnecessary traffic. Devices like printers, support stacks and protocols that aren't in use in the environment can be eliminated. Sometimes, protocols that help manage the network, like routing protocols and SNMP can be found, hogging valuable bandwidth without any purpose.

Know your options

Consider a Web surfing policy. The 2010 FIFA World Cup lasts from June 11 through July 11, 2010 and will likely suck up a lot of bandwidth. Having a set policy in place will help keep traffic down and is an option to be explored. However, getting employees to abide by regulations it's often more of challenge than it's worth.

The fact is, it is important to see new trends approaching and make changes to the network to account for an enterprise's behavior. Popular events can erode profitability and corporate security, as employees not only watch but also participate in social media discussions and gamble online. Review network analysis measures and policies now, so when 2014 rolls around, networks stay healthy and humming.

A few years ago, I wrote a love story about the marriage of decoders, protospecs, and plug-ins.   Technically, it was a story about integration, but when the whole becomes so much greater than the sum of the parts, it becomes love.   Today, I have another love story to tell.    This one is about the marriage of automation and plug-ins for the OmniEngine, and the love here is so strong that I can smell it.   Yep, that's right, love is in the air, and it smells like, well, it just smells really good.

So anyway, as I have been advertising lately, more and more customers and prospects are showing an interest in automating  OmniEngines with scripts, in test labs and other environments, particularly for wireless applications.   There is also an increased interest in developing plug-ins for the OmniEngine.   Up until now though, one of the limitations has been in the lack of a way to use scripts to automate captures on the OmniEngine, and configure settings in plug-ins running in those captures.    And that is where our love story begins.

Just recently, we completed a development for a customer  that coupled the ability to automate captures though scripts and capture templates, with the ability to configure plug-ins with options from those capture templates.   This marriage of the front-end automation capabilities, with the back-end plug-in capabilities is new and very powerful, and will enable many different ways to configure and use OmniEngines.   

But you might say, "come on, how many people actually write automation scripts and plug-ins?".   I would say the question is not how many, but how big is the opportunity?     And not the opportunity to write scripts and plug-ins, but rather the opportunity to invest in an extensible network application platform that can be customized and extended to meet the diverse needs of any company that depends on the network to run their business.   It is about having all of the parts that can be put together and integrated in all kinds of different ways, creating solutions that satisfy many different customer requirements.  

Of course, it is also about supporting these solutions, which is why we have a team of developers at WildPackets, called the Custom Engineering Group, who are dedicated to this, and are very good at working with customers who want to use these capabilities of the OmniEngine to achieve the highest level of ROI possible.

Now where was I?    Oh yes,  what we did for this customer was to port the Cisco AP Capture Adapter to the OmniEngine.   We did this for an automated test lab they are building that uses OmniEngines to capture and analyze traffic from new wireless devices.    Many of our customers are doing this, and the reasons are obvious.   Testing consists of repeating process over and over.   This can be time consuming and expensive for humans, but very cheap and practical when automated.  

This customer is using the Cisco AP Capture Adapter, but there are many different adapters that can be used to capture with, for the wireless and wired traffic.   In fact, most customers use the Linksys WUB600N USB adapters .    Couple that with a USB Ethernet HUB from Belkin, and wow, the configuration possibilities are many and many.

In this case, the customer had already invested in the Cisco APs.    The  Cisco AP Capture Adapter is a plug-in adapter that receives packets from a Cisco AP and inserts them into a capture.   The problem was that the solution had to be automated through scripts, and the Cisco AP Capture Adapter needed to be configured with the port to listen on, and the IP address of the AP to accept packets from. 

To address this limitation, we enhanced the OmniScript automation library so that it can pass options from the script to the adapter plug-in.   This was the missing link which, when addressed, completed the front 2 back automation solution for them.  With the ability to set options in the plug-in through their scripts, they are able to fully automate their test lab deployment.   They still have work to do, but my understanding is at this point they have it working, are quite pleased.

Now is that love or what?   I think so, and I enjoyed being a part of it.   But the real story here is the opportunity for others to take advantage of and benefit from this type of front 2 back integration going forward.   The new version of OmniScript has been posted to MyPeek, along with the Cisco AP Capture Adapter for the OmniEngine.    Over the course of this year, we are also developing a number of new plug-ins that will use this capability.  

That is the end of the story, but while I have your ear, there is another angle to this story that may help to shed light on what is happening here.    Recently, we released a new version of the OmniEngine Plug-in Wizard to MyPeek.   This version of the Plug-in Wizard generates a plug-in with sample code that OmniPeek can use to create a capture template with plug-in options.   The resulting capture template can be used by a script to create a capture and configure the settings of the plug-in.   Wow, how cool is that?   We now have a wizard which creates sample code demonstrating all of this wonderful integration. This is yet another piece of the puzzle falling into place.   

So there you have it.   With the WildPackets OmniEngine, and some tightly integrated tools for  automation and plug-ins, the possibilities are endless.

-Chris

Roaming occurs when a handset moves out of the range of one access point into the range of another. It gives users the mobility to move around within a local coverage area and still be connected to the network. However, roaming is one of the primary reasons why users experience problems on wireless networks. Excessive roaming times lead to poor quality for voice and video over wireless and can lead to dropped calls or data connections.

Roaming usually involves a channel change, but that depends on the type of technology deployed. If it's a multi-channel architecture, which is most likely the case, a channel change is required. When roaming occurs, the client needs to be re-authenticated and re-associated with the new access point, which takes longer than 150 milliseconds in most instances, especially when advanced features like WPA2 and WMM are in use. Most organization's wireless networks are outfitted with multiple access points (APs) and users can experience poor signal strength and performance despite proper coverage in the area if the client is connected to the "wrong" AP. Even in the most modern, centrally managed systems, the wireless client is the one who decides when to switch from one AP to another. This decision is typically determined based on the current signal strength and is executed by the underlying software controlling the wireless client radio (the "supplicant"). This software is different from manufacturer to manufacturer and from device to device, so the way the decision to roam is made varies widely. In most cases, the wireless client will wait too long and as a result the available signal strength lowers, before the client switches to an AP with greater signal strength.

New and improved standards are available that specify the conditions for "fast roaming," enabling transitions that take as little as 5 - 10 milliseconds. These specifications include:

• 802.11i - with opportunistic key caching so there is no re-authentication step
• 802.11r - fast BSS transition, which optimizes the hand-off as clients move from one access point to another
• 802.11k - radio resource management of WLANs allows re- authentication to be maintained between multiple APs and has predictive capabilities

These new standards (802.11i isn't new, but it's still part of an improving situation for roaming) allow APs greater control in determining when roaming should occur and the APs are more in tune with the current performance of, and demands on, the wireless network. However, this situation is even better when the overall wireless network is under the control of a centralized manager. The issue is that adoption of 11k and 11r has been very slow, especially in wireless clients, and until adoption increases significantly users will continue to suffer slow AP transitions when roaming, leading to poor voice and video over IP performance.

In the meantime, the best approach is to carefully monitor and analyze the roaming activity on your network. Obtaining a complete and accurate view requires real-time aggregation of data from multiple channels and APs, with integrated analysis that leads to detailed reporting - who is roaming, how long each event is taking and what does the average look like for each AP. The end result is simple, yet the process is complex, demonstrating why proper network analysis tools are key to staying productive. 

The recent approval for Yahoo to build a massive campus in the heart of Silicon Valley is the perfect example of how wireless networks can grow and evolve. The sprawling campus will consist of thirteen six-story buildings across 3-million square feet. As a result, the 12,000 Yahoo employees will likely become much more dependent on wireless access, demand it from more locations and expect to have the performance and reliability characteristics of the wired network. There are several issues to consider in a wireless campus environment. As new standards come into play like fast roaming, capacity will become even more of an issue. As more people try to access the network in one area, density will increase and put significant strain on the access point resource. Security is always an issue but in a wireless network environment, it's more about the nature of those on the network and their desire to cause harm. Besides coverage, capacity, density and security issues, below are some key considerations for Yahoo and other enterprises that have distributed wireless worlds.

1. Take advantage of a single vendor's access point management system

These management systems are key in a distributed world. Before, there were wireless monitoring vendors called overlays, which offered additional sensors to access points deployed in the network. However, they were costly in terms of power, equipment and network drops. With access point management systems, companies have access to software that can control channels and power on the fly. Depending on the amount of traffic and people accessing one particular point, the power can be adjusted appropriately.

2. Keep an eye on the spectrum

There are still several interferers out there. 802.11n uses an unlicensed spectrum meaning a lot of other technologies share that same spectrum including Bluetooth, cordless phones, video cameras and microwaves. It's important to be aware of interferers, know what they are and how to manage them.

3. Don't forget about troubleshooting

Monitoring the network through access point management systems isn't enough. It only indicates trouble - it doesn't troubleshoot and solve problems. Being at the scene isn't always effective with campus wide networks, especially when you have satellite offices or are spread across acres of land. Enterprises need an analysis and troubleshooting capability on top of management that can be distributed. This can be done through purchasing additional software probes with wireless adapters. Another option is to leverage the network that is already in place by switching thin APs into promiscuous mode, where their only purpose is to receive and collect packets to be analyzed. Lastly, enterprises can equip network USB hubs with wireless adapters that plug into the Ethernet network. This makes the network transparent and businesses can access adapters to use along with their network troubleshooting software anywhere on the network.

Virtualization adoption continues to increase. According to IDC analyst Cindy Borovick, 2010 will be the first year in which the number of deployed virtual servers will actually outnumber the number of physical ones. More and more companies are turning to virtualized environments to streamline application deployment, to simplify IT operations and to allow IT organizations to respond faster to changing business demands. It's worth noting that due to a decrease in price and an increase in administrative tools that make management easier, virtualization is now being adopted even by smaller mid-market organizations.

It's important to remember that network analysis differs a bit in virtual environments because you're dealing with shared resources. In a traditional environment, you would normally span a switch port from a physical Ethernet switch or router and the data would stream across into the appliance. But in the case of a virtual environment, data comes back through a virtual adapter without actually hitting a physical switch. This creates a data blind spot in your appliance and the communication between appliances is never seen meaning problems with the applications go unknown.

In order to combat this blind spot and successfully perform network analysis in a virtual environment it is important to establish goals.  After all there is no big difference between network analysis in a virtual environment and in a traditional one in terms of goals. The only thing that changes is the implementation. Instead of capturing data at the physical layer, you collect data at the level of the virtual switches. As companies consider virtualization, there are five high-level network analysis related considerations that should be addressed... 

• Be specific - know what you want to analyze and focus only on those areas

• Understand your environment - adjust options to only applications or machines needed, settings are there for a reason and you should take advantage of them in order to receive the best performance

• Analyze the essentials - make sure you are only analyzing what you need to know. You should rule out analysis in areas that aren't of importance to your organization

• Know your resources limits - you should have an idea of your CPU expectations, disk space needed and overall CPU load

• Anticipate hardware resources needs - if you want to do post capture, it is essential to make sure you have the proper amount of disk space allocated. It's important to keep everything within a reasonable limit in a virtualized space since you are taking a single computer and optimizing several resources.

The benefits of implementing virtualization for organizations are well documented. Just make sure you're set up to do proper network analysis to keep the network healthy to maximize those benefits. 

While the cost for 10GbE is coming down and adoption is rapidly rising, there remain challenges in analyzing 10GbE traffic, most notably because the industry has yet to achieve real-time analysis at 10GbE line rates. However, 10GbE analysis is available and does not have to be limiting in terms of results. Below are six questions that will help determine if your organization is fully optimized for analyzing 10GbE traffic.

1. Are you being specific enough?

It's important to know exactly what you want to capture and what information is going to be most beneficial for your analysis. Your requirements will likely vary between each network segment and odds are you are going to have to capture data at several locations. An excellent way to analyze 10GbE traffic, especially when utilization is high, is to use post-capture analysis and only save the data to a disk in real-time. Trying to capture and analyze simultaneously, in real-time, on highly utilized network segments puts much more strain on the system than if you just save data to a disk for post-capture analysis.

2. Do you REALLY know your network?

Knowing how you expect the network to be performing is all the more critical when trying to analyze highly utilized 10GbE segments. If you're already embroiled in a complex network analysis firefight it's too late to realize that your ability to assess "normal" conditions on the network may be lacking. To get a sense of "normal" conditions before trouble arises, you should perform and archive baseline measurements across specific network traffic like HTTP and key business applications over typical cycles - like an hour, a day, and a week, for the network as a whole. Other metrics to consider include understanding packets size distribution as well as protocol and node usage over time, uncovering cycles in these metrics, which provide a "fingerprint" of your utilization. That way you will always have a clear view of the network for comparison when trouble arises. Only after convincing yourself that the basic data is in place and being collected and analyzed should you embark on detailed analysis and drill-down of packet-level data.

3.  Are you sticking to the essentials?

The temptation is to try to capture and analyze everything, especially when the source of the problem is not immediately known. But quite often certain conditions can be immediately ruled out, and using these clues to limit the collection and analysis to only what is necessary dramatically improves network analysis performance. You always have the option to customize analysis by turning off modules that are not important to the current exercise. Modules such as wireless network performance can be turned off, especially in 10GbE analysis, because odds are they are not relevant to the problem being investigated. The key is to customize your usage and take advantage of it.

4. Do you know your limits?

Even after analysis has been streamlined to only essential areas of the network, data capture for network analysis on 10GbE networks generates a great deal of data quickly, and managing the data becomes a significant challenge. Regardless of the system used, the data is typically stored for subsequent retrieval and post-capture analysis. The two most common formats are standard packet files and databases. In either case, two metrics to manage closely are file size and frequency of disk writes. Though intuition may lead you to think that the larger the file size the better, this is often not the case as very large files require very large memory footprints to open. If the files are too large they will be unworkable on the computer being used for analysis. Smaller files, however, typically lead to more frequent disk writes, and this can rob the system of precious resources for performing the actual packet capture. Optimum performance is achieved with a balance of these two demands, and this is different depending on the hardware resources available. One rule of thumb to keep in mind is that if files are being created every 30 seconds or less, it's going to increase strain on achieving the maximum packet capture rate significantly. Starting with reasonable sized buffers and files makes all the difference. We recommend that you start with 256MB buffer for packet capture and 128MB for files to be created. After a few captures you'll quickly determine if either of these parameters can be better optimized for your system. Also, try to use the lowest number of simultaneous captures as possible. In several systems, you're allowed to create as many captures as you want, but you need to remember that for each capture you open more memory is reserved for buffering and less is available for data processing.

5. Are you filtering and slicing?

Filtering is a way of limiting the overall number of packets captured and stored based upon user-specified criteria. Slicing captures and stores all of the packets, but it truncates the packets after a certain length, typically allowing you to store the header information but slice off the payloads. In both cases the same result is achieved, the overall amount of data to store is significantly reduced, freeing up more processing power for capture and analysis and more disk space for storing the data that's truly important to the current analysis task.

6. Are you being reasonable?

Most network analysis systems allow multiple users to connect to the hardware performing critical network data capture and analysis tasks. Put a limit on users. Nominate an owner for each system that will monitor filters and captures. Make sure it's understood who has the authority to go kill a capture. Too many users with too many options is a recipe for disaster. You can always scale with additional systems if needed.

Unless you've been on another planet, you've most likely heard of cloud computing - a computing approach in which the delivery, operation and management of applications, typically those used by corporations, is outsourced to a third party capable of highly scalable operations. In fact, not only has everyone heard of cloud computing, they're adopting it. Pew Research Center reports that 69 percent of U.S. Internet users are now utilizing some form of Internet-based computing.

This sudden surge to the cloud is leading to the perception that monitoring the network within enterprises is less pertinent in maintaining a functional system since applications and problems are being outsourced. Will cloud computing lead to the demise of network analysis? 

We think not. In fact, we believe that network analysis is very important, if not more critical, in cloud computing. Moving to the cloud will surely alter current network analysis processes and priorities, but a solid network is still needed to handle communication between users and applications. The only difference is that the focus of network management shifts from managing infrastructure to managing service availability and performance.

Cloud computing can be thought of as merely a shift of your application servers from your facility to a different location. Common issues within the network remain. Potential bottlenecks, bandwidth hogs and unauthorized protocol usage still adversely affect application traffic. In order to combat these issues, more diligence must be applied in monitoring application performance. It is essential to establish clear, long-term baselines before transitioning the application to the cloud. Additionally, verifying the performance of transactions that cross multiple applications is key in a new cloud environment. 

As with everything related to the network, security is always an area of concern and transitioning to the cloud is no exception. Concerns with security start the instant the cloud vendor is selected. The relationship with a third party to store and process highly sensitive data and applications needs to be closely monitored. The cloud vendor's security policies and procedures should be understood and the appropriate test processes should be implemented within your own organization to ensure these policies are not violated.

By utilizing cloud computing, a company is simply moving the location of applications and shouldn't interpret it as an excuse to forget about the network. The most important matter for companies considering the cloud is to be certain they have their network analysis processes firmly in place before the transition. This will ensure that they are not relying solely on the cloud vendor for advice. Transitioning to the cloud for any business is evolutionary and will be slow and methodical, but sufficient network analysis will further support the company's success in implementing this new technology.  

For a more in-depth discussion of how cloud computing impacts network analysis watch our free 1 hour OnDemand Webcast.

The OmniPeek Plug-ins are flying off the shelves.  And they should, since they are free to customers with maintenance.   There are over 40+ plug-ins available on MyPeek.   Some are more popular than others.   To help you decide which ones to try first, we have taken a look at the stats, and have come up with the top 10 most downloaded plug-ins.

Compass Forensics Dashboard with enhanced Milliseconds view