pointer

Monthly Archives: October 2011

The Basics of Flow Analysis

Posted on by .

    When it comes to enterprise network monitoring, flow-based solutions are by far the most popular, with 30-40 major flow-based network monitoring solutions on the market today. With that many solutions, how do they differentiate from one another, and which one will be best for your network? To determine this, let’s start at the beginning, with the basics. How does a flow-based solution work?

    The Data Source

    Switches and routers are the primary sources of flow data. Since every packet is traversing the device, it is relatively easy for the device to extract key data from the packets, of course requiring extra processing. Depending on the protocol being used to analyze the packets, and the current load on the router or switch, sampling may be employed, and this could lower the accuracy of the data being reported. All flow-based reporting protocols categorize packets into a flow based on the following seven characteristics: source IP address, destination IP address, source port, destination port, layer 3 protocol type, TOS byte, and input logical interface. The device keeps track of all the flows, storing the information in available RAM, and once every configured interval packages up the data into a stream of UDP packets following a predefined format (like NetFlow or sFlow) and transmits these packets to a user-configured IP address, known as the Collector.

    The Data Collector

    Once the UDP data stream with the flow-based information leaves the switch or router it is purged and forgotten. It is now the responsibility of the Collector to receive, process, and store the flow-based information. Keep in mind that the original delivery to the Collector is over UDP, which is not a reliable transport, so dropped packets from the switch to the Collector can be a problem (a protocol analyzer like the OmniPeek Network Analyzer can help to identify if this is an issue on your network). Also, the packet stream from the switch is adding to the traffic load on your network, so this should be taken into consideration. Each packet typically contains information on five to ten flows, so a busy network segment can generate a significant number of packets. The frequency of data pushed from the switch to the Collector is something that is configured on the switch, and is typically set to one minute, though you may find a different interval works best in your specific environment.

    The Collector becomes the central repository for all data from that switch or router, and from many others, because a single Collector is designed to support multiple data sources. A Collector employs either a proprietary data structure or database to store the large volume of data that accumulates from the flow-based sources, and retains the data for long periods of times (months, at least) for reporting. A flow-based monitoring solution is a combination of a Collector, or set of Collectors, and a central server which processes user requests, communicates with Collectors, and returns the desired results to the user.

    What is the difference between Flow-Based Solutions?

    Differences between network monitoring solutions based on flow data come in two forms. The first is the type of flow data. Different network device vendors support different flow-based protocols. The most common protocols are NetFlow (Cisco), sFlow (Foundry), JFlow (Juniper), and IPFIX – a proposed industry standard. Each protocol deals with the generation of flow records just a bit differently, with the major difference centered on whether or not sampling is used and how aggressively it is used. The other difference in flow-based network monitoring solutions is in how the vendor presents (displays) the data, and any unique ways each vendor finds to process the data to provide unique results. Unique data processing and presentation is really the only way for vendors to differentiate themselves since the source and format of the data is essentially the same regardless of the underlying flow-based protocol.

    What solution would you find most helpful for your company and why? We always suggest that enterprises have something greater than just a flow-based solution, as flow-based solutions tend to lack all the details required for root-cause analysis on your network. If you are interested in learning more about these issues, check out our blog post, “Is A Flow-Based Solution, A Whole-Based Solution?”.

      Don’t Let the Network Get the Best of You: Take a Proactive Approach

      Posted on by .

        In our last post, we discussed research conducted by Jim Frey from EMA on what is hampering organizations from effectively managing applications and services: poorly documented or controlled changes to applications and infrastructure; poor coordination among support teams; and lengthy troubleshooting and root-cause analysis. If you are experiencing these problems, here are the top three strategies, defined both by EMA and WildPackets, that will take you from reactive problem-solving to a proactive performance assurance angle.

        1. Application Performance Is King.

        As a network professional, you need to know what is happening at the network layer, but the value that is most important and easily perceived by your users and the guys who sign your paycheck is in the application and service layer – i.e., are you quickly delivering information and results over the network?

        Having visibility into your applications is key if you want to quickly troubleshoot and solve issues when they arise. As a network engineer, request tools and develop processes that:

        • Protect the most important applications and services
        • Prioritize actions based on impact
        • Recognize new traffic contributors/aggravators and their sources before they become an issue
        • Find tools that have enterprise-wide visibility– visualize all applications on your network – and use
          them 24×7

        You may need a mix of application-aware instrumentation, from SNMP, flow-based monitoring, packet-based monitoring,  and synthetic and passive agents to cover all areas of your network. WatchPoint 2.0 is an excellent solution since it combines SNMP, flow-based monitoring, and packet-based monitoring in one package to deliver a more comprehensive management solution and keep costs down.

        2. Manage from Cradle to Grave.

        There is value to be gained by moving the typical monitoring, baselining, and characterization approaches that are used during production earlier into the application rollout process. This will help you better understand what impact new applications will have on your system.

        For example, take a VoIP project you may be starting/deploying. Before implementation, you need to establish a baseline of your current network performance, including numbers of users over time, peak usage times, average and peak latency measurements, etc. Networks have rhythms, so it’s best to assess network behavior over a long period of time, at least for several weeks and perhaps even for a month. Organizations can start this process by looking at their Internet connections, WAN links, WLAN environments, and data centers. We suggest you look into network analyzers to help you baseline.

        And of course it’s important to continue to monitor and baseline your network after you roll out your new VoIP deployment so you can quickly see whether or not the impact it has is consistent with your predictions.

        3. Take a Proactive Approach to Troubleshooting.

        Most people consider troubleshooting to be a reactive approach, but troubleshooting can be proactive as well. Proactive troubleshooting implies that constant and comprehensive monitoring is in place so that when errors arise they can be solved immediately, before they become major problems.

        It still surprises me how many enterprises invest in network monitoring and analysis solutions that are designed to operate 24×7, constantly analyzing the network for faults and providing up to the minute network statistics, only to use these solutions in an entirely reactive way – only after a network problem has been reported. You’ve already made the investment; why not leave that highly capable network monitoring and analysis solution running and let it provide ongoing analysis, 24×7, in the background on your system, always ready to alert you to issues on your network? In other words, use these solutions for proactive troubleshooting. For example, OmniEngines and Omnipliances have a whole series of Expert events running in the background, ranging from  layer 2 to layer 7 analyses. When an error occurs, you are automatically alerted and provided with information to isolate and solve the problem immediately.

        A proactive approach is the key to successful network management. Proactive analysis includes baselining your network before new applications and technologies are deployed in order to see exactly how they affect your network and whether or not the impact is as predicted. Proactive analysis also includes leveraging the full value of the network analysis solutions that may already be sitting on your shelf. Don’t let them sit idle! Plug them in and use them 24×7 to provide ongoing Expert analysis and alerts the instant that trouble begins brewing. Taking this approach will make your end users forget all about you, and in network management that’s a good thing! Just make sure the guys who sign your paycheck don’t forget about you…

          How to Better Manage Applications and Services on Your Network

          Posted on by .

            In our “Best Practices in Enterprise-wide Network Performance Management” webinar that we co-hosted with EMA research analyst Jim Frey, Jim provided some key insights into how to better manage applications and services from a network perspective, based on a recent report by EMA. The key research areas included an assessment of factors that hamper organizations’ effectiveness in terms of managing applications and services, how these organizations look to address these issues, and EMA’s opinion on how to approach and solve the identified problems.

            Here is what the study found:

            What is hindering an organizations ability to manage applications and services?

            • Changes to applications and infrastructure are not well documented or controlled.
            • Poor coordination exists between support teams.
            • Troubleshooting and root cause analysis are taking too long.

            What top three products and/or functionalities do organizations currently lack?

            • Consolidated event correlation
            • Change tracking, verification, audits to better understand changes in the infrastructure
            • Better tools for transaction management, problem identification, and root cause analysis

            Not surprisingly, these questions have very similar answers, and certainly reflect the basic problems currently being experienced in enterprise network management – poor communication (probably due to understaffed IT/network engineering departments), poor recording, and lack of appropriate tools, especially those for root cause analysis. The last point is of particular interest to us, since root cause analysis is our business, and we know we can solve this problem for every enterprise network. It’s like dependency, the first step is to admit you have a problem, which apparently enterprises are now doing. And the best part is that you can do much more than address just root cause analysis with a deep packet inspection network analysis solution. You can also provide summary-level statistics for network reporting and monitoring as well as record network activity for forensic analysis and compliance verification.

            Next week we’ll discuss the top three strategies for “Proactive Performance Assurance” as defined by EMA to better address the management of applications and services on the network.