Softpanorama (slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-)

Slightly Skeptical View on NIDS and
Network-level Intrusion Prevention

This page is devoted to network IDS (NIDS). Most NIDS are close relatives of virus scanners and share the same major problems (number of false positives; quality of signature database and timeliness of its updates, architectural weakness of the design, sloppy coding, obscure mini-languages and/or interfaces, etc).

All-in all network IDSs are probably the most over-hyped and the least useful category of IDS. Moreover the author maintains  the view that off the shelf commercial IDS boxes with closed signature sets represent a pretty typical incarnation of cargo cult science. They are toys rather than real tools. The return on investment on a typical signature based NIDS appliance in case of using closed (or even open but generic) signatures is asymptotically close to zero.

Network IDS (NIDS) is what most people understand  under acronym "IDS". They are almost synonymous with the word "IDS" . But actually they are the least useful members of IDS family. Network IDS can help only if and only if they are frequently tuned (and not pretended to be tuned as is the case of  "universal' signature updates typical for NIDS providers like ISS) and reconciled with other tools (host-based integrity checkers are probably the most important secondary source of info).  Usability of NIDS directly depends of positioning of the sensors (the less traffic they need to analyze the better are chances to distinguish signal from noise as well as the amount of efforts spend on customizing signatures (rather expensive activity that needs to be reconciled with the period "generic" upgrades of the ruleset).

The problem of distinguishing a signal from noise is a classical problem of NIDS.  NIDS operate on a very low level (Internet layer level) and all they see are raw packets. That means that they re better suited for detection of the low level (internet layer and transport layer) attacks. For the attacks occurring on higher level (level of application protocols) NIDS events the stream needs to be reconstructed and results of detection need to be correlated with events from  HIDS and log analyzers using ESM systems like Tivoli. But most NIDS are sitting between two chairs: they try to reconstruct the stream but at the same time they are sold as "real time" detection. the results of such dilemmas and related "premature optimization" is weak, contradictive architecture with multiple "ad-hoc" mini languages as we can see in Snort.

Isolated "universal" IDS with standard signature database managed by subscription (especially expensive network IDS appliances) typically are circulating air and the only positive thing about them is that they do not lead to the deterioration of the reliability of the network infrastructure like many other useless security solutions do (although they might provide a good target for attacks as ISS appliances proved more then once).

Taking into account amount of snake oil in major IDS vendors marketing (ISS is a nice example) as well as typical cost of appliances it makes sense to look for open source solutions. They might not provide too much useful information out of the box (in this sense they are not that dissimilar from their commercial counterparts; both mostly circulate air), but they are tunable and in any case you do not need to foot the bill either and while doing politically correct thing (technology-wise) can restrict the spending to the signature subscription service (less then $2K per year for Snort). 

Without customarization after the initial deployment the dreamland of network intrusion detection you either face absence of signals or bunch of false alarms. In the latter case honest approach does not work: you need to slack.

Hundreds of log entries per day, periodic useless vulnerability alarms, IDS alerts start to interfere with your normal duties without providing much return on the investment. You're suddenly faced with an entirely new set of questions, the primary one being, how to stop information overload.

For some unknown to me reason the whole industry became pretty rotten selling mostly hype and FUD. Still I need to admit that FUD sells well. The total size of the world market for network IDS is probably several hundred millions dollars and this market niche is occupied by a lot of snake oil salesmen:

Synergy Research Group reported that the worldwide network security market spending continued to be over the $1 billion in the fourth quarter of 2005, in all segments -- hybrid solutions (firewall/VPN, appliances, and hybrid software solutions), Intrusion Detection/Prevention Systems (IDS/IPS), and SSL VPN.

IDS/IPS sales increased seven percent for the quarter and were up 30 percent over 2004. Read article here.

Most money spent on IDS can be spend with much greater return on investment on ESM software as well as on improving rules in existing or installing additional firewalls, log analyses and host based integrity checking.

That means that network IDS area is a natural area where open source software is more competitive then any commercial software. Simplifying we can even state that the fact of acquisition of commercial IDS by any organization can be a sign or weak or incompetent management ( although reality is more complex and sometimes such an acquisition is just a reaction on pressures outside IT like compliances-related pressures; moreover some implementations were done under the premises of "loss leader" mentality under the motto "let those jerks who want it have this sucker" ).

Actually an organization that is spending money on NIDS without first creating a solid foundation by deploying ESM commits what is called "innocent fraud" ;-). It does not matter what traffic you detect if you do not understand what exactly happening on your servers/workstations and view your traffic as an unstructured stream, a pond out of which IDS magically fish alerts. In reality as most time IDS is crying wolf even few useful alerts are buried in the noise. Also "real time" that is selling point for IDS does not really matter: most organization have no possibility to react promptly on alerts even if we assume that there are (very rare) cases when NIDS pick up useful signal instead on noise.  

A good introduction to NIDS can be found at NIST Draft Special Publication 800-94, Guide to Intrusion Detection and Prevention (IDP) Systems (Adobe PDF (2,333 KB) Zipped PDF (1,844 KB) )

A typical network IDS (NIDS) uses network card(s) in promiscuous mode, sniffing all packets on each network segment the server is connected to. Installations usually consists of several sensors and a central console to aggregate and analyze data (for example Snort can be used as a sensor and Acid as central console). NIDS can be classified into several types:

• A pure network IDS (NIDS) just watch the traffic. This is the most common type NIDS today and the least harmful. May be that's why they are the most widespread :-). Snort is a good example of basic functionality provided.  Shadow is an example more elaborate (and more complex) approach that instead of ad-hoc rules language uses the power of scripting language (Perl) for processing of packets.  Usefulness strongly depends of placement and tuning.  If pure NIDS is listening to large and complex network segment, then typically the useful signal is so buried in noise that the most NIDS are constantly "crying wolf" (producing so called 'security spam" stream of mail alerts) and after short honeymoon after the installation are completely ignored; so their value is totally symbolic: organization have it as a politically correct part of infrastructure so that (often semi-ignorant) vice president of IS can report to absolutely ignorant corporate board about the major success in securing the corporate network. Usefulness strongly depends on the subtype deployed. There are three major subtypes:
  • Signature-based NIDS. Those depends on signatures to detect vulnerabilities. This is typical approach used in rulebase for ISS and Snort (Snort can be used in different ways too) and all IDS most IDS appliances.
  • Policy/traffic rules violation detection. This is closer to imitation of firewall without actual firewall. Like in firewalls the rulebase can be quite elaborate with proper segmentation of IP space. As such they are more useful that plain-vanilla signature-based NIDS but combined approach might also work. This approach sometimes is called policy deviation approach. These policy deviations include detection of unauthorized network services, applications running on unusual ports and logs from network sessions denied by firewalls.
  • Statistically based anomaly detectors. Here abrupt changes in traffic patterns signify the problem.
  • More generic behavior based (including statistically based) anomaly detectors. That usually presuppose using expert system like Tivoli Enterprise Console (TEC) for processing of the stream of the alerts and possibly correlating it with other streams. Often what vendors sell as anomalies detection NIDS is nothing more then adding to signature based core some heuristic methods similar to those used in antivirus scanners. These heuristic methods might be still highly effective for detection of port scans, distributed network probes, new forms of buffer overflows and denial of service attacks.
     
• Intrusion prevention system (IPS)  combines NIDS with the ability to modify firewall rules (or worse it acts as ad-hoc firewall itself).  It can be host-based or LAN based. HOST-based systems are less dangerous (but are dangerous). LAN based IPS are the most dangerous type as it's possible to use this capabilities for denial of service attacks. If return on the investment on pure NIDS is marginal or equal zero, then the return on investment on IPS is usually negative. There are many flavors of IPS, making this type of IDS hard to define. Commercial offerings (for example from ISS) are especially problematic and can do a lot of damage... 
  
  On the other side there are possibilities of adding NIDS functionality to regular firewalls and this is promising path of developing NIDS.  If you think about it NIDS can be considered to be a passive device listen to traffic and analyze only those packets that passed firewall filtering rules. That means that 90% of processing required for NIDS were already done on firewall level.  It is in this narrow sense Gartner 2003 statement that NIDS are irrelevant and we should switch to IPS might be true.
   
• A host-based NIDS that is usually integrated with host firewall (or at least it should). Theoretically those has higher return on investment then general network NIDS as rules can be highly specific and tuned to the services that are actually available on a particular computer. For example firewall can block TFTP port while NIDS can detect and report all traffic to this port (useful for detecting some network worms that used RPC holes and TFTP delivery to propagate). Reporting can be local or centralized.  They can be integrated with HIDS especially with log monitors. In case of presence of both networking IDS and host based IDS (that can communicate with each other) on the same server such an architecture would be called a hybrid IDS.
  
  At the same time like local firewalls they represent a danger to the networking stack of the computer they supposedly protect.

The second important classification of NIDS is the placement:

• Internet-facing NIDS. Most IDS are located on ingress to network (internet connection) either before or after the firewall. In large organization such NIDS usually are overwhelmed by the complexity of the traffic and cannot produce anything but a stream of  expensive "security spam" -- email alerts that for years are ignored by personnel (most times this is hidden -- hypocrisy and political correctness rules, but sometimes it's pretty open attitude). This IDS-generated spam is probably the most expensive spam in existence. Still because this is a politically correct thing to have few have courage to dismantle them. 
• Site-based NIDS. In a large organization site is an important logical unit of network structure and the whole network is composed on multiple interconnected sites. While to understand the whole traffic is very difficult, to understand the traffic on ingress of the site (site router) is easier and often more productive.
• Special purpose NIDS. Those are speculated on listening on certain types of traffic. One common implementation is VPN NIDS that are listening just to VPN traffic.

Organizations rarely have the resources to investigate every "security" event. Instead, they must attempt to identify and address the top issues, using the tools they've been given. This is practically impossible if an IDS is listening to a large traffic stream with many different types of servers and protocols. In this case security personnel, if any, are being forced to practice triage: tackle the highest-impact problems first and move on from there. Eventually it is replaced with even more simple approach: ignore them all ;-). Of course much depends on how well signatures are tuned to particular network infrastructure.  therefore another classification can be based on the type of signature used:

• Generic signature based. In this case like in virus scanners signatures are usually supplied by the vendor using some form of paid subscription.  The usefulness of such NIDS is usually quite low.  This is exacerbated by attempt of IDS vendors to prove their usefulness by trying duplicate functionality of  antivirus mail gateway. Architecturally this is pretty funny attempts,  but they are pretty successful as vendors always can find enough lemmings to sell their wires by speculating on virus-related FUD.
• Custom signature based NIDS. Those are more useful and usually can provide some level of useful signal. They are also more expensive as maintaining custom signature set requires high qualification.

Even is case when you limit traffic to specific segment of the internal network (for example local sites in national or international corporation, which is probably the best NIDS deployment strategy) the effectiveness of network IDS is low but definitely above zero. That can be marginally useful in this restricted environment. Moreover that might have value for network troubleshooting (especially if they also configured to act as a blackbox recorder for traffic; the latter can be easily done using TCPdump as the first stage and processing TCPdump results with snort of Perl scripts, say, each quarter of an hour).  Please not that all those talks about real time detection are 99% pure security FUD. Nothing can be done in most large organizations in less the an hour ;-)

That's why many large enterprise customers (especially those who still staff that have some clue, despite all efforts spend on outsourcing) started to defect commercial IDS vendors approximately in 2003. See my IDS Whitepaper for details.  In order to preserve their business (and revenue stream) IDS vendors started to hype intrusion prevention systems as the next generation of IDS. But IPS is a very questionable idea that mixes the role of firewall with the role of IDS sensor. It's not surprising that it backfired many times for early (and/or too enthusiastic) adopters (beta addicts). 

It is very symptomatic and proves the point about "innocent fraud" that intrusion prevention usually is advertised on the base of its ability to detect mail viruses, network worms threats and Spyware.  For any specialist it is evident that mail viruses actually should be detected on mail gateway and it is benign idiotism to try to detect then on the packet filter level.  Still idiotism might be key to commercial success and most IDS vendors pay a lot of attention to the rules or signatures that provide positive PR and that automatically drives that into virus/worms detection wonderland. There are two very important points here:

• The detection of a SMTP virus should be done on higher level of protocols (it's an extremely questionable idea to use packet filter like snort for detecting email viruses). If a particular virus/worm is propagating by mail, the detection (and blocking) should be done on mail gateway, not in IP packets analyzers. But NIDS vendors needs to survive that this is the most lucrative from PR standpoint area so the results are evident: any technical logic is bypassed by marketing. If an organization attempts to detect SMTP viruses using packet filter like snort it is usually sign on complete lack on architectural vision (as well as networking skills) of personnel or advanced stage of outsourcing.  Buyers of ISS managed services automatically qualify for this litmus test ;-)
• A new network worm typically requires new signatures and commercial IDS vendors are typically very bad (always late) in this area. That actually not completely true as two recent network worms used the same TFTP based propagation but generally copycat attempts fail. New global worm propagation usually happens when the worm author manages to find a new propagation vector.

May be things eventually improve, but right now I do not see how commercial IDS can justify the return on investment and NIDS looks like a perfect area for open source solutions. In this sense please consider this page a pretty naive (missing organizational dynamic and power grab issues in large organizations) attempt to counter "innocent fraud" to borrow the catch phrase used by famous economist John Kenneth Galbraith  in the title of his last book "The Economics of Innocent Fraud".  

Important criteria for NIDS is also the level of programmability:

• Ad-hoc NIDS use some language that evolved during the evolution of a particular implementation. Snort is a good example of this category.
• Some networking IDSs are based on using a scripting languages as the main traffic analysis engine and that's presents a considerable advantage. The early example of this approach was Shadow.  As I mentioned above speed of processing packet is not an issue. Most NIDS can happily do complex processing offline after the initial binary dumps of traffic were created by TCPdump or similar tools. Attempts to use NIDS "inline" lead to premature optimization problem and are generally detrimental to quality of NIDS as they dictate some ad-hoc solutions.  Black box approach is the only right approach if you really want to detect a useful signal a typical amount of network noise. Here Snort managed to improve from its early days and the current version of Snort supports Perl-style regular expressions. Using it offline you can create as complex expressions as you wish. That substantially improves chances to pickup useful signal.

It's rather counterproductive to place NIDS in segments with large network traffic. Mirroring port on the switches work in simple cases, but in complex cases where there are multiple virtual LANs that will not work as usually only one port can be mirrored. Also mirroring increase the load on the switch. Taps are additional component and are somewhat risky on high traffic segments unless they are designed to channel all the traffic in case of failure.  Logically network IDS belongs to firewall and some commercial firewalls have rudimentary IDS functionality. Also personal firewall with NIDS component might be even be more attractive for most consumers as they provide some insight on what is happening. They also can be useful for troubleshooting.   Their major market is small business and probably people connected by DSL or cable who fear that their home computers may be invaded by crackers.

Among open source network Intrusion Detection Systems (IDS) Snort is the most well developed and powerful solution. It covered in a separate page. But along with network-based intrusion detection, one probably should pay more attention to host-based IDS that uses log analysis and integrity checking. One should never put all eggs into one basket. The most popular integrity checker is Tripwire, but it's  too primitive for the intrusion detection. See Softpanorama Integrity Checkers

The problem is that useful signal about probes on actual intrusions is usually buried under mountains of data and wrong signal may drive you in a wrong direction.  A typical way to cope with information overload from network IDS is to rely more on the aggregation of data (for example, detect scans not single probes) and  "anomaly detection"  (imitate firewall detector or use statistical criteria for traffic aggregation). 

Misuse detection is more costly and more problematic that anomaly detection approach with the notable exception of honeypots. It might be beneficial to use a hybrid tools that combine honeypots and NIDS. Just as a sophisticated home security system might comprise both external cameras and sensors and internal monitoring equipment to watch for suspicious activity both outside and within the house - so should an intrusion detection system.

You may not know it, but a surprisingly large number of IDS vendors have license provisions that can prohibit you from communicating information about the quality and usability of their security software. Some vendors have used these software license provisions to file or threaten lawsuits to silence users who criticized software quality in places such as Web sites, Usenet newsgroups, user group bulletin boards, and the technical support boards maintained by software vendors themselves. Here open source has a definite advantage, because it may be not the best but at least it is open, has a reasonable quality (for example Snort is very competitive with most popular commercial solutions) or at least it is the cheapest alternative among several equally bad choices ;-). 

IDS often (and wrongly) are considered to be the key component for the enterprise-level security. Often that is achieved by buying fashionable but mainly useless outsourced IDS services. Generally this idea has a questionable value proposition because of the level of false positives and problems with the internal infrastructure (often stupid misconfigurations on WEB server level, inability to apply patches in a timely manner, etc.) that far outweigh and IDS-inspired capabilities.  If you are buying IDS, the good staring point is to ask to show what attacks they recently detected and negotiate one to six month trial before you pay the money ("try before you buy"). 

The problem of false positives for IDS is a very important problem that is rarely discussed on a sound technological level. I don't think there is a 'best' IDS.  But here are some considerations:

• Quality of custom signatures and the procedure for their update. Any NIDS is as good as the signature database and here we have the major problem: generic signatures while marginally useful  does not provide much interesting information. the most interesting information you can get from custom signatures, that reflect the specifics of the network architecture for a particular organization as well as implicit restriction of the traffic in the sensing point. Without constant work on custom signatures usual installation of Snort generates some interest for a month or so and then becomes completely neglected like home exercise equipment.  The same is even more true for the commercial IDS (people who install Snort are usually more technically savvy that people who buy IDS appliances). While capabilities of the current NIDS are more or OK (Snort 2.4 is one example) generic signature database is usually completely detached from the reality of your network traffic and without careful tuning cannot produce useful signals.  Also the number of rules in the database that is often stressed in marketing does not mean much:  10 000 outdated rules does not buy you anything but additional false positives.  100 current rules can do much better job.
   
• The capabilities of the box (sensor). Under heavy load most NIDS systems that have complex signature databases will perform radically differently on, say, a 1.6Ghz Pentium with 256M of RAM (desktop used for a sensor) compared to a 3.2Ghz dual CPU machine with 4G of RAM.  May be 3.2Ghz and 4G of RAM is overkill (of course it depends on the configuration), but generally you need to ensure that the sensor does not drop packets under heavy load.  Another way to ensure this is to use writing the stream into a file using TCPdump and then analyzing it with snort and other tools.
   
• Level of packet analysis. Packet capture rates don't mean much unless you also take into account what the product can do with packets after it's captured them.  That's why it's important to separate packet capture from packet analysis and do packet analysis in some batches (let's say 15 min).  How much analysis does the IDS actually do and can it recreate the session? This is a really interesting area. For example Snort is usually configured with plug-ins that do decrementing and IP reassembly into valid streams that can be then replayed.
   
• Database used for the storage of packets. What you can do with data written on the disk depends on whether you store it in database of in text (or raw binary) format.  Storing all "suspicious" packets in the database along with alerts is useful and can provide more user friendly front ends, but generally everything that can be done with database can be done without it only slower. You essentially trade speed for disk space.
   
• Integration with ESM. For example if an organization uses Tivoli it should be some Tivoli integration at least for critical NIDS alerts. That can be done via TEC log adapter or other mechanism but it needs to be done.

You probably got the idea at this point: the IQ of the network/security administrators and the ability to adapt the solution to this organization is of primary importance in the IDS area, more important then in, say, virus protection (where precooked signatures sets rules despite being a huge overkill).  

All-in-all the architecture and the level of customarization of the rulebase are more important then the capabilities of the NIDS.

Dr. Nikolai Bezroukov

Old News ;-)

[Feb 20, 2007] NIST SP 800-94 Guide to Intrusion Detection and Prevention Systems (IDPS)  published

A decent introduction for non-professionals into the topic. They avoid blatant exaggerations but there is not much useful into either. Can be useful for managers, though.

Such government documents are usually pretty much castrated as they try to please everybody (or at least to avoid the wrath of vendors for telling the truth :-). Still the authors avoid  exaggerations typical for Gardner group "I love IDS" stance...  

February 2007 (NIST) Adobe PDF (1,279 KB) Zipped Adobe PDF (954 KB)

August 31, 2006 (NIST) Draft Special Publication 800-94, Guide to Intrusion Detection and Prevention (IDP) Systems
 
Adobe PDF (2,333 KB)
Zipped PDF (1,844 KB)
 
NIST announces the release of draft Special Publication (SP) 800-94, Guide to Intrusion Detection and Prevention (IDP) Systems. SP 800-94 seeks to assist organizations in understanding intrusion detection system (IDS) and intrusion prevention system (IPS) technologies and in designing, implementing, configuring, securing, monitoring, and maintaining intrusion detection and prevention (IDP) solutions. It provides practical, real-world guidance for each of four classes of IDP products: network-based, wireless, network behavior anomaly detection software, and host-based. The publication also provides an overview of complementary technologies that can detect intrusions, such as security information and event management software. It focuses on enterprise IDP solutions, but most of the information in the publication is also applicable to standalone and small-scale IDP deployments. This publication replaces NIST SP 800-31, Intrusion Detection Systems.
 
NIST requests comments on SP 800-94 by October 20, 2006. Please submit comments to 800-94comments@nist.gov with "Comments SP800-94" in the subject line.

[Oct 6, 2006] [fw-wiz] FW appliance comparison - Seeking input for the forum

A pretty interesting exchange of ideas on NIDS. Marcus J. Ranum performed much weaker then usual (actually zero constructive ideas) but several "IDS skeptics" like Patrick M. Hausen,  Paul D. Robertson, etc performed really well and provided several interesting insights. Here are several interesting quotes:

• With that much data, and 98% of it being useless, you kind have to ask yourself, "what's the point?"
• Everything I've seen says they look for known-bad-stuff and produce alerts and false positives.  
• I agree that IDS is a waste of time except if you need to provide glossy feel good reports to mgmt. Then theyr'e great!
• Why would you want a signature based IDS at all? They don't work.
  Period. Enumerating badness is a silly idea.
• Sure they do.  The premise may be flawed, but the technology works, even if
  it falls into the "better than nothing" category.  They're smoke detectors
  for a small subset of possible fires.  Using one is still better than
  letting the house burn to the ground each and every time there's a fire.
• [Paul D. Robertson ]To me, IDS is like a left-handed screwdriver, you can send someone around
  and waste their time with it, but when it comes to fixing something, it's
  not there.  If you need IDS, it's because of a failure elsewhere, and if
  you're done everything right, the value is negligable.
• That's the main reason why I don't like IDSs. A default deny
  policy combined with "log everything" achieves just the same.

LISA '06 Technical Sessions/Everything You Know About Monitoring Is Wrong Mazda A. Marvasti, Integrien Corporation

Real-time understanding of the health of a distributed system cannot be done by sucking data from a packet-level fire hose.

This talk will propose an alternative approach to managing complex distributed systems. Mazda will explain why inference from key metrics allows real-time understanding of the health of a distributed system in a way that sucking data from a packet-level fire hose does not. He'll explore the value of a steady stream of out-of-normal alerts, question the value of end-to-end service mapping, and explore whether virtualization is the silver bullet for coping with complexity.

Dr. Mazda Marvasi, Ph.D., is CTO of Integrien Corporation. Mazda leads Integrien's extensive global R&D effort as well as overseeing technology, architecture, and engineering infrastructure development at Integrien. Mazda has deep experience in leading teams in the development, deployment, and monitoring of global enterprise networks and application environments. Mazda has been recognized by "Who's Who in Science and Engineering", and received the Lockheed Leadership Fellowship Award.

Is Intrusion Detection a Dead-End Technology - CSO Talk Back

Also contains some interesting posts in feedback area

A month ago, a Gartner research report declared that intrusion detection systems were a market failure. The report and a graph depicting Gartner’s “Information Security Hype Cycle” indicated that intrusion detection system (IDS) technology had gone beyond the “peak of inflated expectations” and was rapidly sliding toward the “trough of disillusionment.” While, according to Gartner’s Hype Cycles, some technologies emerge from that dread trough and climb respectably to a plateau of usefulness, Gartner had no such hope for IDS. It said the products have failed to provide value relative to costs and will be obsolete by 2005.

That made IDS vendors cross. People who’ve spent a lot of money with them weren’t very psyched about this report, either.

Vendors and spenders alike accept some of the criticisms that Gartner lobs at the young technology, such as its high demands on networks and IT staff, its high requirement for maintenance and its high rate of false positives (one IDS user told Computerworld that his company’s IDS generated more than 600 alerts daily). But they’ve called the report’s prediction for IDS to completely fizzle short-sighted and emotional and alarmist. The products are evolving and improving, they say.

Intrusion detection systems typically work within a network’s firewall to identify and record attempts to break into or misuse the system by sniffing packets off a switch port. They alert administrators to what they find but can’t drop anything out of the flow of traffic.

Another technology often mentioned in the same breath as IDS is intrusion prevention systems (IPS), which are seen to combine the detection function of IDS but, being deployed differently, can respond more directly to perceived intrusion. The Gartner report, however, suggests that IPS is just following IDS along the hype trail to oblivion and that instead, “functionality is moving into firewalls, which will perform deep packet inspection for content and malicious traffic blocking, as well as antivirus activities.”

IDS vendors say no prevention system, be it IPS or advanced firewalls, is going to stop every attack. Therefore IDS is needed for monitoring and audit functions, in order to analyze a system’s weaknesses and adapt prevention policies to that.

Does that ring true to you, or is it wishful thinking from those invested in the hype? Is it time to cut bait and try something else, or is Gartner looking for its own hype?

Snort Enterprise Implementation

The purpose of this guide is to document the installation and configuration of a complete Snort implementation. This guide contains all the necessary information for installing and understanding the architectural layout of the implementation.  [PDF]

SecurityFocus HOME Infocus Evaluating Network Intrusion Detection Signatures Evaluating NID Signatures, Part Two by Karen Kent Frederick September 25, 2002

In this series of articles, we present recommendations that will help readers to evaluate the quality of network intrusion detection (NID) signatures, either through hands-on testing or through careful consideration of third-party product reviews and comparisons. The first installment discussed some of the basics of evaluating NID signature quality, as well selecting attacks to be used in testing. This article will conclude the discussion on criteria for choosing attacks and then provide recommendations for generating attacks and creating a good testing environment. We begin by discussing some methods of acquiring attacks and attack traffic.

Downloading Exploits From the Web

One shortcoming with most NID evaluations is that the attacks that are used to evaluate NID signature quality are chosen in an ad hoc manner. The tester typically visits a few Web sites that have exploit source code and executables, downloads some of the files, and runs them. There are several potential problems with doing this:

• Because exploits are published by so many individuals and groups, and are hosted in so many different places, there is typically no way of verifying their authenticity. The file could have been changed or replaced since it was originally distributed.
• Some exploits claim to do one thing but really do something else; for example, a program that claims to perform a format string attack against an FTP server might actually install a backdoor on the host that runs the attack. This may be done instead of the attack, or in addition to it.
• An exploit that is available in executable form only cannot be reviewed to confirm that it does what it’s supposed to. Exploits that are available in source code form are typically written in any one of several languages; many also include shellcode sequences. In order to review the code of various exploits and confirm that they meet your needs, you’d either need to be at least somewhat familiar with various languages or have access to others in your organization who have the necessary knowledge and the time to assist you.
• Many exploits are purposely “broken” by their authors before they are made publicly available. Certain instructions or values are modified so that the tool doesn’t execute the attack properly. Unless you are skilled in the required languages and exploit techniques, and have the time to dig into the code and figure out how it’s supposed to work and what needs to be fixed, you probably won’t be able to use such exploits in your testing.
• For each exploit, you’ll need to figure out how to use it. What arguments do you need to supply? What other software packages, services or other components is this exploit dependent upon? If the attack will only run from a particular operating system, you may need to acquire or set up a host to run it. If the attack targets a particular operating system, protocol and/or application, you may need to set up a host that meets the target requirements. Many exploits verify the vulnerability of the victim before actually launching the attack; for example, the exploit may connect to the victim, acquire the FTP server banner, and determine whether the FTP server is a vulnerable version based on the information in the banner message. This can rapidly become far too resource-intensive to be practical.
• This may seem like an obvious point, but the exploit is going to attack your systems. Let’s assume that you are using test systems for your attack targets, systems that can be wiped out and rebuilt as needed. Suppose an attack that you run replaces your FTP server with a Trojaned version. If you want to do additional testing of the FTP attack detection capabilities of your intrusion detection system, you may first need to replace the Trojaned FTP server with the original version, as other exploits may be expecting and requiring a particular vulnerable version of the FTP server to be present.

 

If it isn’t already clear at this point, performing intrusion detection signature testing with real exploits is often time- and resource-intensive. It can also be dangerous: running real exploits in any type of live environment is very risky. No matter how cautious you are in acquiring and evaluating exploit code, mistakes can and will still occur. A mistyped target IP address can cause a production server to be breached instead of a designated target server. To be safe and to ensure that they do not inadvertently break their own systems, readers should utilize a completely isolated test network and test hosts. Of course, this further increases the time and resources that are needed to perform IDS testing. Therefore, it is often best for all but the most skilled, enthusiastic and patient testers to avoid the use of published exploit code when performing intrusion detection signature testing.

Using IDS Testing Tools

A safer and faster alternative to using real exploits is to purchase and utilize an IDS testing tool. The best-known IDS testing tool is Blade IDS Informer, from Blade Software. Informer works by replaying IP, UDP and ICMP packets, as well as complete TCP sessions that contain various scans, probes and attacks. You can modify the source and destination IP and MAC addresses that the packets use as needed. Informer comes with hundreds of attacks, divided into categories of related attacks; the user can select which attacks or groups of attacks they would like to use. Blade regularly updates the Informer attack suite so you can keep your IDS testing fairly current with new attacks and attack techniques.

The greatest benefit behind IDS testing tools is that the vendor has gone to the trouble of doing what you might otherwise have to do – downloading and verifying exploits. The IDS testing tool vendor has the expertise to confirm the validity of each exploit and to test it in a controlled environment. The attacks are also “neutralized” by the vendor; although the attack is executed, it is altered as necessary so it does not damage the target host. Besides these attacks, the vendor may also create their own exploits and exploit traffic, based on known vulnerabilities. This gives them full control over the attack without having to rely solely on the use of published exploits.

IDS testing tools typically have other features that can be very helpful in performing intrusion detection signature testing. Testing tools normally have a user-friendly interface, which makes the process of choosing and launching attacks much easier. With a few minutes of configuration and selection, traffic from many attacks can be replayed. Think of the time you would spend setting up an IDS testing tool, versus downloading, evaluating, compiling and running even ten exploits on your own. Now think of the time if you wanted to use five hundred exploits in your testing. An additional benefit is that the tester doesn’t need any knowledge as to how the attacks work as the vendor has verified what each attack does. Report generation is also at least somewhat automated, reducing the time needed to document your results.

Another nice feature of IDS testing tools is that users are not limited to only the attacks that they include – they can also create custom attacks. For example, Informer can capture and replay attack traffic generated by the user. If you want to use a particular attack, you can do so and record its traffic for future use. This points out an important aspect of intrusion detection testing: repeatability. You may wish to validate the strength of your intrusion detection system’s signatures periodically, to confirm that the signatures not only identify new significant threats, but also continue to detect older threats that are still relevant in your environment. To save time, it’s highly desirable for testing procedures to be easily repeatable. Both the use of a testing tool such as Informer and the ability to capture and replay your own attack traffic at a later date, without having to rerun the individual attacks, are great ways to make your intrusion detection testing more efficient and consistent.

IDS testing tools also make it possible to perform IDS testing in a wider variety of environments. We emphasized earlier that when you are testing with live and potentially dangerous exploits, it’s critical to utilize an isolated test environment. If you are utilizing the standard attacks provided by the testing tool, which are modified to prevent damage, then you are probably relatively safe to run them in a production environment. This is particularly true if you modify the IP addresses of the traffic so that the source and destination addresses are unused in your environment. Also, because tools such as Informer are replaying traffic, not actually performing interactive attacks with a server, you can save substantial time and resources in terms of hardware and software deployment. As long as you have a box to run the IDS testing tool from, you can perform your testing.

Issues with Using IDS Testing Tools

As with everything, there are also some potential issues with relying solely on IDS testing tools for evaluating IDS signature strength. First, you must consider the relevance of the attacks used by the testing tool in your environment. In an environment that uses common protocols and popular applications and operating systems, it’s very likely that the attacks provided by the tool will closely match the potential threats against your environment. If your environment is unusual, then you should determine how many of the IDS testing tool’s attacks are pertinent for your environment. It does no good to buy a tool that performs 95% of its tests on signatures for operating systems, protocols and applications that your systems do not run.

Another issue with IDS testing tools is that you must trust that the vendor has neutralized the attacks properly. Imagine what would happen if you ran attacks on a production segment and an improperly modified attack breached one of your servers! If you are sufficiently paranoid – and in the security world, it usually pays to be paranoid – you would utilize an IDS testing tool on an isolated network. It can be as simple as setting up one host to run the IDS testing tool, a second host to run the IDS sensor software, and connecting the two hosts through a hub. Of course, if you want to evaluate the quality of signatures in an IDS sensor that’s already deployed into production, such a test may not be possible. If you trust the IDS testing tool, you can run it in a production environment; if you don’t, you may run its tests in an isolated environment, examine the traffic to confirm that it’s not harmful, and then run the tests again in production.

You should keep in mind that the testing tool vendor may have made mistakes in the attacks so that the attack traffic it generates does not properly reflect the actual attack. This can cause inaccurate testing results – an IDS sensor may be able to detect attack X but the IDS testing tool vendor may be replaying an invalid version of attack X. Also, the IDS signature may be intended to identify a harmful part of the attack that the IDS testing tool vendor has removed from the attack so as not to damage a target system.

A final and more subtle point is that the IDS testing tools themselves may introduce a bias into the testing. If a popular IDS testing tool uses 300 known attacks, an IDS vendor may write signatures that specifically match all these known attacks. This would have the effect of giving that vendor a higher score than others on signature quality. Depending on how the signatures are written, this may be accurate – or not. Vendors may write very weak signatures that simply match the testing tool’s implementation of particular attacks and don’t capture any other instances of the attack. If that’s the case, then the testing results will be unfairly biased toward vendors that choose to implement such weak signatures simply for the purpose of improving their test results. To counteract this, you should not rely simply on an IDS testing tool for your evaluation.

Summary

In this article, we have examined in depth two ways to test intrusion detection signature quality: by downloading published exploits from the Internet, and by utilizing an IDS testing tool, specially designed to test signature capabilities. Although there is no direct cost in downloading exploit code, it can be prohibitively expensive to base your IDS testing on downloaded code because of the extensive knowledge, time and computing resources required. Commercial IDS testing tools reduce most of the need for these resources, but their use alone may caused biased test results. The next article in this series will examine other tools that are useful for performing IDS signature testing, as well as tools that are useful in testing NID anomaly detection and the general ability for IDS sensors to detect novel attacks.

Karen Kent Frederick (kkent@bigfoot.com) is a Senior Intrusion Detection Analyst for EDS in Herndon, Virginia. She is a graduate of the University of Wisconsin-Parkside and is currently completing her master's in computer science, focusing in network security, through the University of Idaho's Engineering Outreach program. Karen has over 10 years of experience in technical support, system administration and information security. She holds several certifications, including SANS GIAC Certified Intrusion Analyst, GIAC Certified Unix Security Administrator, and GIAC Certified Incident Handler. She is one of the authors of "Intrusion Signatures and Analysis" and "Inside Network Perimeter Security: The Definitive Guide to Firewalls, Virtual Private Networks, Routers, and Intrusion Detection Systems".

Relevant Links

Evaluating NID Signatures
Karen Kent Frederick, SecurityFocus

DShield - Distributed Intrusion Detection System

DShield.org is an attempt to collect data about cracker activity from all over the internet. This data will be cataloged and summarized. It can be used to discover trends in activity and prepare better firewall rules.

Right now, the system is tailored to simple packet filters. As firewall systems that produce easy to parse packet filter logs are now available for most operating systems, this data can be submitted and used without much effort.

More complex patterns, such as are used by application level firewalls may be handled in the future.

Dealing with False Positives 

File Format: PDF/Adobe Acrobat - View as HTML
www.raid-symposium.org/raid2000/Materials/ Abstracts/50/Julisch_foils_RAID2000.pdf

ITworld.com - Real hackers go to Usenix

Dr. Blaine Burnham presented an interesting keynote address, "Design Principles of Simplicity." "Why do buffer overflow attacks still work?" he wondered. He went on to stress that security should not be an add-on. In some ways, Dr. Burnham was preaching to the choir. I know several managers and developers who refuse to accept that security needs to be designed into the architecture from the start.

As an example to illustrate his point, he referred to weeds indigenous to the American Southwest known as goatheads. These nasty little weeds produce spiked seeds that are the bane of bicyclists. Dr. Burnham pointed out that experienced cyclists quickly learned to take countermeasures to protect their tires. Why hasn't the software industry learned to take appropriate countermeasures that protect systems before they're flattened? he asked. Security must be designed into the system, not added on later. Intrusion-detection systems (IDSs) and patches are a last resort.

... ... ...

Network Security Wizards

I first noticed Network Security Wizards (NSW) and its Dragon IDS because of the similarity in the company's name and motif to those of my own company, Wizard's Keys. This coincidence reminded me of when Rain Forest Puppy referred me to Ron Gula (NSW's CEO and founder) when I was looking at intrusion-detection systems. RFP was pretty impressed with the performance of the Dragon IDS and knew I was looking for a system to monitor Cisco PIX firewalls. As luck would have it, Ron was manning the NSW booth himself and was happy to answer my inquiries.

Ron explained: "Technically, the network IDS is the Dragon Sensor, and its basic claims to fame are operation at Gigabit Ethernet speeds without hardware acceleration and one of the deeper signature libraries available on the market. The product that analyzes PIX log files is called Dragon Squire. It currently can look at any ASCII log file or stream of SNMP traps. We currently offer log analysis for Cisco, Cisco PIX, ipfilter, ipchains, and ipfw. Checkpoint, Raptor, and Sidewinder log formats are also being produced. In addition to the firewall log files, Dragon Squire also monitors log files for Web, FTP, Sendmail, DNS, syslog, and many other applications. It also performs MD5 analysis of key system files. A version for NT will be available in Q4."

IDC: IDSES AND VULNERABILITY ASSESSMENT REVENUE GOING UP

nice advertizing ;-)

Intrusion detection and vulnerability assessment revenues will pass the $1 billion mark by 2003, a new study says. Surprisingly, though, the growth is being fueled not by fear of intruders, but a need to better monitor the internal use of networks, according to a new International Data Corporation (IDC) market study.

"Organizations are purchasing intrusion detection and vulnerability assessment products to increase control and awareness over the state of their computing environments," says Charles Kolodgy, manager of IDC's Internet Security Research, in a prepared statement. The size and complexity of corporate networks are driving the dual functionality of IDSes and vulnerability assessment products, he adds. "By scanning the network, vulnerability assessment tools provide them the information they need."

The report released last week indicates revenue from intrusion detection software will increase by 37 percent CAGR between 1999 and 2004, while vulnerability assessment products will jump 32 percent. Purchases will shift from buying IDS software outright to having it embedded in appliances or sold as part of monitoring and detection services. "Little reason exists for any company outside the Fortune 200 to not seriously consider outsourcing the management of their intrusion detection software," notes IDC analyst Brian Burke.

Kevin Trosian, vice president of equity research for Banc of America Securities, agrees with IDC's evaluation of the security market. "IT managers today are looking to better utilize current assets, as opposed to purchasing new assets. And the best way to do that is to utilize technologies that can offer more control over the networks."

The decision is one of brains over brawn, Trosian says. "There's a big move--not only among IT manager but among big technology providers out there--to provide more intelligent solutions, as opposed to more powerful solutions," he says. "The networks are big--we've got a lot of power; we've got a lot of throughput through the core of the network. I think the key now is to control the traffic as it comes through. Essentially it's the same as adding a carpool lane or HOV lane to the highway."

Cisco Flow Logs and Intrusion Detection at the Ohio State University by Steve Romig, Mark Fullmer, and Suresh Ramachandran

Invisible Intruders: Rootkits in Practice by David Brumley

Indicators of UNIX Host Compromise by Paul C. Brutch, Tasneem G. Brutch, and Udo Pooch

A Hacker's Approach to ID by Mudge

A Glimpse Into the Future of ID by Tim Bass and Dave Gruber

On Reliability by John Sellens

Defending Yourself: The Role of Intrusion Detection Systems by John McHugh, Alan Christie, and Julia Allen IEEE Software:

What is the role of intrusion detection systems in an organization's overall defensive posture? This article provides guidelines for IDS deployment, operation, and maintenance.

[Jul 27, 2000] Network Intrusion Detection Using Snort By Dave Wreski & Christopher Pallack

[Jul 27, 2000] Snort Portscan Preprocessor

[Jul 27, 2000]  WIN32 port of snort

[Jul 27, 2000]  Snort Tools and Info

ZDNet eWEEK The software that cried wolf

Chances are, your company's intrusion detection software stopped suspicious-looking traffic today. Chances are, it was a false alarm, too.

Network attacks, including distributed denial-of-service and buffer overflow incursions, have put intrusion detection software on the front line in the battle against hackers. But the wider the deployment of intrusion detection, the more administrators are realizing the technology's limits and frustrations.

The reason: Too often, the software puts out false-positive alerts, which warn administrators about traffic that turns out to be innocuous but still send IT managers scurrying to plug security holes.

"It got to an absurd point, where every other day we were literally just blowing away our log file," said Robert Boyle, CEO of Tellurian Networks Inc., a managed-service provider in Newton, N.J.

Technically, false-positive intrusions are a hard problem for software companies to solve. The technology is a slave to a statistical phenomenon called the base rate fallacy. Attacks are rare relative to the amount of traffic coming into a network. The rarer the event, the more accurate the test must be to be useful. Right now, intrusion detection is not accurate enough and returns more false positives than true positives.

Root Prompt -Intrusion Detection by Lance E. Spitzner -- a simple tcpwrapper based  approach.

Your network is being scanned for vulnerabilities. This may happen only once a month or twice a day, regardless, there are people out there probing your network and systems for weaknesses. I can say this with confidence because I have yet to work on a network that has not been probed. My personal network of six systems at home is on a dedicated ISDN line. This network has no valuable data, nor represents any organization, yet I get probed two to four times a week. If you have a system or network connected to the Internet, you become a target. This article will discuss how you can protect yourself by detecting these intrusion attempts. I will then cover what you can do when you discover these attempts.

Setting up Intrusion Detection

The methods we will be discussing are simple in use and implementation. For larger or more security conscientious organizations, you may want to consider third party Intrusion Detection Systems, such as Network Flight Recorder (http://www.nfr.net/nfr). These more advanced IDS systems use traffic analysis and advance algorithms to determine if a probe has been conducted. Our approach will be somewhat simpler.

There are a variety of different probes hackers will attempt. The first type we will prepare for is one of the most common, port scans. Port scans are where an inidvidual attempts to connect to a variety of different ports. The scans can be used on a specific target, or used to scan entire IP ranges, often chosen at random This is one of the most popular information gathering methods used by hackers today as it identifies what ports and services are open.

To detect these scans, we will build a system that emails us alerts whenever someone connects to a predetermined port. First, we identify three to five of the most commonly scanned ports. Then we select two to three systems to listen on these ports. When an intruder scans our network, he will most likely hit our systems listening on these ports. When these ports are scanned, the systems log the attempt, execute various predetermined actions, then email an alert to a point of contact.

[Jun 19, 2000]SECURITY: LinuxSecurity.com: Network Intrusion Detection Using Snort

(Jun 19, 2000, 05:12 UTC) (991 reads) (0 talkbacks) (Posted by mhall) "This document takes you through the basics of intrusion detection, the steps necessary to configure a host to run the snort network intrusion detection system, testing its operation, and alerting you to possible intrusion events."

Recommended Links

• Open Directory - Computers Security Intrusion Detection Systems
• Neohapsis Archives SF-IDS
• Firewall, VPN, Intrusion Detection, and Router Tips pretty large site with good links.
• Intrusion Detection Tools -- good up-to collection of links by "Talisker" with a classification and annotations of several products (still uncompleted). He distinguishes the following types of IDS:
  • Host Based IDS / Event Log Viewers
  • Hybrid IDS: Non-Promiscuous with Event Log Viewer
  • Honey Pots
  • Network Based IDS
  • File Integrity Checkers
  • Identifying Ports
• Intrusion Detection System collection of documents
• NSS Group White Papers -- collection of whitepapers from www.nss.co.uk
  • Web Caching - CacheFlow 100
  • NDS Management Utilities
  • Internet Security
• *** COAST Hotlist - Intrusion Detection - Outdated list compiled by Katherine Price Last modified: Sat Jul 18 10:23:49 EST 1998
• Intrusion Detection Primer -- weak, but some links might be useful.
• SANS Institute Online - Home Page
• Intrusion Detection a useful collection of links
• Original list compiled by Mark Crosbie and Katherine Price, COAST Laboratory, Purdue University.  Additions from a list and bibliography compiled by Michael Sobirey, Brandenburg University of Technology at Cottbus, Germany. Updated by David A. Curry, IBM Emergency Response Service
• http://www.nswc.navy.mil/ISSEC/CID/ Northcutt's SHADOW project stuff (now free)
• See also NSA Intrusion Detection Tools Inventory
• IDS archives for Intrusion Detection Systems (IDS) Re Audit trails -- useful post
• Re wtmpx utility for solaris
• Unix Network Monitoring Tools -- A simple suite of free security tools that might help to detect and analyze network intrusion. It includes:
  •  Argus
  •  Courtney
  •  Gabriel
  •  Netlog
  •  NOCOL

FAQs

*** FAQ Network Intrusion Detection Systems somewhat outdated but useful

SANS Intrusion Detection FAQ

Network IDS FAQ by Robert Graham (nids-faq@RobertGraham.com.)

firewall-seen FAQ This document answers the age old question "I'm seeing XXXX on my firewall, what does it mean?". It also applies to intrusion detection system, it describes today's most common attacks, why the attacker is doing them, and which ones may be false-positives.

Sniffing/wiretap FAQ Describes how "sniffing/wiretap/eavesdropping" works, which is the technology that IDS is base upon. Also describes how to analyze packets in detail, because when you get attacked, you NEED to be able to pull out a protocol analyzer and look at the attack.

Newsgroups

firewall-wizards Info Page

Comparisons

Network Computing Feature Security Dragon Claws its Way to the Top Page 2 August 20, 2001

The idea behind Dragon is quite simple: It's made by the hard-core, for the hard-core. Veteran security administrators and Unix jockeys will dig it, and most Microsoft Windows administrators will choke on it. Dragon's strengths lie in its deep signature set, its robust engine, and its ability to log and display a dizzying amount of data. Its weaknesses? The Web-based console is difficult to navigate and frustrating to use.

Enterasys is one of the few vendors that can realistically talk about watching over ISP links and other carrier-class networks. The engine is rock-solid, and once you have the sensor set up and talking to the console, it hums along. We ran into a few instances where the communications died, and we needed to log into the sensor to troubleshoot, but compared with the other IDS platforms, Dragon was pretty painless.

Another issue users should consider is the number of signatures they choose to enable. Dragon has an open-signature format that lets users create their own signatures. In fact, it's so well-defined (and similar enough to the Snort signature format) that the Whitehats arachNIDS site exports all its signatures to Dragon format as well. The result is a massive signature set. At one point, we pushed out a very aggressive configuration. The Dragon infrastructure was able to keep up, but we were processing more than 2 GB of data per day! At this rate, we could hold only four days of data before we needed to start clearing out old info.

Enterasys Networks Dragon 4

On the management side, Dragon's Web-based user interface leaves much to be desired. It can be confusing, though using it gets easier after a few days. Another problem is that there are too many different Web-based tools, including Dragon Fire, Dragon Console and Dragon Rider. The consoles are bolted onto the CLI (command-line interface) tools, which are still available by SSHing into either the sensor or the console. Most of the pull-down menus and filterable options are simply flags and arguments that are passed to the back-end CLI commands. The conceptual link to this impressive and powerful array of CLI tools is presented at the top of the queries that are created through the Dragon Fire tool. The display limits on the Web pages and the frequently inefficient methods of finding the right box to supply the desired filter will quickly encourage you to use the CLI. It's raw, but it works.

IDS Test Summary

Summary - Performance Testing

In the performance tests we noticed a range of results from the excellent to the “not so good”. We suspect that the Cisco Secure IDS 4210 may well have been suffering from pre-production problems, and it will hopefully be re-evaluated in our labs at some point in the future when the problems have been ironed out. In the meantime, you should evaluate this product carefully under load in your own environment if you are considering purchase.

RealSecure, SecureNet Pro, Snort and Dragon (under Red Hat Linux) all demonstrated some problems with handling detection on a network saturated with 64 byte packets, causing them to miss attacks under load. Both Snort and SecureNet Pro, however, proved themselves to be more than a match for our “real world” tests and would, we believe, perform well on any “normal” corporate network.

Unfortunately, we were unable to re-test Dragon and RealSecure in time for this report. Given that in Edition 1 BlackICE Sentry provided the best overall performance coupled with a small footprint and very low CPU utilisation, we are looking forward to testing RealSecure 7 (which will incorporate the BlackICE technology) for the next Edition.

With the removal of BlackICE as a stand-alone product this year (because of its acquisition by ISS), only Symantec NetProwler and Cisco Secure IDS Model 4230 achieved a 100 per cent detection rate across the board, although NFR’s NID-200 gave them a run for their money. The latter two products also represent the only turnkey appliances in our test (although Intrusion Inc. also produces appliances), which may be important to some. When partnered with the netForensics product, Cisco offered some of the best reporting capabilities of all the products on test (although this makes it a very expensive combination). The value for money offered by the excellent NFR product, however, is hard to ignore.

Unfortunately, although it performed well under load, Symantec’s NetProwler tended to misrepresent many of the attacks detected and was the only one of the group that was outwitted by our IDS evasion techniques. This product is in need of updating, and we look forward to evaluating it again next year once that has happened.

Of course, performance is not everything. In terms of management and monitoring, and in terms of signature database updates, all of the Network IDS products we have examined leave something to be desired in one way or another.

Ideally, we would be looking for clear and intelligible alerting, and detailed reporting that can be printed directly or exported to a range of output options. An intuitive, easy-to-use interface that makes it simple to manage one or more remote sensors is very important, as is the ability to acquire signature updates automatically and distribute them throughout the organisation at the click of a mouse. The ability to schedule and automate signature distribution may also be useful to many organisations.

Unfortunately, not one product quite manages to combine every one of our requirements – at least not yet. RealSecure continues to be one of the easiest to deploy and configure, and provides excellent real-time alerting and reporting capabilities. Computer Associates’ new centralised administration console for eTrust is also impressive, although reporting is limited.

Symantec and nSecure both offer good centralised management capabilities, but are let down slightly by the fact that their interfaces are not always the most intuitive.

Unfortunately, both NFR and Intrusion provide only a one-to-one management console out of the box. At least NFR does offer its Central Management Server as an option, but such a capability has yet to be developed for SecureNet Pro. The latter does have an optional centralised reporting and data mining capability, but no equivalent option for centralised policy distribution and signature update (although automatic signature update via RPMs is available on the SecureNet appliances). This would currently make it the least scalable of the products tested, and we are told that this will be addressed in a future release.

The host-based IDS’ were slightly more straightforward, since all performed their allotted tasks well. It is not easy to do a side-by-side comparison, however, since they do not all perform the same set of tasks.

With the current uncertainty as to the ongoing availability of last year’s favourite “traditional” Host IDS system – CyberSafe Centrax – it is left to Symantec’s Intruder Alert to carry the flag in this area. Although the interface can be a little daunting at first, it is a very powerful tool once it has been mastered.

For those who feel constantly bewildered by the abundance of cryptic information in their Windows Event Logs, LANguard S.E.L.M. is an essential purchase, since it provides a minimal-impact means of consolidating all Event Log information from multiple servers into a central database. It also does an excellent job of explaining the meaning of the log entries.

Tripwire continues to lead the way in File Integrity Assessment, and the product has continued to evolve and improve upon the version we examined last year. Tripwire should always be considered as complimentary to any other host-based IDS you may purchase.

Finally, Entercept is the one product that we would unequivocally recommend to everyone, but especially to those who are forced to run Microsoft Web servers on a public-facing network. Entercept is the only intrusion prevention product we have evaluated, and the ability not only to detect and log attacks, but actually prevent them from happening helps to make systems that much more secure.

Kernel-level agent software helps protect the host against both known and unknown attacks, and the provision of Web Server Agents secures Web server software within an almost impregnable vault where virtually all attacks will be prevented before hitting the server. Whatever other IDS software you buy, try to make room in your budget for Entercept.

Review Intrusion-detection products grow up, 10-08-01

The IDS products from Cisco, Enterasys and Intrusion.com are appliances, while CA and ISS provided software-based systems to test. ISS also offers an IDS appliance through a partnership with Nokia, but we did not test that.

The network-based IDS products consisted of separate "sensoit management and reporting, was included on a separate console.

A variety of sensor platforms are employed, including Windows, Unix, Red Hat Linux and Solaris. With the exception of Enterasys, which uses Web browser access for management, the other products employmanagement server and/or client software. In addition to its Secure Policy Manager application, Cisco Secure IDS is also supported by Cisco Secure Director, which we did not test. This alternative management software runs on Hewlett-Packard's OpenView.

Performance

Several tests were conducted to measure performance. First, we measured how well the product could detect a random sample of commonly recognized intrusion attacks, such as ping floods, Jolt2 attacks, SYN floods, finger bombs and others. These were tested initially under no background traffic load. To achieve a passing score, the IDS had to correctly identify the attack within 5 minutes of the attack's launch. We tallied whether the intrusion was recorded, if it was correctly identified and the approximate time it took to recognize the attack.

Next, we ran stress tests to see how the products would work as background traffic load increased from 40M to 60M bit/sec, then up to 90M bit/sec. If we determined that an IDS could not detect an attack under the "no load" condition, we eliminated that attack from the stress tests. A third test determined whether the products could detect attacks specifically designed to avoid traditional IDS systems.

Performance was scored on three things: number of attacks detected, ability to detect attacks under load (the stress tests) and fault tolerance.

Enterasys' IDS Dragon took the gold in performance. In addition to its excellent showing in the first two performance tests, Dragon also beat the competition by detecting attacks that are specifically designed to avoid traditional IDS systems. IDS Dragon also performed with near bulletproof reliability - demonstrating minimal performance degradation under traffic load and solid system stability all during the tests.

The IDS Dragon missed only three out of 27 random attacks and detected 24 out of the resulting 24 attacks sent to it under the 40M and 60M bit/sec traffic load. With the 90M bit/sec traffic, IDS Dragon correctly detected 21 out of 24 attacks - very good under maximum load.

No other product performed as well with the basic intrusion-detection and stress tests, although Cisco Secure IDS performed well under load, detecting 19 out of 21 attacks sent to it under 40M-, 60M- and 90-M bit/sec loads. The ISS RealSecure performed well under 40M and 60M bit/sec loads, detecting 22 out of 24 attacks, but fell down to 17 attacks out of 24 when the traffic load went to 90M bit/sec.

Intrusion.com's SecureNet Pro had the hardest time under heavy background traffic loads. After a strong start - detecting 24 out of 27 attacks with no load - performance steadily declined as load subsequently increased. The SecureNet detected only four out of 27 attacks under the 90M-bit/sec load. Curiously, SecureNet detected the highest number of attacks (25) under no load, but supported the smallest database of known attack signatures of the products tested.

All the products tested did well in detecting certain attacks - including Whisker (various types), Targa3 and Bind - that are specifically designed to evade network-based IDS products. Cisco, CA, Enterasys and Intrusion.com detected 16 out of 17 attacks, while ISS got them all.

While CA's eTrust IDS performed adequately in our stress tests, it did not perform consistently under high (90M bit/sec) loads. It appeared that the longer we let the background traffic stream run (up to 10 minutes or more) the less consistently the eTrust could detect the attacks. It was for this reason that we rated eTrust's performance a 3 out of 5.

With few exceptions, the products tested were otherwise stable. While the ISS RealSecure was generally stable, its performance was affected in one instance. When we sent 90M bit/sec of traffic over 10 minutes, then launched an extended ping flood for several minutes, RealSecure could not detect any of the Internet Information Server exploits we tried. We also noticed that running a Jolt2 attack seemed to easily blind Intrusion.com's SecureNet for a brief time after the attack subsided.

Managing all this

All the vendors tested made huge strides in their management applications. They all performed well in generating reports, and they all exhibited their ability to adequately manage events and large deployments of IDS sensors.

Managing a large network of sensors is typically achieved through a three-tiered architecture: a central management console, sensors and an event collector that off-loads processing from the management console, but reports back to it. Under this arrangement, one event collector manages, for example, up to 50 sensors, but each management console supports multiple event collectors, thus facilitating scalability. All the vendors except CA have embraced this model. CA doesn't use the event collector, just the sensor and management console.

Cisco and ISS tied for top honors in this category. Cisco's Secure Policy Manager, which runs on Windows NT/98/2000, supports the best event management along with a highly intuitive, logically designed interface that was a breeze to use. Items were color-coded and easily sorted, and we could configure which fields we wanted displayed, easily viewing more (or less) detail, as we specified. The Secure Policy Manager also has excellent reporting and statistics, featuring easy-to-use templates for generating reports. Functions were well integrated into Secure Policy Manager, which is slated to eventually become the single manager platform for all of Cisco's security devices, including its PIX firewall.

In addition to Secure Policy Manager, Cisco supports Unix-based management and an HP OpenView-based platform, running the Cisco Secure Director plug-in, which was not tested for this review. Our only issue was its management of multiple sensors. While powerful, it was a bit cumbersome for us to set up and maintain.

The ISS RealSecure Manager, which resides on Win 2000/NT or Solaris platforms, is on par with Secure Policy Manager, supporting excellent event management, good reporting and the best integration of applications. One of the earlier IDS vendors, ISS has had more time to better integrate functions. Incorporation of a three-tired architecture facilitates management of multiple devices, and sensors are easily deployed.

CA, Enterasys and Intrusion.com were a step below, but were still good in this category. CA's eTrust Intrusion Detection Management, which runs on Win 98/NT/

2000 and Millennium Edition platforms, delivered the best statistics reporting of all five products tested. They were comprehensive and complete. But eTrust's was limited by the use of several different applications that should be integrated. For example, there were separate applications for real-time statistics and detailed monitoring. They would have been more useful if combined. With that, eTrust's management would be on par with Cisco and ISS.

While Enterasys' Web-browser based Dragon Policy Manager had good reports (we especially liked one that let us pick the most common attack and observe it closely) and statistics, its event management wasn't as robust as the other products. Events were displayed but couldn't be sorted, and we couldn't designate which fields to display. We could filter events, but we found this a tedious process. We also found the Dragon Policy Manager difficult to navigate.

However, it included a good forensics tool that let us drill down into the precise details of attacks and analyze them. While all the other products supported a similar feature, Enterasys did it best. Those familiar with Unix will like IDS Dragon's Unix-based command-line interface, which is traditional and familiar. But like CA, Enterasys needs to better integrate its varied application tools to make its management more effective and efficient.

Intrusion.com offers two management applications: the SecureNet Pro, an X-windows management application for Linux-based platforms, and the SecureNet Provider Win 2000-based application, which was developed for better management of larger deployments of sensors. The Windows application is well suited for

larger environments because it employs a three-tired architecture while the Linux application does not. While both are good, they offer distinctly different feature sets, which we found problematic.

Installation and ease of use

All products tested were easier to install and configure than in our previous tests. Overall, the appliances were easier to install than the software-based products because the software-based products also require "hardening" of the platform's operating systems, which turns off unnecessary servers (such as telnet or FTP) that could affect security or performance.

Intrusion.com's SecureNet PDS appliance was the easiest to install. Within 15 minutes, we were up and running with minimal tweaks. The Cisco Secure IDS, an appliance, was also easy to install, but because the product supports so many advanced settings and configurations, it was easy to get lost trying to find things. Also, Cisco doesn't always make good use of screen space. For example, on Secure Policy Manager's General Signatures screen, there's a small window for scrolling through signatures that the user can't expand, although there was lots of available and unused gray space around the window.

Installing CA's eTrust was intuitive and logical, but, as noted, it had too many separate applications, which limited its ease of use. CA says it is addressing this issue for future releases.

Enterasys' initial screen gives the user five options, such as policy configurations and real-time monitoring, but once you drill down into any one area, it's not easy to get back out to another. And because we were working with a Web browser, we had to repeatedly hit the "back" key to return to the main home page and then go forward to another. While it was possible to open multiple windows and switch between them, we found that cumbersome. Unix users will find that setting up the IDS Dragon Sensor through the Unix-based command-line interface fairly straightforward, although Windows users might get frustrated.

ISS's RealSecure software installation took the longest, but still took less than 30 minutes (minus hardening the server). RealSecure had the most logical and intuitive graphical user interface - almost everything we needed to see to install the system was visible on one screen so it was possible to manage devices and events without navigating through different screens. However, we found the RealSecure's "wizards" the least intuitive feature. Instead of stepping us through a process from beginning to end, the wizard presented a list of options for further selection. The wizards would have been more helpful if they were more focused.

Features

All the products supported a full complement of IDS features. With few exceptions, the products supported nearly all the specific features we looked for during our analysis.

Cisco Secure ID supported the largest database of known attack signatures, while Intrusion.com's was the smallest (which didn't prevent the SecureNet from recognizing the highest number of attacks in our random attack tests). Enterasys supported the most granular attack database, providing more details about attacks than the other products. However, some end users just want to know they've been attacked, so it's up to the individual to decide how much detail is a good thing.

Intrusion.com's SecureNet was the only product that did not support automatic update of attack signatures. It requires a manual download and installation of signatures, which forces the administrator to update each remote sensor individually - a tedious process and the main reason SecureNet scored a 3 out of 5 for features.

All the products supported a detailed explanation of attacks, including the Common Vulnerability and Exposures database of known vulnerabilities, and Bugtraq IDs, a Web site on which most security professionals release an exploit once it is discovered (http://www.securityfocus.com/).

All the products also supported a stealth-mode monitoring interface, which lets a network interface card (NIC) see all the traffic so the IDS can analyze it. NICs typically look at media access control addresses and listen only to traffic that is directed to it; software on the IDS puts the NIC into stealth (or "promiscuous") mode so that it can see everything.

Vendors have based their features development on addressing previously known vulnerabilities, such as Unicode detection and TCP reset prevention mechanisms, into their products. Unicode is a uniform coding scheme that allows communication between diverse texts (the current version is Unicode 3.1.1; see http://www.unicode.org/ for specifics). Previously, IDSes could not read Unicode, and attackers took advantage of that vulnerability. A TCP reset lets the IDS send out a spoof packet that terminates a TCP connection instead of telling a firewall to stop all packets.

All the IDSes also now support shunning, or prevention, of attacks, but it is turned off by default, and it's up to the user whether to use it.

Configuration

The Cisco Secure IDS was the bulkiest of the appliances (it is a 4-U appliance; the others were 1-U appliances), and it reported power status and disk activity, but had no link-level LEDs. It was the only product that was not offered as a stand-alone software product (as well as a stand-alone hardware/software appliance).

CA's eTrust, one of the two software-based products tested, had low minimum-system requirements, which might not support enough horsepower for high-speed-link monitoring. CA was the only vendor that did not offer an appliance-based IDS product - software only.

Enterasys offers appliance and software-versions of IDS products. We tested the appliance version, which supports LEDs that were small, recessed, not clearly labeled and hard to read. The IDS Dragon is well architected, though, incorporating a multitiered architecture for improved scalability. However, the event collector runs on the management console rather than on a separate platform, which would better conserve management resources.

Intrusion.com had the most well-designed appliance, with LEDs that were especially useful and easy to read, although there was no power button on the front of the device, which would have been a plus.

The ISS RealSecure, which we tested in software, has the advantage in scalability. RealSecure has distinctly different event collectors, and management is run on a separate machine, leaving it free to use all its resources for management.

Wrapping it up

IDS products have undergone a metamorphosis during the past year, blossoming into sophisticated machines that are easier to install, and incorporate better management and performance. But they could still use improvements in event correlation and management, and their ability to support speeds beyond 100M bit/sec.

Related links

Yocom is senior editor, Brown is lab test engineer and Van Derveer is lab technician at Miercom, a network test lab and consultancy in Princeton Junction, N.J. They can be reached at byocom@mier.com, kbrown @mier.com and dvanderveer@mier.com.

Intrusion battleground evolves The future of intrusion detection is hybrids of network- and server-based products, faster speeds and improved event correlation and analysis. But prices won't fall.

How we did it Our testing methods explained.

Interactive Buyer's Guide Use our buyer's guide to find the intrusion-detection tool that's right for you. We've got specs on 42 hardware and software-based tools.

Scorecard and NetResults

Network World on Security Sign up for our free e-mail newsletter.

Breaking intrusion-detection news

Big net names push security wares 05/13/02 Cisco, Enterasys Networks and Nortel last week used NetWorld+Interop 2002 Las Vegas to push new intrusion-detection and VPN products, recognizing that network security might be among the few things on which customers currently are willing to spend money.

Air Force goes on net security offensive 05/06/02 The U.S. Air Force is adding firepower to its network defenses by increasing intrusion-detection measures at dozens of bases around the country as the threat of cyberattacks escalates in the post-Sept. 11 age of terrorism.

Network Associates, ISS to gang up on net intruders 05/06/02 Network Associates and Internet Security Systems last week announced an alliance under which they will use each other's technologies and marketing power - an effort spurred by a rise in network security incidents known as 'blended threats.'

N+I - Cisco expands intrusion-detection lineup 05/06/02 Aiming to address the flood of network-borne threats to security from both inside and outside enterprises, Cisco Systems on Monday expanded its lineup of intrusion detection system hardware and software with new devices and management capabilities.

Network Associates, ISS forge alliance 05/02/02 Network Associates and Internet Security Systems have forged an alliance to use each other's technologies and marketing power in what top executives from the two firms say is a three-year agreement with many details still being hammered out.