Able to propagate quickly and change their payload with each infection, polymorphic worms have been able to evade even the most advanced intrusion detection systems (IDS). And, because zero-day worms require only seconds to launch flooding attacks on your servers, using traditional methods such as manually creating and storing signatures to defend against these threats is just too slow. Bringing together critical knowledge and research on the subject, Automatic Defense Against Zero-day Polymorphic Worms in Communication Networks details a new approach for generating automated signatures for unknown polymorphic worms. It presents experimental results on a new method for polymorphic worm detection and examines experimental implementation of signature-generation algorithms and double-honeynet systems. If you need some background, the book includes an overview of the fundamental terms and concepts in network security, including the various security models. Clearing up the misconceptions about the value of honeypots, it explains how they can be useful in securing your networks, and identifies open-source tools you can use to create your own honeypot. There's also a chapter with references to helpful reading resources on automated signature generation systems. The authors describe cutting-edge attack detection approaches and detail new algorithms to help you generate your own automated signatures for polymorphic worms. Explaining how to test the quality of your generated signatures, the text will help you develop the understanding required to effectively protect your communication networks. Coverage includes intrusion detection and prevention systems (IDPS), zero-day polymorphic worm collection methods, double-honeynet system configurations, and the implementation of double-honeynet architectures.
Preface xiii
About The Authors xvii
Chapter 1 The Fundamental Concepts 1 (24)
1.1 Introduction 1 (21)
1.1.1 Network Security Concepts 1 (19)
1.1.2 Automated Signature Generation for 20 (2)
Zero-day Polymorphic Worms
1.2 Our Experience and This Book's Objective 22 (1)
References 23 (2)
Chapter 2 Computer Networking 25 (22)
2.1 Computer Technologies 25 (1)
2.2 Network Topology 26 (8)
2.2.1 Point-to-Point Topology 26 (1)
2.2.2 Daisy-Chain Topology 27 (1)
2.2.3 Bus (Point-to-Multipoint) Topology 27 (1)
2.2.4 Distributed Bus Topology 27 (2)
2.2.5 Ring Topology 29 (1)
2.2.6 Dual-Ring Topology 29 (1)
2.2.7 Star Topology 29 (1)
2.2.8 Star-Wired Bus Topology 30 (1)
2.2.9 Star-Wired Ring Topology 31 (1)
2.2.10 Mesh Topology 32 (1)
2.2.11 Hierarchical or Tree Topology 32 (1)
2.2.12 Dual-Homing Topology 32 (2)
2.3 Internet Protocol 34 (1)
2.4 Transmission Control Protocol 34 (1)
2.5 IP Routers 35 (1)
2.6 Ethernet Switch 35 (1)
2.7 IP Routing and Routing Table 36 (1)
2.8 Discussion on Router 37 (5)
2.8.1 Access Mechanisms for Administrators 37 (1)
2.8.2 Security Policy for a Router 38 (2)
2.8.3 Router Security Policy Checklist 40 (2)
2.9 Network Traffic Filtering 42 (1)
2.9.1 Packet Filtering 42 (1)
2.9.2 Source Routing 43 (1)
2.10 Tools Used for Traffic Filtering or 43 (1)
Network Monitoring
2.10.1 Packet Capture 44 (1)
2.11 Concluding Remarks 44 (1)
References 45 (2)
Chapter 3 Intrusion Detection And Prevention 47 (38)
Systems (IDPSs)
3.1 Introduction 47 (7)
3.2 IDPS Detection Methods 54 (5)
3.2.1 Signature-Based Detection 54 (1)
3.2.2 Anomaly-Based Detection 55 (2)
3.2.3 Stateful Protocol Analysis 57 (2)
3.3 IDPS Components 59 (1)
3.4 IDPS Security Capabilities 60 (1)
3.5 Types of IDPS Technologies 61 (16)
3.5.1 Network-Based IDPSs 62 (4)
3.5.2 Wireless IDPSs 66 (4)
3.5.3 NBA Systems 70 (3)
3.5.4 Host-Based IDPS 73 (4)
3.6 Integration of Multiple IDPSs 77 (1)
3.6.1 Multiple IDPS Technologies 77 (1)
3.6.2 Integration of Different IDPS Products 78 (1)
3.7 IDPS Products 78 (5)
3.7.1 Common Enterprise Network-Based IDPSs 78 (1)
3.7.2 Common Enterprise Wireless IDPSs 78 (1)
3.7.3 Common Enterprise NBA Systems 78 (1)
3.7.4 Common Enterprise Host-Based IDPSs 78 (5)
3.8 Concluding Remarks 83 (1)
References 83 (2)
Chapter 4 Honeypots 85 (42)
4.1 Definition and History of Honeypots 85 (11)
4.1.1 Honeypot and Its Working Principle 85 (4)
4.1.2 History of Honeypots 89 (6)
4.1.3 Types of Honeypots 95 (1)
4.2 Types of Threats 96 (5)
4.2.1 Script Kiddies and Advanced Blackhat 96 (4)
Attacks
4.2.2 Attackers' Motivations 100 (1)
4.3 The Value of Honeypots 101 (8)
4.3.1 Advantages of Honeypots 101 (3)
4.3.2 Disadvantages of Honeypots 104 (1)
4.3.3 Roles of Honeypots in Network Security 105 (4)
4.4 Honeypot Types Based on Interaction Level 109 (4)
4.4.1 Low-Interaction Honeypots 110 (1)
4.4.2 High-Interaction Honeypots 111 (1)
4.4.3 Medium-Interaction Honeypots 112 (1)
4.5 An Overview of Five Honeypots 113 (10)
4.5.1 BackOfficer Friendly 113 (1)
4.5.2 Specter 113 (1)
4.5.3 Honeyd 114 (1)
4.5.4 ManTrap 114 (1)
4.5.5 Honeynets 115 (8)
4.6 Conclusion 123 (1)
References 124 (3)
Chapter 5 Internet Worms 127 (32)
5.1 Introduction 127 (1)
5.2 Infection 127 (7)
5.2.1 Code Injection 128 (2)
5.2.2 Edge Injection 130 (3)
5.2.3 Data Injection 133 (1)
5.3 Spreading 134 (2)
5.4 Hiding 136 (3)
5.4.1 Traffic Shaping 136 (1)
5.4.2 Polymorphism 137 (1)
5.4.3 Fingerprinting 138 (1)
5.5 Worm Components 139 (1)
5.5.1 Reconnaissance 139 (1)
5.5.2 Attack Components 139 (1)
5.5.3 Communication Components 139 (1)
5.5.4 Command Components 140 (1)
5.5.5 Intelligence Capabilities 140 (1)
5.6 Worm Life 140 (3)
5.6.1 Random Scanning 141 (1)
5.6.2 Random Scanning Using Lists 142 (1)
5.6.3 Island Hopping 142 (1)
5.6.4 Directed Attacking 142 (1)
5.6.5 Hit-List Scanning 143 (1)
5.7 Polymorphic Worms: Definition and Anatomy 143 (6)
5.7.1 Polymorphic Worm Definition 143 (1)
5.7.2 Polymorphic Worm Structure 143 (1)
5.7.3 Invariant Bytes 144 (1)
5.7.4 Polymorphic Worm Techniques 144 (4)
5.7.5 Signature Classes for Polymorphic 148 (1)
Worms
5.8 Internet Worm Prevention Methods 149 (3)
5.8.1 Prevention of Vulnerabilities 149 (2)
5.8.2 Prevention of Exploits 151 (1)
5.9 Conclusion 152 (1)
References 153 (6)
Chapter 6 Reading Resources On Automated 159 (10)
Signature Generation Systems
6.1 Introduction 159 (6)
6.1.1 Hybrid System (Network Based and Host 160 (1)
Based)
6.1.2 Network-Based Mechanisms 161 (3)
6.1.3 Host-Based Mechanisms 164 (1)
References 165 (4)
Chapter 7 Signature Generation Algorithms For 169 (92)
Polymorphic Worms
7.1 String Matching 169 (16)
7.1.1 Exact String-Matching Algorithms 170 (9)
7.1.2 Approximate String-Matching Algorithms 179 (6)
7.2 Machine Learning 185 (27)
7.2.1 Supervised Learning 185 (4)
7.2.2 Algorithm Selection 189 (2)
7.2.3 Logic-Based Algorithms 191 (4)
7.2.4 Learning Set of Rules 195 (7)
7.2.5 Statistical Learning Algorithms 202 (6)
7.2.6 Support Vector Machines 208 (4)
7.3 Unsupervised Learning 212 (42)
7.3.1 A Brief Introduction to Unsupervised 212 (7)
Learning
7.3.2 Dimensionality Reduction and 219 (5)
Clustering Models
7.3.3 Expectation-Maximization Algorithm 224 (3)
7.3.4 Modeling Time Series and Other 227 (7)
Structured Data
7.3.5 Nonlinear, Factorial, and 234 (1)
Hierarchical Models
7.3.6 Intractability 235 (1)
7.3.7 Graphical Models 236 (5)
7.3.8 Exact Inference in Graphs 241 (7)
7.3.9 Learning in Graphical Models 248 (4)
7.3.10 Bayesian Model Comparison and 252 (2)
Occam's Razor
7.4 Concluding Remark 254 (1)
References 254 (7)
Chapter 8 Zero-Day Polymorphic Worm Collection 261 (16)
Method
8.1 Introduction 261 (1)
8.2 Motivation for the Double-Honeynet System 261 (1)
8.3 Double-Honeynet Architecture 262 (2)
8.4 Software 264 (2)
8.4.1 Honeywall Roo CD-ROM 264 (1)
8.4.2 Sebek 265 (1)
8.4.3 Snort_inline 265 (1)
8.5 Double-Honeynet System Configurations 266 (8)
8.5.1 Implementation of Double-Honeynet 266 (1)
Architecture
8.5.2 Double-Honeynet Configurations 267 (7)
8.6 Chapter Summary 274 (1)
References 274 (3)
Chapter 9 Developed Signature Generation 277 (20)
Algorithms
9.1 Introduction 277 (1)
9.2 An Overview and Motivation for Using 278 (1)
String Matching
9.3 The Knuth-Morris-Pratt Algorithm 279 (4)
9.3.1 Proposed Substring Extraction 280 (2)
Algorithm
9.3.2 A Modified Knuth-Morris-Pratt 282 (1)
Algorithm
9.3.3 Testing the Quality of the Generated 282 (1)
Signature for Polymorphic Worm A
9.4 Modified Principal Component Analysis 283 (4)
9.4.1 An Overview of and Motivation for 283 (1)
Using PCA in Our Work
9.4.2 Our Contributions in the PCA 283 (1)
9.4.3 Determination of Frequency Counts 284 (1)
9.4.4 Using PCA to Determine the Most 284 (3)
Significant Data for Polymorphic Worm
Instances
9.4.5 Testing the Quality of the Generated 287 (1)
Signature for Polymorphic Worm A
9.5 Clustering Method for Different Types of 287 (1)
Polymorphic Worms
9.6 Signature Generation Algorithm Pseudocodes 288 (7)
9.6.1 Signature Generation Process 288 (6)
9.6.2 Testing the Quality of the Generated 294 (1)
Signature for Polymorphic Worm A
9.7 Chapter Summary 295 (1)
9.8 Conclusion and Recommendations for Future 295 (1)
Work
References 296 (1)
Index 297