Nmap Network Scanning Guide Gordon Lyon
User Manual:
Open the PDF directly: View PDF .
Page Count: 465
Download | ![]() |
Open PDF In Browser | View PDF |
Nmap Network Scanning Official Nmap Project G u ide to Network Discovery and Security Sca n n i ng Gordon "Fyodor" Lyon From port scanning basics for novices to the type of packet crafting used by advanced hackers, this book by Nmap's author and maintainer suits all levels of security and networking professionals. Rather than simply document what every Nmap option does, Nmap Network Scanning demonstrates how these features can be applied to solve real world tasks such as penetration testing, taking network inventory, detecting rogue wireless access points or open proxies, quashing network worm and virus outbreaks, and much more. Examples and diagrams show actual communication on the wire. This book is essential for anyone who needs to get the most out of Nmap, particularly security auditors and systems or network administrators. Nmap Network Scanning : Official Nmap Project Gu ide to Network Discovery and Security Scanning by Gordon "Fyodor" Lyon Book URL: http://nmap.org!bookl ISBN-13: 9 7 8 -0-9 7 9 9 5 8 7 - 1 - 7 ISBN- 10: 0 - 9 7 9 9 5 8 7 - 1 - 7 Library of Congress Control Number (LCCN): 2 00 8 9 4 0 5 8 2 Library Of Congress Subject Headings: I . Computer networks--Security measures 2. Computer security Published by Insecure.Com LLC. For information on bulk purchases, special sales, rights, book distributors, or translations, please contact us directly: Insecure.Com LLC 370 Altair Way # 1 1 3 Sunnyvale, CA 94086-6161 United States Email: sales @insecure.com; Phone: +1 -650-989-4206; Fax: +l-650-989-4206 Revision History: First Edition: December 2008 Defcon Pre-Release: August 2008 Zero-Day Release: May 2008 Copyright © 2008 by Insecure.Com LLC. All rights reserved. Except where noted otherwise in this work, no part may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner. Nmap is a registered trademark of lnsecure.Com LLC. Other product and company names mentioned herein may be the trademarks of their respective owners. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals. The authors and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein. Table of Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi I. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi 2. Intended Audience and Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi 3. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxii 4. Other Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii 5. Request for Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx iv 6. 1 . Technology Used to Create This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv 7. TCP/IP Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi I . Getting Started with Nmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . l J . l . Introduction . . . 1 1 .2. Nmap Overview and Demonstration . . . . . l 1 .2. 1 . Avatar Online . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . l 1 .2.2. Saving the Human Race . 8 1 .2.3. MadHat in Wonderland . . . 9 1 .3. The Phases of an Nmap Scan . . 12 1 .4. Legal Issues . . 13 1 .4 . 1 . Is Unauthorized Port Scanning a Crime? . . 14 1 .4.2. Can Port Scanning Crash the Target Computer/Networks? . . . 19 1 .4.3. Nmap Copyright . . . . , . 20 1.5. The History and Future of Nmap . . . . . 20 2. Obtaining, Compiling, Installing, and Removing Nmap 25 2.1. Introduction . 25 . . . . 25 2. 1 . 1 . Testing Whether Nmap is Already Installed . 2.1 .2. Command-line and Graphical Interfaces . . . 25 2.1 .3. Downloading Nmap . . . 26 2.1 .4. Verifying the Integrity of Nmap Downloads . 26 2.1 .5. Obtaining Nmap from the Subversion (SYN) Repository 28 2.2. Unix Compilation and Installation from Source Code 29 2.2. 1 . Configure Directives . . . . 30 2.2.2. If You Encounter Compilation Problems . 32 2.3. Linux Distributions . . 33 2.3.1. RPM-based Distributions (Red Hat, Mandrake, SUSE, Fedora) . . 33 2.3.2. Updating Red Hat, Fedora, Mandrake, and Yel low Dog Linux with Yum 34 2.3.3. Debian Linux and Derivatives such as Ubuntu . . . 35 2.3.4. Other Linux Distributions . . . . 35 2.4. Windows 36 2.4. 1 . Windows 2000 Dependencies . . . . . 37 2.4.2. Wi ndows Self-installer . . . . 37 2.4.3. Command-line Zip Binaries 37 37 Installing the Nmap zip binaries 2.4.4. Compile from Source Code . . . . 38 2.4.5. Executing Nmap on Windows . . . . . . 39 2.5. Sun Solaris . . . . . . 40 2.6. Apple Mac OS X . . . . 41 2.6. 1 . Executable Installer . 41 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 2.6.2. Compile from Source Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Compile Nmap from source code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Compile Zen map from source code . 2.6.3. Third-party Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.4. Executing Nmap on Mac OS X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7. FreeBSD I OpenBSD I NetBSD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. 7. 1 . OpenBSD Binary Packages and Source Ports Instructions 2.7.2. FreeBSD Binary Package and Source Ports Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installation of the binary package . . Installation using the source ports tree . 2.7.3. NetBSD Binary Package Instructions . 2.8. Amiga, HP-UX, IRIX, and Other Platforms 2.9. Removing Nmap . 41 41 42 42 42 43 43 44 44 44 44 44 45 3. Host Discovery (Ping Scanning) 47 3 . 1 . Introduction 47 3 .2. Specifying Target Hosts and Networks . 47 3.2. 1 . Input From List (-iL) 48 3.2.2. Choose Targets at Random (-iR) 48 3.2.3. Excluding Targets (--exclude, --excludefile ) 48 3.2.4. Practical Examples . 49 3.3. Finding an Organization's IP Addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.3. 1 . DNS Tricks 50 3.3.2. Whois Queries Against IP Registries 54 3.3.3. I nternet Routing Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.4. DNS Resolution 56 3.5. Host Discovery Controls . 57 3.5. 1 . List Scan (-sL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 3.5.2. Ping Scan (-sP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 3.5.3. Disable Ping (-PN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 3 .6. Host Discovery Techniques 60 3.6. 1 . TCP SYN Ping (-PS ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1 3.6.2. TCP ACK Ping (-PA ) 62 3.6.3. UDP Ping (-PU ) . . 63 3.6.4. ICMP Ping Types (-PE, -PP, and -PM) 64 3 .6.5. IP Protocol Ping (-PO ) 64 3.6.6. ARP Scan (-PR) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.6.7. Default Combination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 3.7. Putting It All Together: Host Discovery Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 3.7. 1 . Related Options 66 3.7.2. Choosing and Combining Ping Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 TCP probe and port selection . 68 UDP port selection . 70 ICMP probe selection 70 Designing the ideal combinations of probes 70 3.8. Host Discovery Code Algorithms 72 4. Port Scanning Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 4. 1 . Introduction to Port Scanning 73 4. 1 . 1 . What Exactly is a Port? 73 4.1 .2. What Are the Most Popular Ports? 75 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV 4.1 .3. What is Port Scanning? 4.1 .4. Why Scan Ports? . . . 77 78 4.2. A Quick Port Scanning Tutorial . . . . 79 4.3. Command-line Flags . . . . . . 82 4.3. 1 . Selecting Scan Techniques . . . . . . . 82 4.3.2. Selecting Ports to Scan . . . 83 4.3.3. Timing-related Options . . . . 85 4.3.4. Output Format and Verbosity Options . . . . . . . . . 85 4.3.5. Firewall and IDS Evasion Options . . 87 4.3.6. Specifying Targets . . . . 87 4.3.7. Miscellaneous Options . . . . . . . 87 4.4. 1Pv6 Scanning (-6) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 4.5. SOLUTION: Scan a Large Network for a Certain Open TCP Port . . 88 4.5.1. Problem . . . . . . . . 88 4.5.2. Solution . . . . . . . . 89 4.5.3. Discussion . . . 89 4.5.4. See Also . . . . . 94 5. Port Scanning Techniques and Algorithms . . 95 5 . 1 . Introduction 95 5.2. TCP SYN (Stealth) Scan (-sS) . 96 5.3. TCP Connect Scan (-sT) . . . JOO . 101 5.4. UDP Scan (-sU) . . . 5.4. 1 . Disambiguating Open from Filtered UDP Ports 1 02 5.4.2. Speeding Up UDP Scans . . . . I 05 5.5. TCP FIN, NULL, and Xmas Scans (-sF, -sN, -sX) . 1 07 5.6. Custom Scan Types with --scanftags . . 111 5.6. 1 . Custom SYN/FIN Scan 111 5.6.2. PSH Scan . . . . . . . 1 12 5.7. TCP ACK Scan (-sA) . . 1 13 5.8. TCP Window Scan (-sW) . .. . . . . . 1 15 5.9. TCP Maimon Scan (-sM) . . . . . . 1 16 5. 10. TCP Idle Scan (-sl) . . .:................................................................... 1 17 5 .10. 1 . Idle Scan Step by Step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 8 5. 10.2. Finding a Working Idle Scan Zombie Host . . . . . . . 1 20 . 121 5.10.3. Executing an Idle Scan . . . . . . . 5. 10.4. Idle Scan Implementation Algorithms . . . . . . 1 22 5. 1 1 . IP Protocol Scan (-sO) . . 1 25 5.12. TCP FTP Bounce Scan (-b) . . . . 1 27 5.13. Scan Code and Algorithms . . . . . . . 1 28 5 . 1 3 . 1 . Network Condition Monitoring . 1 29 5 . 1 3.2. Host and Port Parallelization . 1 29 5 . 1 3.3. Round Trip Time Estimation . . 1 30 5 . 1 3.4. Congestion Control . 1 30 5 . 1 3.5. Timing probes · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·-· · · · · · · · · · · · · · · · · · 1 32 5 . 1 3.6. Inferred Neighbor Times . . . . . 1 32 5 . 1 3.7. Adaptive Retransmission . . . 1 32 5 . 1 3.8. Scan Delay . . . . 1 32 6. Optimizing Nmap Performance . . 1 35 6.1. Introduction . 1 35 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v 6.2. Scan Time Reduction Techniques 6.2. 1 . Omit Non-critical Tests 6.2.2. Optimize Timing Parameters 6.2.3. Separate and Optimize UDP Scans 6.2.4. Upgrade Nmap . . 6.2.5. Execute Concurrent Nmap Instances 6.2.6. Scan From a Favorable Network Location 6.2.7. Increase Available Bandwidth and CPU Time 6.3. Coping Strategies for Long Scans . . . 6.3 . 1 . Use a Multi-stage Approach . 6.3.2. Estimate and Plan for Scan Time 6.4. Port Selection Data and Strategies . . 6.5. Low-Level Timing Controls . . . . . . 6.6. Timing Templates (-T) . . . 6.7. Scanning 676,352 IP Addresses in 46 Hours . . . .. . . . . . . . . . . . . . . . . .. . . . . . 1 35 1 36 1 37 1 37 . . 1 37 1 38 . 1 38 . . . . . . 1 38 . . . . . 1 39 1 39 . 1 40 . . . . 1 40 . 141 1 42 . . 1 43 7. Service and Application Version Detection . . . . . 1 45 7. 1 . Introduction 1 45 7.2. Usage and Examples . . . . . . . 1 47 7.3. Technique Described . . . . . . . . . . . . . . . 1 49 7.3 . 1 . Cheats and Fallbacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 5 1 7.3.2. Probe Selection and Rarity . . 1 52 7.4. Technique Demonstrated 1 52 7.5 . Post-processors . . . . . 1 55 7.5 . 1 . Nmap Scripting Engine Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 55 . . . 1 56 7.5.2. RPC Grinding 7.5.3. SSL Post-processor Notes . . . 1 57 7.6. nmap-service-probes File Format 1 58 7.6. 1 . Exclude Directive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 58 7.6.2. Probe Directive 1 59 7.6.3. match Directive . . . . . 1 59 7.6.4. softmatch Directive . . . . . 161 7.6.5. ports and sslports Directives . . 1 62 7.6.6. totalwaitms Directive . . . 1 62 7.6.7. rarity Directive . . . . . . 162 . . . . . 163 7.6.8. fallback Directive . . . . . . . 7.6.9. Putting It All Together . . . . . . . . . . . .. . 1 63 7.7. Community Contributions . . . . . . . 1 64 7.7. 1 . Submit Service Fingerprints . . . . . . . 1 64 7.7.2. Submit Database Corrections . 164 7.7.3. Submit New Probes . . . . 1 65 7.8. SOLUTION: Find All Servers Running an Insecure or Nonstandard Application Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 66 7.8 . 1 . Problem . . . . . . . 1 66 7.8.2. Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 66 7.8.3. Discussion . . .. . . . . . . 167 7.9. SOLUTION: Hack Version Detection to Suit Custom Needs, such as Open Proxy Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 68 7.9 . 1 . Problem . .. . 168 7.9.2. Solution . . 1 69 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ·. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.9.3. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 69 8. Remote OS Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 7 1 8.1. Introduction 171 8 . 1 . 1 . Reasons for OS Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 7 1 Determining vulnerability of target hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 7 1 Tailoring exploits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 7 1 Network i nventory and support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 72 Detecting unauthorized and dangerous devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 72 Social engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 72 8.2. Usage and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 72 8.3. TCP/IP Fingerprinting Methods Supported by Nmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 76 8.3.1. Probes Sent I 77 Sequence generation (SEQ, OPS, WIN, and T l ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 77 ICMP echo (IE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 7 8 TCP explicit congestion notification (ECN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 79 TCP (T2-T7) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 79 UDP (U I ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 79 8.3.2. Response Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 80 TCP ISN greatest common divisor (GCD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 80 TCP ISN counter rate (ISR) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 80 TCP ISN sequence predictability index (SP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 80 TCP IP ID sequence generation algorithm (Tl) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 8 1 ICMP IP ID sequence generation algorithm (II) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 8 1 Shared IP ID sequence Boolean (SS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 82 TCP timestamp option algorithm (TS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 82 TCP options (0, 01 -06) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 83 TCP i nitial window size (W, W I -W6) . . . 1 83 Responsiveness (R) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 84 IP don't fragment bit (DF) I 84 Don't fragment (ICMP) (DFI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 84 IP initial time-to-live (T) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 84 IP initial time-to-live guess (TG) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 85 Explicit congestion notification (CC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 85 TCP miscellaneous quirks (Q) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 85 TCP sequence number (S) . . . . . I 86 ICMP sequence number(SI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 86 TCP acknowledgment number (A) . . . . . 1 86 TCP flags (F) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 87 TCP RST data checksum (RD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 87 IP type of service (TOS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 87 IP type of service for ICMP responses (TOSI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 87 IP total length (IPL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 88 Unused port unreachable field nonzero (UN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 88 Returned probe IP total length value (RIPL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 88 Returned probe IP ID value (RID) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 88 Integrity of returned probe IP checksum value (RIPCK) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 88 Integrity of returned probe UDP length and checksum (RUL and RUCK) . . . . . . . . . . . . . . 1 88 Integrity of returned UDP data (RUD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 88 ICMP response code (CD) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 89 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vil IP data length for ICMP responses (DLI) . . . . 8.4. Fingerprinting Methods Avoided by Nmap . . . . 8.4. 1 . Passive Fingerprinting . ... . . . 8.4.2. Exploit Chronology . . . . 8.4.3. Retransmission Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.4. IP Fragmentation . . . . . 8.4.5. Open Port Patterns . . . . . . . . . . . . .. . . . . . . 8.5. Understanding a n Nmap Fingerprint . . . . 8.5. 1 . Decoding the Subject Fingerprint Format . . .. .. . . .. . .. . . Decoding the SCAN line of a subject fingerpri nt . . . .. . 8.5.2. Decoding the Reference Fingerprint Format . . . . . . . . . . . . Free-form OS description (Fingerprint line) . . .. . . . .. . . . . Device and OS classification (Class lines) . . . . .. . . . .. . . Test expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6. OS Matching Algorithms . .. . . . . . .. . . . 8.7. Dealing with Misidentified and Unidentified Hosts . .. . . . . . . . . . . 8.7. 1 . When Nmap Guesses Wrong . . . . . 8.7.2. When Nmap Fails to Find a Match and Prints a Fingerprint . . . . 8.7.3. Modifying the nmap-os-db Database Yourself . . . . . . . . . . 8.8. SOLUTION: Detect Rogue Wireless Access Points on an Enterprise Network . 8.8. 1 . Problem . .. . .. . . . . . . . . . . . . . . . 8.8.2. Solution . ... .... . . ....... . . . . . . . . . . . . . 8.8.3. WAP Characteristics . .. .... ... . .. ... . . . . . . 9. Nmap Scripting Engine . . . . . . . . . . . .. . 9.1. Introduction . . . .. .. . . . .... . .. .. . . . . . . . . . 9.2. Usage and Examples . . . . . . ........ . . ... . . . . . . . . . . . 9.2. 1 . Script Categories . . . . . . . . 9.2.2. Command-line Arguments . . . . . 9.2.3. Arguments to Scripts . . . . . . . . . .. . . 9.2.4. Usage Examples . . . . . . . .. . .. . . . . . . . 9.3. Script Format . . . . . . . . . . . . . . . . . . . 9.3 . l . description Field . . . . . . . . . . . . . .. . 9.3.2. categories Field . . ...... . . . . . . . 9.3.3. author Field . . . . . . . . . . . . . . . . .. . 9.3.4. license Field . . . . . . . . . . . . . . . . . . . . . . . 9.3.5. runlevel Field . . . . . . . . . . . . . . . .. .. . . 9.3.6. Port and Host Rules . . . . . . . ....... ... 9.3.7. Action . . . . . . . .. .. . . . 9.4. Script Language . .. . .. .. . .. . . 9.4. 1 . Lua Base Language . . .. .. .... . . . 9.5. NSE Scripts . . . . . . . . . . . . 9.6. NSE Libraries . . . . . . . . . . ... . . . 9.6. 1 . List of All Libraries . . . .. . . . 9.6.2. Adding C Modules to Nselib . . . . . . . . . .. . . 9.7. Nmap API 9.7. 1 . Information Passed to a Script . . . . . ... . . . . 9.7.2. Network 110 API . . . . . . . . . . . . . . Connect-style network 1/0 . . . . . . .. . . . . . . . . . Raw packet network 1/0 . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 89 1 89 1 89 1 90 1 90 191 191 191 1 92 1 93 1 94 1 95 1 96 1 97 1 98 1 99 200 20 1 202 202 202 202 203 205 205 206 207 209 210 210 21 1 21 1 21 1 21 1 21 1 21 1 212 212 212 212 213 236 236 237 239 239 24 1 24 1 242 9.7.3. Thread Mutexes . . . . . . . . . . . . . . . . . . . . . . . . 243 9.7.4. Exception Handling . . . . . . . . . . . . . . . . . . . . . . 244 9.7.5. The Registry . . . . . . . . ... . . . . . . . . 245 9.8. Script Writing Tutorial . . . . . . . . . . . . . . 245 9.8. 1 . The Head . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 9.8.2. The Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 9.8.3. The Mechanism . . ....... ... .... . ..... 247 9.9. Writing Script Documentation (NSEDoc) . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 9.9. 1 . NSE Documentation Tags . . . . . . . . . ..... . ..... ....... . . . . . . . . . . . . . 250 9.10. Version Detection Using NSE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 1 9.1 I. Example Script: finger.nse . . . ... .. .. ........... ... ........ .. . . . . . . . . . . . . . 253 9.12. Implementation Details . . . . . . . . . . . . . . ... . . . . . . . 254 9.12. 1. Initialization Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254 9.1 2.2. Matching Scripts with Targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...... .. 255 255 9.12.3. Script Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10. Detecting and Subverting Firewalls and Intrusion Detection Systems . 257 JO. I . Introduction 257 10.2. Why Would Ethical Professionals (White-hats) Ever Do This? . . . 257 10.3. Determining Firewall Rules . . . . 258 10.3.1. Standard SYN Scan . . 258 Sneaky firewalls that return RST 259 10.3.2. ACK Scan 260 10.3.3. IP ID Tricks . : 262 10.3.4. UDP Version Scanning . . .. . . . . 264 10.4. Bypassing Firewall Rules 265 10.4.1. Exotic Scan Flags . . 265 10.4.2. Source Port Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 10.4.3. 1Pv6 Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 10.4.4. IP ID Idle Scanning . . . . . ... . . . . .. ... . . ... . . . 269 10.4.5. Multiple Ping Probes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 269 10.4.6. Fragmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.7. Proxies . .. .... . ...... . ... .... .. . ... . . 270 10.4.8. MAC Address Spoofing . . . . . . ... . . . . . . . . . . . 270 27 1 10.4.9. Source Routing . . . . . . . . . ... ...... . . . .. . .. ... . . 10.4.10. FTP Bounce Scan . . . . . . ... .. . . .......... . . . . 272 10.4.1 1 . Take an Alternative Path . . .......... . . . . . . . . . 272 10.4.12. A Practical Real-life Example of Firewall Subversion . . . . . . . . . . . . 272 : . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 10.5. Subverting Intrusion Detection Systems ... .. 10.5.1. Intrusion Detection System Detection . . . . . .. . . 276 Reverse probes . . . ..... . ........ . . . . . . . 276 Sudden firewall changes and suspicious packets . . .. . . . ... . . . 277 Naming conventions . . .... . .... . . . . . . ... 277 Unexplained TIL jumps . . . . . . . . ... .. . .. .... . .. . 278 10.5.2. Avoiding Intrusion Detection Systems . . . ...... . . . .... . . 279 Slow down . . . . . .... . . .. . .. ... . . . . . . 280 Scatter probes across networks rather than scanning hosts consecutively . . . . . 282 Fragment packets . ..... ..... . . . . . .. . . . 282 .. . . . . .. . . . . . . .. . 283 E vade specific rules Avoid easily detected Nmap features . . ... . .. . . . . . . . . 284 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix 10.5.3. Misleading Intrusion Detection Systems . . . .. . 284 Decoys . .. . 284 Port scan spoofing . .. . . . . . . 286 Idle scan . . . . . 286 DNS proxying . . . . . .. . . . . 286 10.5.4. DoS Attacks Against Reactive Systems . . . 287 10.5.5. Exploiting Intrusion Detection Systems . .. .. . . 288 10.5 .6. Ignoring Intrusion Detection Systems . 288 10.6. Detecting Packet Forgery by Firewall and Intrusion Detection Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 10.6. 1 . Look for TIL Consistency . . . . 289 10.6.2. Look for IP ID and Sequence Number Consistency . . 290 . . . 29 1 10.6.3. The Bogus TCP Checksum Trick . . . 10.6.4. Round Trip Times . . . . . . 292 10.6.5. Close Analysis of Packet Headers and Contents . . 293 10.6.6. Unusual Network Uniformity . . . . . . . 293 1 1 . Defenses Against Nmap .. . . . . . . . . . . 295 I I . I . Introduction . . . . . . 295 1 1 .2. Scan Proactively, Then Close or Block Ports and Fix Vulnerabilities .. . . . 295 1 1 .3. Block and Slow Nmap with Firewalls . . .. 296 1 1 .4. Detect Nmap Scans .. . . . . . . . . . . 297 . . . 298 1 1 .5. Clever Trickery . 1 1 .5. 1 . Hiding Services on Obscure Ports . . . 299 1 1 .5 .2. Port Knocking . . . . . . . . . 300 1 1 .5.3. Honeypots and Honeynets . . . . . 30 I 1 1 .5.4. OS Spoofing . . . . . . . . 302 1 1 .5.5. Tar Pits . . . . . . . . . . . 303 1 1 .5.6. Reactive Port Scan Detection . . . . . . . . . . . 304 . . 304 1 1 .5.7. Escalating Arms Race . . . . . . . 12. Zenmap GUI Users' Guide . . . . . . 307 12.1. Introduction . . . . . . . . . ... . 307 12.1 . l . The Purpose of a Graphical Frontend for Nmap . . . . 307 1 2.2. Scanning 308 1 2.2. 1 . Profiles . .. . . . . . . . 309 1 2.2.2. Scan Aggregation . . . . 309 12.3. Interpreting Scan Results . . . 31 1 1 2.3. 1 . Scan Results Tabs . . . . 31 1 The Nmap Output tab . . . . 312 The Ports I Hosts tab . . . . . . . .. 3 1 2 The Topology tab .. . . . . . 313 The Host Details tab . . . . . 314 The Scans tab . . . . . . . 315 1 2.3.2. Sorting by Host . . . . . . 315 1 2.3.3. Sorting by Service . . . . . . . . . . 316 1 2.4. Saving and Loading Scan Results . . 316 1 2.4. 1 . The Recent Scans Database . . . . . 317 1 2.5. Surfing the Network Topology . . . . . . . . . . 317 1 2.5. 1 . An Overview of the Topology Tab . . . . . . 318 1 2.5.2. Legend .. . . . . . . .. 3 1 8 1 2.5.3. Controls . . .. . . . .. . . . 319 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Action controls . .. . ... . . . .. .. . 319 Interpolation controls . . . . 320 Layout controls . ...... .... . ..... . . . . . . . . . . . . . . . . . 320 View controls . . .. .. . .. . . . . . . . 321 Fisheye controls .. .. . . . . . . . 32 1 1 2.5.4. Keyboard Shortcuts . . . . 322 1 2.5.5. The Hosts Viewer . . . 322 1 2.6. The Nmap Command Constructor Wizard . . . . . 322 1 2.7. The Profile Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 1 2.7. 1 . Creating a New Profile . . . . . . . . . . . . . . . . 324 12.7.2. Editing a Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324 12.7.3. Deriving a New Profile from an Old One . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325 12.8. Searching Saved Results . ......... . .. . . . . . . . . . . . . 325 12.9. Comparing Results . . . . . . . . . . . . . . . . . . . .. . . ..... .. . . 328 1 2.9. 1 . Graphical Comparison . . . . . . . . . . . .. . . . . . . . 329 1 2.9.2. Text Comparison . . . . . . .. . 329 12.1 0. Files Used by Zenmap . . . . . ... ... .. .. . 330 12.10. 1 . The nmap Executable 330 12. 10.2. System Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 1 12. 10.3. Per-user Configuration Files . . . . 332 12. 10.4. Output Files . 332 12.l l. Description of zenmap.conf . 333 1 2.1 1 . 1 . Sections of zenmap.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333 12.12. Command-line Options . . . 335 12.12. l . Synopsis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 1 2. 1 2.2. Options Summary . . 335 1 2.1 2.3. Error Output . . . . . . 336 1 2. 1 3. History 336 13. Nmap Output Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 1 3 . l . Introduction . . . . . . . .. . . . .. . . . . . 337 1 3.2. Command-line Flags . . . 338 13.2. l . Controlling Output Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338 13.2.2. Controlling Verbosity of Output . . . . . . . . 339 13.2.3. Enabling Debugging Output . . . . .. . . . . . . . 343 13 .2.4. Handling Error and Warning Messages . . . . . . . . . . . . . . . . . . . . 344 13 .2.5. Enabling Packet Tracing . . . . . . . . . . . . 345 13.2.6. Resuming Aborted Scans . . . . . . . . . . . . 346 13.3. Interactive Output . . . . ... .. . . . . . . . . 346 13.4. Normal Output (-oN) . . ...... ... . . . . . 346 . . .. . . 347 1 3.5. $crlpT klddl3 OuTPut (-oS) . 1 3.6. XML Output (-oX) ....... . . .. .... ... 348 13.6. 1 . Using XML Output . . . ... . .. . . 350 13.7. Manipulating XML Output with Perl . . . .. .. . 352 13.8. Output to a Database .. . . . .. .... 354 13.9. Creating HTML Reports . . . . . . . . . . . . . . . . . 355 1 3.9. 1 . Saving a Permanent HTML Report . . . . . . . . 355 13. 10. Grepable Output (-oG) . .. . .... . . . . . . . . 356 13.10.1. Grepable Output Fields . . . . . . . . . . . . . . . . . 357 Host field . . . . . .. . . . . . . . . . . . . . . . . . . . 357 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Ports field 357 Protocols field . 359 Ignored State field 359 OS field . . . . 359 Seq Index field 360 IP ID Seq field 360 Status field 360 13. 10.2. Parsing Grepable Output on the Command Line 36 1 14. Understanding and Customizing Nmap Data Files 363 14.1. Introduction . 363 14.2. Well Known Port List: nmap-services . 363 14.3. Version Scanning DB: nmap-service-probes 365 14.4. SunRPC Numbers: nmap-rpc . . 366 14.5. Nmap OS Detection DB: nmap-os-db . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366 14.6. MAC Address Vendor Prefixes: nmap-mac-prefixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368 14.7. IP Protocol Number List: nmap-protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 14.8. Files Related to Scripting 369 14.9. Using Customized Data Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370 1 5 . Nmap Reference Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 1 5 . 1 . Description 373 1 5.2. Options Summary 374 1 5.3. Target Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376 1 5.4. Host Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378 15.5. Port Scanning Basics 383 1 5.6. Port Scanning Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 1 5.7. Port Specification and Scan Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389 1 5.8. Service and Version Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390 1 5.9. OS Detection . 392 1 5 . 10. Nmap Scripting Engine (NSE) 393 1 5. 1 1 . Timing and Performance 394 1 5. 1 2 . Firewall/IDS Evasion and Spoofing 399 1 5 . 1 3 . Output 403 1 5.14. Miscellaneous Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408 15.15. Runtime Interaction 410 1 5.16. Examples 410 15.17. Bugs 41 1 1 5.18. Author . 41 1 1 5. 1 9. Legal Notices 412 1 5.19. 1 . Nmap Copyright and Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1 2 1 5. 19.2. Creative Commons License for this Nmap Guide 413 15. 19.3. Source Code Availability and Community Contributions 413 1 5. 19.4. No Warranty 413 1 5 . 19.5. Inappropriate Usage . 414 1 5. 19.6. Third-Party Software 414 . 414 1 5.19.7. United States Export Control Classification A . Nmap XML Output DTD 415 A. I . Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1 5 A.2. The Full DTD 415 Index 423 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . · . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii List of Fig u res I . 1Pv4 header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx vii 2. TCP header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx viii 3. UDP header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx viii 4. ICMP header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx ix I . I. Trinity begins her assault . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.2. Trinity scans the Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1 .3. Strong opinions on port scanning legality and morality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 4 2. 1 . Executing Nmap from a Windows command shell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.1. A business card explains everything . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.2. Netcraft finds 36 Target web servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 5.1. ICMPv4 destination unreachable header layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 5.2. SYN scan of open port 22 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.3. SYN scan of closed port 1 1 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.4. SYN scan of filtered port 1 39 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.5. Connect scan of open port 22· (nmap -sT -p22 scanme.nmap.org) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 00 5.6. Idle scan of an open port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 9 5.7. Idle scan of a closed port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 9 5.8. Idle scan of a filtered port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 9 5.9. Congestion window and threshold . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 3 1 5.10. Scan rate as affected by scan delay 1 33 8. 1 . ICMP echo request or reply header layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 77 8.2. ICMP destination unreachable header layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 77 10. l . BlackICE discovers an unusual intruder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 10.2. An attacker masked by dozens of decoys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 12.1. Typical Zenmap screen shot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 12.2. Zenmap's main window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308 12.3. Target and profile selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 12.4. Host selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 5 12.5. OS icons 316 12.6. Service selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 6 12.7. Grouping a host's children . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 9 12.8. Highlighting regions of the topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320 12.9. Choosing a profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 12.10. The profile editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324 12.1 1. The search dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325 12.12. Keyword search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326 12.13. Expressions search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326 12.14. Comparison tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 12.15. Graphical comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 12.16. Text mode comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330 13.1. XML output in a web browser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 1 · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ' · . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xlll !.-. List of Tables I . Formatting style conven�ions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx iii 3.1. First pass at listing target.com IPs . . .. . . . . . . . . . . . . .. .... .. 51 3.2. Most valuable TCP probe ports, i n descending order of accessibility . . . . . . . . . . . . 68 5.1. ICMP destination unreachable (type 3) code values . 96 5.2. How Nmap interprets responses to a SYN probe . 98 5.3. How Nmap interprets responses to a UDP probe . . . . . ... . . 101 5.4. How Nmap interprets responses to a NULL, FIN, or Xmas scan probe 1 07 5.5. How Nmap interprets responses to an ACK scan probe 1 13 5.6. How Nmap interprets responses to a Window scan ACK probe 1 15 5.7. How Nmap interprets responses to a Maimon scan probe 1 17 5.8. How Nmap interprets responses to an IP protocol probe . . . 1 26 6. 1. Required --top-ports values for reaching various effectiveness levels . 141 6.2. Low-level timing controls by function . 1 42 6.3. Timing templates and their effects . . .. . . . 1 42 7. 1 . versioninfo field formats and values . .. . . 1 60 8. 1. 0 test values 1 83 8.2. DFI test values . . . . . . . 1 84 8.3. CC test values . . .. . .... 1 85 8.4. S test values . . . .. . . . .. . . . . 1 86 8.5. SI test values .. . .. .. . . . . . . . 1 86 8.6. A test values . . . . . . . . ..... . . . . i 86 8.7. F test values . . . . . . . . 1 87 8.8. TOSI test values . ... . . 1 87 8.9. CD test values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . .. . 1 89 8.10. DLI test values . . . .. .. . . . . . . . . 1 89 8.1 1. Reference fingerprint test expression operators . ... .. . . 1 98 9.1 . port. version values . . . . . . 240 12.1 . Text di ff character codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . : . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330 . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv '' .i" , '• ..!'..�.- List of Examples A typical Nmap scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii Nmap list scan against Avatar Online IP addresses . . . .. . . . . . . . . . . . . . 3 1.2. Nmap results against an AO firewall . . . . . . . . ..... . . . . . . . . 5 1.3. Another interesting AO machine . . . . . . . . . . . . . . . . . 7 1.4. nmap-diff typical output . . . . .. . . .. . . . .. . . . . . . 11 1.5. nmap-report execution . . . . . . . . . . . . . . 12 2.1. Checking for Nmap and determining its version number . . . . . . . . 25 2.2. Verifying the Nmap and Fyodor PGP Key Fingerprints . . . . . . . . . 27 2.3. Verifying PGP key fingerprints (Successful) . . . . . . . . . . . 27 2.4. Detecting a bogus file . . . . . . . . . . . . 27 . . . . . . 28 2.5. A typical Nmap release digest file . . .. . .. . . . . . 2.6. Verifying Nmap hashes . . . .. . . . . . . . . . . . . 28 2.7. Successful configuration screen . . . . . . . 30 2.8. Installing Nmap from binary RPMs . . . . . . .. . ... . . . . 33 2.9. Building and installing Nmap from source RPMs . . 34 2.10. Installing Nmap from a system Yum repository . . . . . . . . . . 35 3.1. Using the host command to query common DNS record types . . . . . . . . . . . . . . . . . . . . . . 51 3.2. Zone transfer failure and success . . . . . .. . . . . . . ... . 52 3.3. Nmap reverse-DNS and traceroute scan against www.target.com . . .. . 53 3.4. Using whois to find owner of www.target.com IP address . . 53 3.5. Using whois to fi nd netblock containing 161 .225 . 1 30. 163 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.6. Enumerating hosts surrounding www.stanford.edu with list scan . . . . . . 58 3.7. Discovering hosts surrounding www.lwn.net with a ping scan . . . . . 59 3.8. Attempts to ping popular Internet hosts . . . . . . . . . 61 3.9. Retry host discovery using port 80 SYN probes . . . . . . 62 3.10. Attempted ACK ping against Microsoft . .... ... . . . . . 63 3. 1 1 . Raw IP ping scan of an offtine target . . . . . . . 65 3.12. ARP ping scan of an offtine target . . . . . 65 3.13. Generating 50,000 IP addresses, then ping scanning with default options . . 71 3.14. Repeating ping scan with extra probes . . . . . . . . 71 4.1. Viewing and increasing the ephemeral port range on Linux . . . 74 4.2. Simple scan: nmap scanme.nmap.org . . . . . .. . 80 4.3. More complex: nmap -pO- -v -A -T4 scanme.nmap.org . . 81 4.4. A simple IPv6 scan 88 4.5. Discovering Playboy's IP space . . . . . . 90 4.6. Pinging Playboy's web server for a latency estimate . . . . .. . . . . 90 4.7. Digging through Playboy's DNS records . . . . . . . . . 91 4.8. Pinging the M X servers . . . . . .. . . 92 4.9.' TCP pinging the MX servers . . .. . . . . . . 92 4.10. Launching the scan . . . . . . . . . . . 93 4.1 1 . Egrep for open ports . .. . . . . . . 94 5. 1 . A SYN scan showing three port states . . .. . . . . . 97 5.2. Using --packet-trace to understand a SYN scan .. . . . . 99 5.3. Connect scan example . . . .. . . . . . . . . . 101 5.4. UDP scan example . . . . . . . . . .. .. . . . . . . . . . 1 02 5.5. UDP scan example . . . . . . . . . . . . . . . 1 02 I. I.I. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii 5 .6. Improving Felix's UDP scan results with version detection . . . .. . . . .. .. ... ... . . . 1 03 5.7. Improving Scanme's UDP scan results with version detection . . . . . . . . . . . 1 04 5.8. Attempting to disambiguate UDP ports with TIL discrepancies .. . .. . . . . . 1 05 5.9. Optimizing UDP Scan Time . .. . . .. . . . ... . 1 07 5 . 10. Example FIN and Xmas scans . . . . . . . . . . . . . 1 09 5 . 1 1 . SYN scan of Docsrv .. . . . . . . . . . ... . . 1 09 5 . 1 2 . FIN scan of Docsrv . ... . . . . . .. ... . 1 10 5 . 1 3 . A SYN/FIN scan of Google .. . .. . . . . . . . . .. . . I 12 5. 14. A custom PSH scan . .. . . . . . . . . .. . 1 12 . . .. . . 1 14 5 . 1 5 . A typical ACK Scan . . . ... . . . . . . . 5 . 16. An ACK scan of Docsrv . . . . . ... . . . . .. ... 1 15 5 . 17. Window scan of docsrv.caldera.com . . . . . . . . . . . . . . . . . . . . 1 16 5 . 1 8 . A failed Maimon scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 7 5. 19. An idle scan against the RIAA . . . . .. .. . . . . . . . . .. . .. . . 121 5.20. IP protocol scan of a router and a typical Linux 2.4 box . . ... .. . . . . 1 27 5.2 1 . Attempting an FTP bounce scan . . . . ...... .. . . . . . . . . 1 28 5.22. Successful FTP bounce scan . . . . ..... . . . . . 1 28 6. 1 . Bandwidth usage over local 100 Mbps ethernet network . . . . . . . . 1 39 6.2. Estimating scan time . . . . . . . . . . . . . . 1 40 7. 1 . Simple usage of version detection . . . . . . . . . . .. . . 1 46 7.2. Version detection against www.microsoft.com . . . . . . 1 47 7.3. Complex version detection . . . . . . . . . . . . .. . . . . . . ... . . 1 48 7.4. NULL probe cheat example output . . . . .. .. . . . . . . . 151 7.5. Enumerating RPC services with rpcinfo . . . . . . . . 1 56 7.6. Nmap direct RPC scan . . . . . . . . ... . . . . . . 1 57 7.7. Version scanning through SSL . . . . . . . .. 1 58 8 . 1 . OS detection with verbosity (-0 -v) . . . . . . . 1 73 8.2. Using version scan to detect the OS . . . . . . . . . . . ..... 1 75 8.3. A typical subject fingerprint . . . . . .... . . 1 92 8.4. A cleaned-up subject fingerprint . . . . . . . . . .. . . . 1 93 8.5. A typical reference fingerprint . . . . . 1 95 8.6. Some typical fingerprint descriptions and corresponding classifications . . . 1 97 8. 7. The MatchPoints structure . . . . . . . . . . . . . . 1 99 8.8. Scan results against a consumer WAP . .. . . . . 203 9 . 1 . Typical NSE output . . . . . . . . . . 206 9.2. Connect-style 110 242 9.3. Mutex manipulation . . . . . . . . . . . 244 9.4. Exception handling example . . . . . . . . . 244 9.5. An NSEDoc comment for a function . . . . . . 249 9.6. An NSEDoc comment for a module . . . . . . . 249 9.7. An NSEDoc comment for a script . . . . . . . . ... ... . .. 250 9.8. A typical version detection cript (Skype version 2 detection) . . . . .. .. . ..... .... .. .. . .. 252 10.l. Detection of closed and fil tered TCP ports . . . 259 \0.2 . ACK scan against Scanme . . . . . 260 I0.3. Contrasting SYN and ACK scans against Para . . . . . . . 26 1 1 0.4. UDP scan against firewalled host . . . . . . 264 . . . . . . . . . . 265 10.5. UDP version scan against firewalled host 10.6. FIN scan against stateless firewall . . .. . . . . . . . . . . . . 266 I 0.7. Bypassing Windows IPsec filter using source port 88 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8. Comparing IPv4 and IPv6 scans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268 10.9. Exploiting a printer with the FfP bounce scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 10. 10. Some interesting hosts and networks at Megacorp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 10. 1 1 . Ping scan against the target network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 10.12. Packet trace against a single IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 10.13. Testing an idle scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274 10.14. Testing source routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 10.15. Success at last . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 10.16. Host names can be deceiving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 10. 17. Noting TTL gaps with traceroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 10.18. Using the IP record route option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 10.19. Slow scan to bypass the default Snort 2.2.0 Flow-portscan fixed time scan detection method 28 1 10.20. Default Snort rules referencing Nmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 10.21. Using DNS Proxies (Recursive DNS) for a Stealth List Scan of SecurityFocus . . . . . . . . . . . . . . . . . . . . . . . 287 10.22. Detection of closed and filtered TCP ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290 10.23. Testing IP ID sequence number consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 1 10.24. Finding a firewall with bad TCP checksums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 1 I I . I . An all-TCP-port version scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 1 1 .2. Deceiving Nmap with IP Personality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 1 3 . l . Scanrand output against a local network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 13.2. Grepping for verbosity conditionals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 1 13.3. Interactive output without verbosity enabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342 13.4. Interactive output with verbosity enabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343 13.5. Some representative debugging lines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344 13.6. Using --packet-trace to detail a ping scan of Scanme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 13.7. A typical example of normal output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 1 3.8. A typical example of $crlpt KiDDi3 OutPut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348 13.9. An example of Nmap XML output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 13. 10. Nmap XML port elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350 1 3 . l l . Nmap::Parser sample code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353 13. 12. Nmap:: Scanner sample code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354 13.13. A typical example of grepable output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 13.14. Grepable output for IP protocol scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 13.15. Ping scan grepable output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 1 13. 16. List scan grepable output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 13. 17. Parsing grepable output on the command line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 14. 1 . Excerpt from nmap-services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364 14.2. Excerpt from nmap-service-probes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365 14.3. Excerpt from nmap-rpc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366 14.4. Excerpt from nmap-os-db . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 14.5. Excerpt from nmap-mac-prefixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368 14.6. Excerpt from nmap-protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 15.1. A representative Nmap scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix Preface 1 . I ntroduction On September I , 1997, I released a security scanner named Nmap in the fifty-first issue of Phrack magazine. My goal was to consolidate the fragmented field of special-purpose port scanners into one powerful and flexible free tool, providing a consistent interface and efficient implementation of all practical port scanning techniques. Nmap then consisted of three files (barely 2,000 lines of code) and supported only the Linux operating system. It was written for my own purposes, and released in the hope that others would find it useful. From these humble beginnings, and through the power of Open Source development, Nmap grew into the world's most popular network security scanner 1 , with millions of users worldwide. Over the years, Nmap has continued to add advanced functionality such as remote OS detection, version/service detection, IP ID idle scanning, the Nmap Scripting Engine, and fast multi-probe ping scanning. It now supports all major Unix, Windows, and Mac OS platforms. Both console and graphical versions are available. Publications including Linux Journal, Info World, LinuxQuestions. Org, and the Codetalker Digest have recognized Nmap as "security tool of the year". It was even featured in several movies2, including The Matrix Reloaded, The Bourne Ultimatum, and Die Hard 4. Nmap ("Network Mapper") is a free and open source utility for network exploration and security auditing. Many systems and network administrators also fi nd it useful for tasks such as network inventory, managing service upgrade schedules, and monitoring host or service uptime. Nmap uses raw IP packets in novel ways to determine what hosts are available on the network, what services (application name and version) those hosts are offering, what operating systems (and OS versions) they are running, what type of packet filters/firewalls are in use, and dozens of other characteristics. It was designed to rapidly scan large networks, but works fine against single hosts. While Nmap is extremely powerful, 1t 1s also complex. More than 100 command-line options add expressiveness for networking gurus, but can confound novices. Some of its options have never even been documented. This book documents all Nmap features and, more i mportantly, teaches the most effective ways of using them. It has taken nearly four years to write, with constant updating as Nmap has evolved. This book is dedicated to the Nmap community of users and developers. Your passion, ideas, patches, feature requests, flame wars, bug reports, and midnight rants have shaped Nmap into what it is today. -Gordon "Fyodor" Lyon < f yodor @ i n s ecure . org> 2. Intended Aud ience and Organ ization This book documents the free Nmap Security Scanner, from port scanning basics for novices to the types of packet crafting used by advanced hackers. It should benefit Nmap users (or potential users) of all experience levels. 1 Ba�ed on download frequency, number of Google hits, and Freshrneat.Net software "popularity" ranking. 2 http://nmap.org/movies.html 1 . Introduction xxi Starting with the basics, this book gives an overview of Nmap by example in Chapter I . Then Chapter 2 covers obtaining, compiling and installing Nmap. Chapters 3 through 5 cover features in the order you might use them when conducting a penetration test. First comes host discovery ("ping scanning"), which determines the avai lable hosts on a network. Next, port scanning is covered in depth. In Chapter 5, all the Nmap scanning techniques are detailed, with advice and examples. Scanning a large network can take a long time, so Chapter 6 is full of performance optimization advice. Chapter 7 details service and application version detection, in which Nmap queries ports to determine exactly what is running rather than simply guessing based on the port number. Chapter 8 covers one of Nmap's most loved features: remote OS detection. Chapter 9 details one of Nmap's newest features: the Nmap Scripting Engine. NSE allows users and developers to easily extend Nmap with new features by writing simple scripts to be efficiently executed against target machines. My favorite chapter is number 10: Detecting and Subverting Firewalls and lnt.rusion Detection Systems. For balance, that is followed by a chapter on defending against Nmap scans. Chapter 1 2 then fully documents the Zenmap multi-platform Nmap GUI and results viewer. The next two chapters cover output formats and data files. The fi nal and longest chapter is the Nmap Reference Guide, a quick resource for looking up specific Nmap options. Scattered throughout the book are detailed instructions for performing common tasks such as scanning a network for a certain single open TCP port or detecting wireless access points by scanning from the wired side. First each problem is described, then an effective solution is provided. A final discussion section describes the solution in more depth and may provide alternative solutions and insights into similar problems. 3. Conventions Nmap output is used throughout this book to demonstrate principles and features. The output is often edited to cut out lines which are irrelevant to the point being made. The dates/times and version numbers printed by Nmap are generally removed as well, since some readers find them distracting. Sensitive information such as hostnames, IP addresses, and MAC addresses may be changed or removed. Other information may be cut or li nes wrapped so that they fit on a printed page. Similar editing is done for the output of other applications. Example I gives a glimpse at Nmap's capabilities while also demonstrating output formatting. xx ii 3. Conventions Example 1. A typical Nmap scan # nmap -A -T4 s canme . nmap . or g Start ing Nmap ( http : / / nmap . or g I ntere s t i ng por t s on s canme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) : Not shown : 9 9 4 f i ltered port s PORT STATE SERV I CE VERS I ON 2 2 / t cp open ssh OpenSSH 4 . 3 ( protocol 2 . 0 ) 2 5 / t cp c l o sed smtp 5 3 / tcp open doma i n I SC B I ND 9 . 3 . 4 7 0 /tcp c losed gopher 8 0 /tcp open http Apa che httpd 2 . 2 . 2 ( ( Fedor a ) } I _ HTML t i t l e : Go ahead and S canMe ! 1 1 3 / t cp c l osed auth Device type : general purpose Runn ing : Linux 2 . 6 . X OS deta i l s : L i nux 2 . 6 . 2 0 - 1 ( Fedora Core 5 ) TRACEROUTE ( u s i ng port 8 0 / t cp } HOP RTT ADDRESS [ Cut f i r s t seven hops for brev i t y ) 1 0 . 5 9 so-4 - 2 - 0 . mpr3 . pa o l . us . above . ne t ( 6 4 . 1 2 5 . 2 8 . 1 4 2 ) 8 9 1 1 . 0 0 metroO . sv . svco l o . com ( 2 0 8 . 1 8 5 . 1 6 8 . 1 7 3 ) 1 0 9 . 9 3 scanme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) Nmap done : 1 I P addre s s ( 1 host up } s canned i n 1 7 . 0 0 s e conds Special formatting is provided for certain tokens, such as filenames and application commands. Table demonstrates the most common formatting conventions. Table 1. Formatting style conventions Token type Example literal string I get much more excited by ports in the open state than those reported as c l o s e d or f i l t ered. Command-line options One of the coolest, yet least understood Nmap options is --pack e t - t r a c e . Filenames Follow the -iL option with the input filename such as C : \ n e t \dhcp- l e a s e s . t x t or / home / h 4 x / ho s t s -t o-pwn . l s t . Emphasis Using Nmap from your work or school computer to attack banks and military targets is a bad idea. Application commands Trinity scanned the Matrix with the command nmap v -sS -0 10.2.2.2. Replaceable variables - Let be the machine running Nmap and be m i c r o s o f t . com. 4. Other Resou rces While this book is an important reference for Nmap, it isn't the only one. The Nmap web page at http://nmap.org is not just for downloads. It also provides substantial documentation from Nmap developers 4. Other Resources xxiii and third parties. For example, you can fi nd the Nmap Reference Guide translated into a dozen languages there. Other books, videos, and articles covering Nmap are also available. The official web site for this book is at http://nmap.org/book/. Go there for errata, updates, and many sample chapters. Any serious Nmap user should subscribe to the nmap-hackers mailing list for announcements about Nmap and Insecure.Org. Traffic is very light (about six posts per year) because it is reserved for only the most important announcements. Developers and particularly devoted users can also subscribe to the nmap-dev mailing list. Traffic is much higher (hundreds of posts per month), but it is a great place to learn about and try new features before they are released and to pick up tips from advanced users. Subscription information and archives for both lists are available at http://seclists.org. While Nmap can be useful, it won't solve all of your security problems. Every few years I do a survey of thousands of Nmap users to determine what other tools they like. The list is posted at http://sectools. org, which has become one of my most popular web sites. Read through the list and you are sure to find many gems you had never even heard of. Most of the tools are free and open source. 5. Req uest for Comments While I tried m y best t o make this book comprehensive, accurate, and up-to-date, we a l l make mistakes. If you find any problems or just have suggestions for making this book better, please let me know by email at < f yodor @ i n s ecure . org>. The open source principle of many readers and contributors is just as viable for documentation as for software. As the next section attests, dozens of people have already generously contributed their time and skills to make this book a success. If you have a question or comment about Nmap (rather than this book itself), it is best sent to the Nmap development list as described at Section 1 5 . 17, "Bugs" [4 1 1 ). 6. Acknowledgements When I first floated the idea o f writing a n Nmap book to the nmap-hackers mailing list, I was inundated with suggestions and offers to help. This outpouring of enthusiasm convinced me to proceed. My complete naivety about how much work was involved also contributed to my decision. It has been quite an undertaking, but what kept me going chapter by chapter was a private review group called the nmap-writers. They provided invaluable feedback, advice, and detailed review notes throughout the process. In particular, I would like to thank the following people: • David Fifield is listed first (everyone else is alphabetical) because he was a tremendous help during the book writing process. He solved a number of technical DocBook problems, created many of the final illustrations from my terrible drafts, dramatically improved the index, helped with proofreading, and even wrote Chapter 1 2, Zenmap GUI Users ' Guide [307). • Matt Baxter allowed the use of his beautiful TCP/IP header diagrams (in Section 7, "TCP/IP Reference" [xx vi]). Several other diagrams in this book were done in that style to match. • Saurabh Bhasin contributed detailed feedback on a regular basis. xxiv 5 . Request for Comments • Mark Brewis could always be counted on for good advice. • Ellen Colombo was a big help from the beginning. • Patrick Donnelly helped improve Chapter 9, Nmap Scripting Engine [205] . • Brandon Enright printed out the whole book and reviewed it chapter by chapter. • Brian Hatch has always been a big help. • Loren Heal was a continual source of ideas. • Dan Benage provided advice and proofread numerous chapters. • Tor Houghton reviewed every chapter, probably giving me more feedback than anyone else. • Doug Hoyte documented the many Nmap features he added, and also handled most of the book indexing. • Marius Huse Jacobsen reviewed many chapters, providing detailed feedback. • Kris Katterjohn performed thorough reviews of several chapters. • Eric Krosnes sent useful technical review feedback and also regularly nagged me about book progress. This was helpful since I didn't have a traditional editor to do so. • Vlad Alexa Mancini created the Nmap eye logo for the cover (and the Nmap web site). • • Michael Naef kindly reviewed many chapters. Bill Pollock of No Starch Press was always happy to provide advice and answer book publishing questions based on his decades of experience. • David Pybus was one of the most frequent contributors of ideas and proofreading. • Tyler Reguly helped by reviewing multiple chapters just when it was most needed. • Chuck Sterling provided both high level advice and detailed proofreading of several chapters. • Anders Thulin provided detailed reviews of many chapters. • Bennett Todd sent dozens of suggestions. • Diman Todorov wrote an initial draft of Chapter 9, Nmap Scripting Engine [205 ] . • Catherine Tornabene read many chapters and sent extremely detailed feedback. 6.1 . Technology Used to Create Th is Book As an author of open source tools myself, I'm a big believer in their power and capability. So I made a n effort to use them wherever possible in creating this book. I wasn't about to write it in Microsoft Word and then handle layout with Adobe FrameMaker! 6. Acknowledgements xxv Nmap Network Scanning was written with the GNU Emacs text editor in the DocBook XML format. The free online chapters are created from the XML using Norman Walsh's XSL Stylesheets and the xsltproc XSL processor. The print version also uses Norman's stylesheets and xsltproc, but the output is to the XSL-FO format3 . An XSL-FO processor is then used to build a PDF. I would like to use Apache FOP4 for this, but a footnote-related bug5 prevents this, so I switched to the RenderX XEP Engine. XEP i s proprietary, but at least it runs on Linux. I hope to switch back to FOP after the footnote bug is fixed. Cover layout was done with Scribus and (due to printing company format requirements) Adobe InDesign. Raster graphics for the cover and internal ill ustrations were created with The Gimp, while Inkscape was used for vector graphics. Subversion was used for revision control and the free web chapters are serviced by Apache httpd. 7. TCP/I P Reference This book assumes basic familiarity with TCP/IP and networking concepts. You won't find a primer on the OSI seven-layer model or a rundown of the Berkeley Socket API within these pages. For a comprehensive guide to TCP/IP, I recommend "The TCP/IP Guide" by Charles Kozierok or the old classic "TCP/IP Illustrated, Volume f' by W. Richard Stevens. While TCP/IP familiarity is expected, even the best of us occasionally forget byte offsets for packet header fields and flags. This section provides quick reference diagrams and field descriptions for the 1Pv4, TCP, UDP, and ICMP protocols. These beautiful diagrams from http://www.fatpipe. org!-mjb/Drawings are used by permission of author Matt Baxter. 3 http://en. wikipedia.org/wiki/XSL_Formatting_Objects http://xmlgraphics.apache.org/fopl 5 https://issues.apache.org/bugzil/a/show_b11g.cgi ?id=37579 4 xx vi 7. TCP/IP Reference Figure 1. 1Pv4 header 0 .i .l ..l. 1 ..l. ..l. ..l. ..l. ..l. ..l. I Lerigth) .l .1 .l ..l. ..l. 2 ..l. ..l. .l -1 .l .l .l J3.i .l .l .l .l .l .l � Flags TI J l �r s IHL Internet Header I 1 l -.7 T -3 --------t� 7. TCP/IP Reference xx vii Figure 2. TCP header OJ_ J_ J_ ..l J. J. J. 11..l_ J_ J_ ..l J. J. J. 2i J. J. ...l J. J. J_ l3J_ J_ J_ ..l J_ J_ J_ 4 1 Jc .. ·r TI *I B I: T I ··2 I . 3 l --------tM �I Figure 3. UDP header 2 -------1.i xx viii 7. TCP/IP Reference s Figure 4. ICMP header ,._ � � � � � � � � ....� .. � � � � � � ....� . � � � � � � � � � � � � � � __.. + l ............ ... .... .. .... . .... .. ........ .. .... . .... . .... .. ... . ........ .. .... . .. ........ . .... ... ... --1... Word --------..� 7. TCP/IP Reference xx ix - Chapter 1 . Getting Started with Nmap 1 .1 . I ntrod uction Nmap ("Network Mapper") i s a free and open source utility for network exploration and security auditing. Many system s and network administrators also find it useful for tasks such as network inventory, managing service upgrade schedules, and monitoring host or service uptime. Nmap uses raw IP packets in novel ways to determine what hosts are available on the network, what services (application name and version) those hosts are offering, what operating systems (and OS versions) they are running, what type of packet fil ters/firewalls are in use, and dozens of other characteristics. It was designed to rapidly scan large networks, but works fine against single hosts. Nmap runs on all m ajor computer operating systems, and both console and graphical versions are available. This chapter uses fictional stories to provide a broad overview of Nmap and how it is typically used. An important legal section helps users avoid (or at least be aware ot) controversial usage that could lead to ISP account cancellation or even civil and criminal charges. It also discusses the risks of crashing remote machines as well as miscellaneous issues such as the Nmap license (GNU GPL), and copyright. 1 .2 . Nmap Overv.i ew and Demonstration Sometimes the best way to understand something is to see i t in action. This section includes examples of Nmap used in (mostly) fictional yet typical circumstances. Nmap newbies should not expect to understand everything at once. This is simply a broad overview of features that are described in depth in later chapters. The "solutions" incl uded throughout this book demonstrate many other common Nmap tasks for security auditors and network administrators. 1 .2.1 . Avatar On l i ne Felix dutifully arrives at work on December 15th, although he does not expect many structured tasks. The small San Francisco penetration-testing firm he works for has been quiet lately due to impending holidays. Felix spends business hours pursuing his latest hobby of building powerful Wi-Fi antennas for wireless assessments and war driving exploration. Nevertheless, Felix is hoping for more business. Hacking has been his hobby and fascination since a childhood spent learning everything he could about networking, security, Unix, and phone systems. Occasionally his curiosity took him too far, and Felix was almost swept up in the 1990 Operation Sundevil prosecutions. Fortunately Felix emerged from adolescence without a criminal record, while retaining his expert knowledge of security weaknesses. As a professional, he is able to perform the same types of network intrusions as before, but with the added benefit of contractual immunity from prosecution and even a paycheck! Rather than keeping his creative exploits secret, he can brag about them to client management when presenting his reports. So Felix was not disappointed when his boss interrupted his antenna soldering to announce that the sales department finally closed a pen-testing deal with the Avatar Online gaming company. Avatar Online (AO) is a small company working to create the next generation of massive m u lti-player online role-playing games (MMORPGs). Their product, inspired by the Metaverse envisioned in Neil Stevenson's 1 . 1 . Introduction Snow Crash, is fascinating but still highly confidential. After witnessing the high-profile leak 1 of Valve Software's upcoming game source code, AO quickly hired the security consultants. Felix's task is to initiate an external (from outside the firewall) vulnerability assessment while his partners work on physical security, source code auditing, social engineering, and so forth. Felix is permitted to exploit any vulnerabilities found. The first step in a vulnerability assessment is network discovery. This reconnaissance stage determi nes what IP address ranges the target is using, what hosts are available, what services those hosts are offering, general network topology details, and what firewall/filtering policies are in effect. Determining the IP ranges to scan would normally be an elaborate process involving ARIN (or another geographical registry) lookups, DNS queries and zone transfer attempts, various web sleuthing techniques, and more. But in this case, Avatar Online explicitly specified what networks they want tested: the corporate network on 6.209.24.0/24 and their production/DMZ systems residing on 6.207.0.0/22. Fel ix checks the 2 ARIN IP allocation records anyway and confirms that these IP ranges belong to A0 . Felix subconsciously 3 decodes the CIDR notation and recognizes this as 1 ,280 IP addresses. No problem. Being the careful type, Felix first starts out with what is known as an Nmap list scan ( - s L option). This feature simply enumerates every IP address in the given target netblock(s) and does a reverse-DNS lookup (unless -n was specified) on each. One reason to do this first is stealth. The names of the hosts can hint at potential vulnerabilities and allow for a better understanding of the target network, all without raising alarm 4 bells . Felix is doing this for another reason-to double-check that the IP ranges are correct. The systems administrator who provided the IPs might have made a mistake, and scanning the wrong company would be a disaster. The contract signed with Avatar Online may act as a get-out-of-jail-free card for penetrating their networks, but will not help if Felix accidentally roots another company's server! The command he uses and an excerpt of the results are shown in Example 1 . 1 . 1 http://www.smh.com.au/articles/2003/I0/0311064988378345.html 2 These IP addresses are actually registered to the United States Army Yuma Proving Ground, which is used to test a wide variety of artillery, missiles, tanks, and other deadly weapons. The moral is to be very careful about who you scan, lest you accidentally hit a highly sensitive network. The scan results in this story are not actually from this IP range. 3Classless Inter-Domain Routing (CIDR) notation is a method for describing networks with more granularity than class A (CIDR /8), class B (CIDR /16), or class C (CIDR /24) notation. An excellent description is available at http://public.pacbell.net/dedicatedlcidr.html. 4 It is possible that the target nameserver will log a suspicious bunch of reverse-DNS queries from Felix's nameserver, but most organizations don't even keep such logs, much less analyze them. 2 1 .2. Nmap Overview and Demonstration Example 1.1. Nmap list scan against Avatar Online IP addresses felix> nmap - s L 6 . 2 0 9 . 2 4 . 0 / 2 4 6 . 2 0 7 . 0 . 0 / 2 2 Starting Nmap ( http : / / nmap . or g ) Host 6 . 2 0 9 . 2 4 . 0 not s canned Host fw . corp . avataron l ine . com ( 6 . 2 0 9 . 2 4 . 1 ) not s canned Host dev2 . corp . avataron l i n e . com ( 6 . 2 0 9 . 2 4 . 2 ) not s canned Host 6 . 2 0 9 . 2 4 . 3 not s canned Host 6 . 2 0 9 . 2 4 . 4 not s canned Host 6 . 2 0 9 . 2 4 . 5 not s canned Host Host Host Host Host Host dhcp-2 1 . corp . avataron l ine . com dhcp-2 2 . corp . avataron l ine . com dhcp- 2 3 . corp . avataron l i ne . com dhcp- 2 4 . corp . avataron l ine . com dhcp- 2 5 . corp . avataron l ine . com dhcp- 2 6 . corp . avataron l ine . com Host Host Host Host Host Host Host Host Host 6 . 2 0 7 . 0 . 0 not s canned gw . avataron l ine . com ( 6 . 2 0 7 . 0 . 1 ) not s canned n s l . avataron l ine . com ( 6 . 2 0 7 . 0 . 2 ) not s canned ns2 . avataronl ine . com ( 6 . 2 0 7 . 0 . 3 ) not s canned ftp . avataron l ine . com ( 6 . 2 0 7 . 0 . 4 ) not s canned 6 . 2 0 7 . 0 . 5 not scanned 6 . 2 0 7 . 0 . 6 not scanned www . avataron l ine . com ( 6 . 2 0 7 . 0 . 7 ) not s canned 6 . 2 0 7 . 0 . 8 not scanned Host Host Host Host Host cluster-c l 2 0 . avataron l ine . com cluster-c l 2 1 . avataron l ine . com cluster-c l 2 2 . avataron l i n e . com cluster-c l 2 3 . avataron l ine . com cluster-c l 2 4 . avataron l ine . com ( 6 . 209 . 24 . 21 ) ( 6 . 209 . 24 . 22 ) ( 6 . 209 . 24 . 23 ) ( 6 . 209 . 24 . 24 ) ( 6 . 209 . 24 . 25 ) ( 6 . 20 9 . 24 . 26 ) ( 6 . 207 . 2 . 12 0 ) (6 . 207 . 2 . 121) (6 . 207 . 2 . 122) ( 6 . 207 . 2 . 12 3 ) ( 6 . 207 . 2 . 12 4 ) not not not not not not not not not not not s canned s canned s canned s canned s canned s canned s canned s canned s canned s canned s canned Host 6 . 2 0 7 . 3 . 2 5 3 not s canned Host 6 . 2 0 7 . 3 . 2 5 4 not scanned Host 6 . 2 0 7 . 3 . 2 5 5 not s canned Nmap done : 1 2 8 0 IP addres s e s s canned in 3 3 1 . 4 9 seconds felix> Reading over the results, Felix finds that all of the machines with reverse-DNS entries resolve to Avatar Online. No other businesses seem to share the IP space. Moreover, these results give Felix a rough idea of how many machines are in use and a good idea of what many are used for. He is now ready to get a bit more intrusive and try a port scan. He uses Nmap features that try to determine the application and version number of each service I istening on the network. He also requests that N map try to guess the remote operating system via a series of low-level TCP/IP probes known as OS fingerprinting. This sort of scan is not at all stealthy, but that does not concern Felix. He is interested in whether the administrators of AO even notice these blatant scans. After a bit of consideration, Felix settles on the following command: nmap -sS -p- -PS22,80,1 13,33334 -PAS0,1 13,21000 -PU19000 -PE -A -T4 -oA avatartcpscan-121503 6.209.24.0/24 6.207.0.0/22 1 .2. Nmap Overview and Demonstration 3 These options are descri bed in later chapters, but here is a quick summary of them. -ss Enables the efficient TCP port scanning technique known as SYN scan. Felix would have added a U at the end if he also wanted to do a UDP scan, but he is saving that for later. SYN scan is the default scan type, but stating it explicitly does not hurt. -p- Requests that Nmap scan every port from 1 -65535. The default is to scan only ports one through 1024, plus about 600 others explicitly mentioned in the nmap - s e r v i c e s database. This option format is simply a short cut for -p l - 6 5 5 3 5 . He could have specified -p 0 - 6 5 5 3 5 if he wanted to scan the rather illegitimate port zero as well. The -p option has a very flexible syntax, even allowing the specification of a differing set of UDP and TCP ports. - P S 2 2 , 8 0 , 1 1 3 , 3 3 3 3 4 -PAB 0 , 1 1 3 , 2 1 0 0 0 -PU 1 9 0 0 0 -PE These are a l l ping types used in combination to determine whether a host i s really available and avoid wasting a lot of time scanning IP addresses that are not in use. This particular incantation sends a TCP SYN packet to ports 22, 80, 1 1 3, and 33334; a TCP ACK packet to ports 80, 1 1 3, and 21000; a UDP packet to port 1 9000; and a normal ICMP echo request packet. If Nmap receives a response from the target host itself to any of these probes, it considers the host to be up and available for scanning. This is more extensive than the Nmap default, which simply sends an echo request and an ACK packet to port 80. In a pen-testing situation, you often want to scan every host even if they do not seem to be up. After all, they could just be heavily filtered in such a way that the probes you selected are ignored but some other obscure port may be available. To scan every IP whether it shows an available host or not, specify the -PN option instead of all of the above. Felix starts such a scan in the background, though it may take a day to complete. -A This shortcut option turns on Advanced and Aggressive features such as OS and service detection. At the time of this writing it is equivalent to - s V - s C -0 - - t r aceroute (version detection, Nmap Scripting Engine, remote OS detection, and traceroute). More features may be added to -A later. -T4 Adjusts timing to the aggr e s s ive level (#4 of 5). This is the same as specifying -T aggr e s s ive, but is easier to type and spell. In general, the -T4 option is recommended if the connection between you and the target networks are faster than dialup modems. -oA avat a r t cp s can- 1 2 1 5 0 3 Outputs results in every format (normal, XML, grepable) to fi les named avat a r t cp s can- 1 2 1 5 0 3 . where the extensions are .nmap, .xml, and .gnmap respectively. All of the output formats include the start date and time, but Felix likes to note the date 5 explicitly in the filename. Normal output and errors are sti ll sent to stdout as well. 5 stdout i s the "C" notation for representing the standard output mechanism fo r a system, such a s t o the Unix xterm o r Windows command window in which Nmap was initiated. 4 1 .2. Nmap Overview and Demonstration 6 . 209 . 2 4 . 0 / 2 4 6 . 20 7 . 0 . 0 / 2 2 These are the Avatar Online netblocks discussed above. They are given in CIDR notation, but Nmap allows them to be specified in many other formats. For example, 6 . 2 0 9 . 2 4 . 0 I 2 4 could instead be specified as 6 . 2 0 9 . 2 4 . 0 - 2 5 5 . Since such a comprehensive scan against more than a thousand IP addresses could take a while, Felix simply starts it executing and resumes work on his Yagi antenna. A couple hours later he notices that it has fi nished and takes a peek at the results. Example 1 .2 shows one of the machines discovered. Example 1.2. Nmap results against an AO firewall Interest ing por t s on fw . corp . avataron l ine . corn ( 6 . 2 0 9 . 2 4 . 1 ) : (The 6 5 5 3 0 por t s s canned but not shown be l ow are in s t a t e : f i l t ered ) PORT STATE SERV I CE VERS I ON 22/ tcp open ssh OpenSSH 3 . 7 . lp 2 ( protocol 1 . 9 9 ) 53 /tcp open doma i n I SC B I ND 9 . 2 . 1 Courier pop3d 1 1 0 /tcp open pop3 1 1 3 / t cp cl osed auth 143/tcp open irnap Cour i e r Imap 1 . 6 . X - 1 . 7 . X 3 1 2 8 / t cp open ht tp-proxy Squ i d webproxy 2 . 2 . STABLE5 Device type : gener a l purpose Running : Linux 2 . 4 . X l 2 . 5 . X OS deta i l s : L i nux Kernel 2 . 4 . 0 - 2 . 5 . 2 0 Upt ime 3 . 1 3 4 days To the trained eye, this conveys substantial information about AO's security posture. Felix first notes the reverse DNS name-this machine is apparently meant to be a firewall for their corporate network. The next line is important, but all too often ignored. It states that the vast majority of the ports on this machine are in the f i l t er e d state. This means that Nmap is unable to reach the port because it is blocked by firewal l rules. The fact that a l l ports except for a few chosen ones are in this state is a sign o f security competence. Deny-by-default is a security mantra for good reasons-it means that even if someone accidentally left SunRPC (port 1 1 1 ) open on this machine, the firewall rules would prevent attackers from communicating with it. Felix then looks at every port line in turn. The first port is Secure Shell (OpenSSH). Version 3.7. l p2 is common, as many administrators upgraded to this version due to potentially exploitable buffer management bugs affecting previous versions. Nmap also notes that the SSH protocol is 1 .99, suggesting that the inferior legacy SSHv l protocol is supported. A truly paranoid sysadmin would only allow SSH connections from certain trusted IP addresses, but one can argue for open access i n case the administrator needs emergency access while far from home. Security often involves trade-offs, and this one may be justifiable. Felix makes a note to try his brute force password cracker and especially his private timing-based SSH user enumeration tool against the server. Felix also notes port 53. It is running ISC BIND, which has a long history of remotely exploitable security 6 holes. Visit the BIND security page for further details. BIND 9.2 . l even has a potentially exploitable buffer overflow, although the default build is not vulnerable. Felix checks and finds that this server is not vulnerable to the libbind issue, but that is beside the point. This server almost certainly should not be running an externally-accessible nameserver. A firewall should only run the bare essentials to minimize the risk of a disastrous compromise. Besides, this server is not authoritative for any domains-the real nameservers are 6 http://www.isc.org/products/BIND/bind-security.html 1 .2. Nmap Overview and Demonstration 5 on the production network. An administrator probably only meant for clients within the firewall to contact this nameserver, but did not bother locking it down to only the internal interface. Felix will later try to gather important information from this unnecessary server using zone transfer requests and intrusive queries. He may attempt cache poisoning as well. By spoofing the IP of wi ndows updat e . mi c r o s o f t . c om or another important download server, Felix may be able to trick unsuspecting internal client users into running a trojan-horse program that provides him with ful l network access behind the firewall. The next two open ports are 1 10 (POP3) and 143 (IMAP). Note that 1 1 3 (auth) between them is c l o s ed instead of ope n . POP3 and IMAP are mail retrieval services which, like BIND, have no legitimate place on this server. They are also a security risk in that they generally transfer the mail and (even worse) authentication credentials unencrypted. Users should probably VPN in and check their mail from an internal server. These ports could also be wrapped in SSL encryption. Nmap would have then listed the services as s s l /pop3 and s s l / imap. Felix will try his user enumeration and password guessing attacks on these services, which will probably be much more effective than against SSH. The final open port is a Squid proxy. This is another service that may have been intended for internal client use and should not be accessible from the outside (and particularly not on the firewall). Felix's initially positive opinion of the AO security administrators drops further. Felix will test whether he can abuse this proxy to connect to other sites on the Internet. Spammers and malicious hackers often use proxies in this way to hide their tracks. Even more critical, Felix will try to proxy his way into the internal network. This 7 common attack is how Adrian Lamo broke into the New York Times internal network in 2002. Lamo was caught after he called reporters to brag about his exploits against the NY Times and other companies in detail8 . The following li nes disclose that this is a Linux box, which is valuable information when attempting exploitation. The low three-day uptime was detected during OS fingerprinting by sending several probes for the TCP timestamp option value and extrapolating the line back to zero. Felix then examines the Nmap output for another machine, as shown in Example l .3. 7 8 6 http://e11. wikipedia.orglwik/Adrian_Lamo i http:llwww.sec11rityfocus.com/11ewsl340 1 .2. Nmap Overview and Demonstration Example 1.3. Another interesting AO machine Intere st ing por t s on dhcp- 2 3 . corp . avataron l i ne . com ( 6 . 2 0 9 . 2 4 . 2 3 ) : ( The 6 5 5 2 6 por t s s canned but not s hown below are i n s t a t e : c l osed ) PORT STATE SERV I C E VERS I ON 135/tcp f i l t ered msrpc 136 /tcp f i l t ered prof i l e 1 3 7/tcp f i ltered netbios-ns 1 3 8 /tcp f i ltered netbi o s -dgm f i l tered netbios - s s n 1 3 9 /tcp 445 /tcp open micros o f t - d s Microsoft Windows XP micros o f t -d s windows - i cfw? 1 0 0 2 /tcp open 1 025/t cp open msrpc Mi crosoft Wi ndows msrpc 165 52 /tcp open unknown Device type : general purpos e Running : Microsoft Wi ndows N T / 2 K/XP OS detai l s : Microsoft Wi ndows XP Profes s i onal RC l + through final r e le a s e Felix smiles when he spies this Windows XP box on the Network. Thanks to a spate of MS RPC vulnerabilities, those machines are trivial to compromise if the OS patches aren't up-to-date. The second line shows that the default state is c l o s ed, meaning the firewall does not have the same deny-by-default policy for this machine as for itself. Instead they tried to specifically block the Windows ports they consider dangerous on 135-1 39. This filter is woefully inadequate, as MS exports MS RPC functionality on many other ports in Windows XP. TCP ports 445 and 1025 are two examples from this scan. While Nmap failed to recognize 16552, Felix has seen this pattern enough to know that it is probably the MS Messenger Service. If AO had been using deny-by-default filtering, port 16552 would not be accessible in the first place. Looking through the results page, Felix sees several other Windows machines on this DHCP network. Felix cannot wait to try his favorite DCOM RPC exploit against them. It was written by HD Moore and is available at http://www.metasploit.com/toolsldcom.c. If that fails, there are a couple newer MS RPC vulnerabilities he will try. Felix continues poring over the results for vulnerabilities he can leverage to compromise the network. On the production network, he sees that gw . avat aron l i ne . com is a Cisco router that also acts as a rudimentary firewal l for the systems. They fall into the trap of only blocking privileged ports (those under 1024), which leaves a bunch of vulnerable SunRPC and other services accessible on that network. The machines with names like cl u s t - * each have dozens of ports open that Nmap does not recognize. They are probably custom daemons running the AO game engine. www . avataron l i ne . com is a Linux box with an open Apache server on the HTTP and HTTPS ports. Unfortunately, it is linked with an exploitable version of the OpenSSL library. Oops! Before the sun sets, Felix has gained privileged access to hosts on both the corporate and production networks. As Felix has demonstrated, Nmap is frequently used by security auditors and network administrators to help locate vulnerabilities on client/corporate networks. Subsequent chapters describe the techniques used by Felix, as well as many other Nmapfeatures, in much greater detail. 1 .2. Nmap Overview and Demonstration 7 1 .2.2. Saving the Human Race Figure 1.1. Trinity begins her assault Trinity is in quite a pickle! Having discovered that the world we take for granted is really a virtual "Matrix" run by machine overlords, Trinity decides to fight back and free the human race from this mental slavery. Making matters worse, her underground colony of freed humans (Zion) is under attack by 250,000 powerful alien sentinels. Her only hope involves deactivating the emergency power system for 27 city blocks in less than five minutes. The previous team died trying. In life's bleakest moments when all hope seems to be lost, what should you turn to? Nmap, of course ! But not quite yet. She first must defeat the perimeter security, which on many networks involves firewalls and intrusion detection systems (IDS). She is well aware of advanced techniques for circumventing these devices (covered later in this book). Unfortunately, the emergency power system administrators knew better than to connect such a critical system to the Internet, even indirectly. No amount of source routing or IP ID spoofed scanning will help Trinity overcome this "air gap" security. Thinking fast, she devises a clever plan that involves jumping her motorcycle off the rooftop of a nearby building, landing on the power station guard post, and then beating up all of the security guards. This advanced technique is not covered in any physical security manual, but proves highly effective. This demonstrates how clever hackers research and devise their own attacks, rather than always utilizing the script-kiddie approach of canned exploits. Trinity fights her way to the computer room and sits down at a terminal. She quickly determines that the network is using the private 1 0.0.0.0/8 network address space. A ping to the network address generates responses from dozens of machines. An Nmap ping scan would have provided a more comprehensive list 8 1 .2. Nmap Overview and Demonstration of available machines, but using the broadcast technique saved precious seconds. Then she whips out Nmap9. The terminal has version 2.54BETA25 installed. This version is ancient (2001 ) and less efficient than newer releases, but Trinity had no time to install a better version from the future. This job will not take long anyway. She runs the command nmap -v -sS -0 10.2.1.3. This executes a TCP SYN scan and OS detection against 10.2.1.3 and provides verbose output. The host appears to be a security disaster-AIX 3.2 with well over a dozen ports open. Unfortunately, this is not the machine she needs to compromise. So she runs the same command against 10.2.2.2. This time the target OS is unrecognized (she should have upgraded Nmap !) and only has port 22 open. This is the Secure Shell encrypted administration service. As any sexy PVC-clad hacker goddess knows, many SSH servers from around that time (2001 ) have an exploitable vulnerability in the CRC32 compensation attack detector. Trinity whips out an all-assembly-code exploit and utilizes i t to change the root password o f the target box to z 1 ON O 1 0 1 . Trinity uses much more secure passwords under normal circumstances. She logs in as root and issues a command to disable the emergency backup power system for 27 city blocks, finishing just in time ! Here is a shot of the action-squint just right and you should be able to read the text. Figure 1.2. Trinity scans the Matrix In addition, a terminal-view video showing the whole hack is available on the Internet JO. At least it will be until the MPAA finds out and sends sentinels or lawyers after the webmasters. 1 .2 . 3 . Mad Hat i n Wonderland This story differs from the previous ones in that it i s actually true. Written by frequent Nmap user and contributor MadHat, it describes how he enhanced and customized Nmap for daily use in a large enterprise. 9 A sexy leather-clad attacker from the previous team actually started the session. It is unclear at what point she died and left the remaining tasks to Trinity. IO http://11map.org/movies.ht111/ 1 .2. Nmap Overview and Demonstration 9 In true open source spirit, he has released these valuable scripts on his Web site 1 1 • IP addresses have been changed to protect the corporate identity. The remainder of this section is in his own words. After spending the past couple of decades learning computers and working my way up from tech support through sysadmin and into my dream job of Information Security Officer for a major Internet company, I found myself with a problem. I was handed the sole responsibility of security monitoring for our entire IP space. This was almost 50,000 hosts worldwide when I started several years ago, and it has doubled since then. Scanning all of these machines for potential vulnerabilities as part of monthly or quarterly assessments would be tough enough, but management wanted it done daily. Attackers will not wait a week or month to exploit a newly exposed vulnerability, so I can 't wait that long to find and patch it either. Looking around for tools, I quickly chose Nmap as my port scanner. It i s widely considered to be the best scanner, and I had already been using it for years to troubleshoot networks and test security. Next I needed software to aggregate Nmap output and print differences between runs. I considered several existing tools, including HD Moore's Nlog 12 • Unfortunately none of these monitored changes in the way I desired. I had to know whenever a router or firewall access control list was misconfigured or a host was publicly sharing inappropriate content. I also worried about the scalability of these other solutions, so I decided to tackle the problem myself. The first issue to come up was speed. Our networks are located worldwide, yet I was provided with only a single U.S.-based host to do the scanning. In many cases, firewalls between the sites slowed the scanning down significantly. Scanning all 100,000 hosts took over 30 hours, which is unacceptable for a daily scan. So I wrote a script called nmap-wrapper which runs dozens of Nmap processes in parallel, reducing the scan time to fifteen hours, even including OS detection. The next problem was dealing with so much data. A SQL database seemed like the best approach for scalability and data-mining reasons, but I had to abandon that idea due to time pressures. A future version may add this support. Instead, I used a flat file to store the results of each class C address range for each day. The most powerful and extensible way to parse and store this information was the Nmap XML format, but I chose the "grepable" (-oG option) format because it is so easy to parse from simple scripts. Per-host timestamps are also stored for reporting purposes. These have proven quite helpful when administrators try to blame machine or service crashes on the scanner. They cannot credibly claim a service crash at 7: 12AM when I have proof that the scan ran at 9:45AM. The scan produces copious data, with no convenient access method. The standard Unix diff tool is not smart enough to report only the changes I care about, so I wrote a Perl script named nmap-diff to provide daily change reports. A typical output report is shown in Example 1 .4. 11 12 http://www.unspecific.com/mnap/ http:llwww.securiteam.com/tools/3T5QMQONFK.html IO 1 .2. Nmap Overview and Demonstration Example 1.4. nmap-diff typical output > nmap-di f f . pl - c 3 5 IPs showed change s 1 0 . 1 2 . 4 . 8 ( ftp-box . foocompan y . bi z ) open 2 1 / t cp ftp open 8 0 / t cp http 4 4 3 / tcp https open IIS open 1 0 2 7 / tcp open ms - l sa + 1 0 2 9 / t cp landesk -cba 3 8 2 9 2 / t cp open OS : Microsoft Wi ndows Mi l l ennium Edi t i on ( Me ) Windows 2 0 0 0 Profe s s i ona l or Advanced Server or Wi ndows XP 1 0 . 1 6 . 2 3 4 . 3 ( medi a . foocompan y . bi z ) http open 8 0 / t cp rtsp open + 5 5 4 / tcp r e a l server open + 7 0 7 0 /tcp 19 2 . 1 6 8 . 1 0 . 1 8 6 ( te s tbox . foocompan y . bi z ) open black i c e - a l e r t s + 8 0 8 2 / t cp OS : Linux Kernel 2 . 4 . 0 - 2 . 5 . 2 0 1 72 . 2 4 . 1 2 . 5 8 (mtafoocompan y . bi z ) smtp + 2 5 / tcp open OS : FreeBSD 4 . 3 - 4 . 4 PRERELEASE 1 72 . 2 3 . 76 . 2 2 ( media 2 . foocorp . bi z ) http open 8 0 /tcp IIS open 1 0 2 7 / t cp netsa int + 1 0 4 0 / tcp open wms 1 7 5 5 / t cp open open msdtc 3 3 7 2 / t cp 6 6 6 6 / t cp i r c- serv open 7 0 0 7 / tcp a f s 3 -bos open OS : Microsoft Wi ndows M i l l enn ium Edit ion ( Me ) Windows 2 0 0 0 Prof e s s ional or Advanced Server or Wi ndows XP Management and staff were impressed when I demonstrated this new system at an internal company security symposium. But instead of allowing me to rest on my laurels, they began asking for new features. They wanted counts of mail and web servers, growth estimates, and more. This data was all available from the scans, but was difficult to access. So I created yet another Perl script, nmap-report, which made querying the data much easier. It takes specifications such as open ports or operating systems and finds all the systems that matched on a given day. One problem with this approach to security monitoring is that employees do not always place services on their !ANA-registered official ports. For example, they might put a web server on port 22 (SSH) or vice versa. Just as I was debating how to address this problem, Nmap came out with an advanced service and version detection system (see Chapter 7, Service and Application Version Detection [ 1 45]). nmap-report now has a rescan feature that uses version scanning to report the true services rather than guessing based on port 1 .2. Nmap Overview and Demonstration 11 number. I hope to further integrate version detection in future versions. Example 1 .5 shows nmap-report listing FTP servers. Example 1.5. nmap-report execution > nmap-report -p2 1 -rv [. . .] 1 72 . 2 1 . 1 9 9 . 76 ( ft p l . foocorp . bi z ) 2 1 / t cp open s s l l ftp Serv-U ftpd 4 . 0 1 9 2 . 1 6 8 . 1 2 . 5 6 ( f tp2 . foocorp . bi z ) NcFTPd ftp open 2 1 / t cp 1 9 2 . 1 6 8 . 1 3 . 1 3 0 ( dropbox . foocorp . bi z ) 2 1 / t cp open ftp WU-FTPD 6 . 0 0 LS While being far from perfect, these scripts have proven themselves quite valuable at monitoring large networks for security-impacting changes. Since Nmap itself is open source, it only seemed fair to release my scripts to the public as well. I have made them freely available at http://www. unspecific.com/nmap. 1 .3. The Phases of an Nmap Scan Now that we've seen some applications of Nmap, let's look at what happens when an Nmap scan runs. Scans proceed in phases, with each phase finishing before the next one begins. As you can see from the phase descriptions below, there is far more to Nmap than just port scanning. Target enumeration. In this phase, Nmap researches the host specifiers provided by the user, which may be a combination of host DNS names, IP addresses, CIDR network notations, and more. You can even use ( - i R) to ask Nmap to choose your targets for you ! Nmap resolves these specifiers into a list of 1Pv4 or IPv6 addresses for scanning. This phase cannot be skipped since it is essential for further scanning, but you can simplify the processing by passing just IP addresses so Nmap doesn't have to do forward resolution. If you pass the - s L - n options (list scan with no reverse-DNS resolution), Nmap will print out the targets and perform no further scanning. This phase is discussed in Section 3.2, "Specifying Target Hosts and Networks" [47] and Section 3.5. 1 , "List Scan (-sL)" [57]. Host discovery (ping scanning). Network scans usually begin by discovering which targets on the network are online and thus worth deeper investigation. This process is called host discovery or ping scanning. Nmap offers many host discovery techniques, ranging from quick ARP requests to elaborate combinations of TCP, ICMP, and other types of probes. This phase is run by default, though you can skip it (simply assume all target IPs are online) using the -PN (no ping) option. To quit after host discovery, specify -sP -n. Host discovery is the subject of Chapter 3 [47] . Reverse-DNS resolution. Once Nmap has determined which hosts to scan, i t looks u p the reverse-DNS names of all hosts found online by the ping scan. Sometimes a host's name provides clues to its function, and names make reports more readable than providing only IP numbers. This step may be skipped with the -n (no resolution) option, or expanded to cover all target IPs (even down ones) with -R (resolve all). Name resolution is covered in Section 3.4, "DNS Resolution" [56] . Port scanning. This is Nmap's fundamental operation. Probes are sent, and the responses (or non-responses) to those probes are used to classify remote ports into states such as open, c l o s ed, or f i l t e red. That 12 1 .3. The Phases of an Nmap Scan brief description doesn't begin to encompass Nmap's many scan types, configurability of scans, and algorithms for improving speed and accuracy. An overview of port scanning is in Chapter 4 [73]. Detailed information on algorithms and command-line options are in Chapter 5 [95]. Port scanning is performed by default, though you can skip it and still perform some of the later traceroute and partial Nmap Scripting Engine phases by specifying their particular command-line options (such as --t raceroute and - - s c r ip t ) along with a ping scan (-sP). Version detection. If some ports are found to be open, Nmap may be able to determine what server software is running on the remote system. It does this by sending a variety of probes and matching the responses against a database of thousands of known service signatures. Version detection is enabled by the -sV option. It is fully described in Chapter 7 [ 1 45]. OS detection. If requested with the -0 option, Nmap proceeds to OS detection. Different operating systems implement network standards in subtly different ways. By measuring these differences it is often possible to determine the operating system running on a remote host. Nmap matches responses to a standard set of probes against a database of more than a thousand known operating system responses. OS detection is covered in Chapter 8 [ 1 7 1 ] . Traceroute. Nmap contains a n optimized traceroute implementation, enabled by the - - t ra c e r o u t e option. It can find the network routes to many hosts in parallel, using the best available probe packets as determined by Nmap's previous discovery phases. Traceroute usually involves another round of reverse-DNS resolution for the intermediate hosts. More information is found in Section 1 5.4, "Host Discovery" [378] . Script scanning. The Nmap Scripting Engine (NSE) uses a collection of special-purpose scripts to gain even more information about remote systems. NSE is powered by the Lua programming language and a standard library designed for network information gathering. Among the facilities offered are advanced version detection, notification of service vulnerabilities, and discovery of backdoors and other malware. NSE is a large subject, fully discussed in Chapter 9 [205]. NSE is not executed unless you request it with options such as - - script or - s c. Output. Finally, Nmap collects all the information it has gathered and writes it to the screen or to a file. Nmap can write output in several formats. Its default, human-readable format (interactive format) is usually presented in this book. Nmap also offers an XML-based output format, among others. The ins and outs of output are the subject of Chapter 13 [337]. As already discussed, Nmap offers many options for controlling which of these phases are run. For scans of large networks, each phase is repeated many times since Nmap deals with the hosts in smaller groups. It scans each group completely and outputs those results, then moves on to the next batch of hosts. 1 .4 . Legal Issues When used properly, Nmap helps protect your network from invaders. But when used improperly, Nmap can (in rare cases) get you sued, fired, expelled, jailed, or banned by your ISP. Reduce your risk by reading this legal guide before launching Nmap. 1 .4. Legal Issues 13 1 .4.1 . Is U nauthorized Port Scanning a Crime? The l�gal ramifications of scanning networks with Nmap are complex and so controversial that third-r:arty _ orgamzat1ons have even printed T-shirts and bumper stickers promulgating opinions on the matter 3, as shown in Figure 1.3. The topic also draws many passionate but often unproductive debates and flame wars. If you ever participate in such discussions, try to avoid the overused and ill-fitting analogies to knocking on someone's home door or testing whether his door and windows are locked. Figure 1.3. Strong opinions on port scanning legality and morality While I agree with the sentiment that port scanning should not be il legal, it is rarely wise to take legal advice from a T-shirt. Indeed, taking it from a software engineer and author is only slightly better. Speak to a competent lawyer within your jurisdiction for a better understanding of how the law applies to your particular situation. With that important disclaimer out of the way, here is some general information that may prove helpful. The best way to avoid controversy when using Nmap is to always secure written authorization from the target network representatives before initiating any scanning. There is still a chance that your ISP will give you trouble if they notice it (or if the target administrators accidentally send them an abuse report), but this is usually easy to resolve. When you are performing a penetration test, this authorization should be in the Statement of Work. When testing your own company, make certain that this activity clearly falls within your job description. Security consultants should be familiar with the excellent Open Source Security Testing 14 Methodology Manual (OSSTMM) , which provides best practices for these situations. 1 3These are from the now-defunct AmericanSushi.Com. 14 hllp:llwww.osstmm.org/ 14 I .4. Legal Issues While civil and (especially) criminal court cases are the nightmare scenario for Nmap users, these are very rare. After all, no United States federal laws explicitly make port scanning illegal. A much more frequent occurrence is that the target network will notice a scan and send a complaint to the network service provider where the scan initiated (your ISP). Most network administrators do not seem to care or notice the many scans bouncing off their networks daily, but a few complain. The scan source ISP may track down the user corresponding to the reported IP address and time, then chide the user or even kick them off the service. Port scanning without authorization is sometimes against the provider's acceptable use policy (AUP). For example, the AUP for the huge cable-modem ISP Comcast presently says 15 : Network probing or port scanning tools are only permitted when used in conjunction with a residential home network, or if explicitly authorized by the destination host and/or network. Unauthorized port scanning, for any reason, is strictly prohibited. Even if an ISP does not explicitly ban unauthorized port scanning, they might claim that some "anti-hacking" provision applies. Of course this does not make port scanning illegal. Many perfectly legal and (in the United States) constitutionally protected activities are banned by ISPs. For example, the AUP quoted above also prohibits users from transmitting, storing, or posting "any information or material which a reasonable person could deem to be objectionable, offensive, indecent, pornographic, ... embarrassing, distressing, vulgar, hateful, racially or ethnically offensive, or otherwise inappropriate, regardless of whether this material or its dissemination is unlawful." In other words, some ISPs ban any behavior that could possibly offend or annoy someone. Indiscriminate scanning of other people's networks/computers does have that potential. If you decide to perform such controversial scanning anyway, never do it from work, school, or any other service provider that has substantial control over your well-being. Use a dialup or commercial broadband provider instead. Losing your DSL connection and having to change providers is a slight nuisance, but it is immeasurably preferable to being expelled or fired. While legal cases involving port scanning (without follow-up hacking attacks) are rare, they do happen. One of the most notable cases involved a man named Scott Moulton who had an ongoing consulting contract to maintain the Cherokee County, Georgia emergency 91 1 system. In December 1999, he was tasked with setting up a router connecting the Canton, Georgia Police Department with the E91 1 Center. Concerned that this might jeopardize the E91 I Center security, Scott initiated some preliminary port scanning of the networks involved. In the process he scanned a Cherokee County web server that was owned and maintained by a competing consulting firm named VC3. They noticed the scan and emailed Scott, who replied that he worked for the 91 1 Center and was testing security. VC3 then reported the activity to the police. Scott lost his E91 1 maintenance contract and was arrested for allegedly violating the Computer Fraud and Abuse Act of America 16 Section l030(a)(5)(B) • This act applies against anyone who "intentionally accesses a protected computer without authorization, and as a result of such conduct, causes damage" (and meets other requirements). The damage claimed by VC3 involved time spent investigating the port scan and related activity. Scott sued VC3 for defamation, and VC3 countersued for violation of the Computer Fraud and Abuse Act as well as the Georgia Computer Systems Protection Act. The civil case against Scott was dismissed before trial, implying a complete lack of merit. The ruling made many Nmap users smile: "Court holds that plaintiffs act of conducting an unauthorized port scan and throughput test of defendant's servers does not constitute a violation of either the Georgia Computer IS 16 ht1p:l/www.comcast.net/termsluse.jsp ht1p:llwww4.law.cornell.edu/11scode/J81J030.html 1 .4. Legal Issues 15 Systems Protection Act or the Computer Fraud and Abuse Act."-Civ. Act. No. I :00-CV-434-TWT (N.D. Ga. November 6, 2000) This was an exciting victory in the civil case, but Scott still had the criminal charges hanging over his head. Fortunately he kept his spirits high, sending the following note 17 to the nmap-hackers mailing list: I am proud that I could be of some benefit to the computer society in defending and protecting the rights of specialists in the computer field, however it is EXTREMELY costly to support such an effort, of which I am not happy about. But I will continue to fight and prove that there is nothing illegal about port scanning especially when I was just doing my job. Eventually, the criminal court came to the same conclusion and all charges were dropped. While Scott was vindicated in the end, he suffered six-figure legal bills and endured stressful years battling through the court system. The si lver lining is that after spending so much time educating his lawyers about the technical issues involved, Scott started a successful forensics services company 18 • While the Moulton case sets a good example (if not legal precedent), different courts or situations could still lead to worse outcomes. Remember that many states have their own computer abuse laws, some of which can arguably make even pinging a remote machine without authorization illegal 19• 20 Laws in other nations obviously di ffer as well. For example, A 17-year-old youth was convicted in Finland of attempted computer intrusion for simply port scanning a bank. He was fined to cover the target's investigation expenses. The Moulton ruling might have differed if the VC3 machine had actually crashed and they were able to justify the $5,000 damage figure required by the act. 21 At the other extreme, an Israeli judge acqui tted Avi Mizrahi in early 2004 for vulnerability scanning the Mossad secret service. Judge Abraham Tennenbaum even praised Avi as follows: In a way, Internet surfers who check the vulnerabilities of Web sites are acting in the public good. If their intentions are not malicious and they do not cause any damage, they should even be praised. 22 23 In 2007 and 2008, broad new cybercrime laws took effect in Germany and England . These laws are meant to ban the distribution, use, and even possession of "hacking tools". For example, the UK amendment to the Computer Misuse Act makes it illegal to "supply or offer to supply, believing that it is likely to be used to commit, or to assist in the commission of [a Computer Misuse Act violation]". These laws have already led some security tool authors to close shop or move their projects to other countries. The problem is that most security tools can be used by both ethical professionals (white-hats) to defend their networks and black-hats to attack. These dangerous laws are based on the tool author or user's intent, which is subjective and hard to divine. Nmap was designed to help secure the Internet, but I'd hate to be arrested and forced to defend my intentions to a judge and jury. These laws are unlikely to affect tools as widespread and popular 17 18 http:l/seclists.orglnmap-hackers/200/10026.html http://www.forensicstrategy.com/ 19 An excellent paper on this topic by lawyer Ethan Preston is available at http:/lgrove.ujl.edul-techlawlvol6/issuel/pres1011.html. He has also written an excellent paper relating to the legal risks of publishing security information and exploits at http://www.mcandl.com/computer-security.html. http://insecure.orglstflfin.html 2 1 http://www.theregister.co.11k/2004/03/0Ilmossad_website_hacker_walksJree/ 22 http://www.beskerming.comlcomme11tary/2007/08//21249/German_Sec11rity_Professionals_i11_1he_Mist 2 3 http://www.theregister.co. uk/2008101102/hacker_toll_ban_guidance/ 20 16 1 .4. Legal Issues as Nmap, but they have had a chil ling effect on smaller tools and those which are more commonly abused by computer criminals (such as exploitation frameworks). Regardless of the legal status of port scanning, ISP accounts will continue to be terminated if many complaints are generated. The best way to avoid ISP abuse reports or civil/criminal charges is to avoid annoying the target network administrators in the first place. Here are some practical suggestions: • Probably at least 90% of network scanning is non-controversial. You are rarely badgered for scanning your own machine or the networks you administer. The controversy comes when scanning other networks. There are many reasons (good and bad) for doing this sort of network exploration. Perhaps you are scanning the other systems in your dorm or department to look for publicly shared files (FTP, SMB, WWW, etc.). Or maybe you are just trying to find the IP of a certain printer. You might have scanned your favorite web site to see if they are offering any other services, or because you were curious what OS they run. Perhaps you are just trying to test connectivity, or maybe you wanted to do a quick security sanity check before handing off your credit card details to that e-commerce company. You might be conducting Internet research. Or are you performing initial reconnaissance in preparation for a break-in attempt? The remote administrators rarely know your true intentions, and do sometimes get suspicious. The best approach is to get permission first. I have seen a few people with non-administrative roles land in hot water after deciding to "prove" network insecurity by launching an intrusive scan of the entire company or campus. Administrators tend to be more cooperative when asked in advance than when woken up at 3 A M by an IDS alarm claiming they are under massive attack. So whenever possible, obtain written authorization before scanning a network. Adrian Lamo would probably have avoided jail if he had asked the New York Times to test their security rather than telling reporters about the flaws afterward. Unfortunately they would likely have said no. Be prepared for this answer. • Target your scan as tightly as possible. A ny machine connected to the Internet is scanned regularly enough that most administrators ignore such Internet white noise. B ut scanning enough networks or executing very noisy/intrusive scans increases the probability of generating complaints. So if you are only looking for web servers, specify -p8 0 rather than scanning all 65,536 TCP ports on each machine. If you are only trying to find available hosts, do an Nmap ping scan rather than full port scan. Do not scan a CIDR /16 (65K hosts) when a /24 netblock suffices. The random scan mode now takes an argument specifying the number of hosts, rather than running forever. So consider - i R 1 0 0 0 rather than - i R 1 0 0 0 0 if the former is sufficient. Use the default timing (or even -T p o l i t e) rather than -T i n s ane. Avoid noisy and relatively intrusive scans such as version detection (-sV). Similarly, a SYN scan ( - s s ) is quieter than a connect scan (-sT) while providing the same information and often being faster. • A s noted previously, do not do anything controversial from your work or school connections. Even though your intentions may be good, you have too much to lose if someone in power (e.g. boss, dean) decides you are a malicious cracker. Do you really want to explain your actions to someone who may not even understand the terms packet or port scanner? Spend $40 a month for a dial up, shell, or residential broadband account. Not only are the repercussions less severe if you offend someone from such an account, but target network administrators are less likely to even bother complaining to mass-market providers. A lso read the relevant AUP and choose a provider accordingly. If your provider (like Comcast discussed above) bans any unauthorized port scanning and posting of "offensive" material, do not be surprised if you are kicked off for this activity. In general, the more you pay to a service provider the more accommodating they are. A T L provider is highly unlikely to yank your connection without notice because someone reported being port scanned. A dialup or residential DSUcable provider very well might. This can happen even when the scan was forged by someone else. 1 .4. Legal Issues 17 • Nmap offers many options for stealthy scans, including source-IP spoofing, decoy scanning, and the more recent idle scan technique. These are discussed in the IDS evasion chapter. But remember that there is always a trade-off. You are harder to find if you launch scans from an open WAP far from your house, with 17 decoys, while doing subsequent probes through a chain of nine open proxies. But if anyone does track you down, they will be mighty suspicious of your intentions. • Always have a legitimate reason for performing scans. An offended administrator might write to you first (or your ISP might forward his complaint to you) expecting some sort of justification for the activity. In the Scott Moulton case discussed above, VC3 first emailed Scott to ask what was going on. If they had been satisfied with his answer, matters might have stopped there rather than escalating into civil and criminal litigation. Groups scanning large portions of the Internet for research purposes often use a reverse-DNS name that describes their project and run a web server with detailed information and opt-out forms. Also remember that ancillary and subsequent actions are often used as evidence of intent. A port scan by itself does not always signify an attack. A port scan followed closely by an ITS exploit, however, broadcasts the intention loud and clear. This is important because decisions to prosecute (or fire, expel, complain, etc.) are often based on the whole event and not just one component (such as a port scan). One dramatic case involved a Canadian man named Walter Nowakowski, who was apparently the first person to be charged in Canada with theft of communications (Canadian Criminal Code Section S.342. 1 ) for accessing the Internet through someone's unsecured Wi-Fi network. Thousands of Canadian "war drivers" do this every 24 day, so why was he singled out? Because of ancillary actions and i ntent. He was allegedly caught driving the wrong way on a one-way street, naked from the waist down, with laptop in hand, while downloading child pornography through the aforementioned unsecured wireless access point. The police apparently considered his activity egregious enough that they brainstormed for relevant charges and tacked on theft of communications to the many child pornography-related charges. Similarly, charges involving port scanning are usually reserved for the most egregious cases. Even when paranoid administrators notify the police that they have been scanned, prosecution (or any further action) is exceedingly rare. The fact that a 91 1 emergency service was involved is likely what motivated prosecutors in the Moulton case. Your author has scanned hundreds of thousands of Internet hosts while writing this book and received no complaints. To summarize this whole section, the question of whether port scanning is legal does not have a simple answer. I cannot unequivocally say "port scanning is never a crime", as much as I would like to. Laws differ dramatically between jurisdictions, and cases hinge on their particular details. Even when facts are nearly identical, different judges and prosecutors do not always interpret them the same way. I can only urge caution and reiterate the suggestions above. For testing purposes, you have permission to scan the host s c a nme . nmap . org. You may have noticed that it was used in several examples already. Note that this permission only includes scanning via Nmap and not testing exploits or denial of service attacks. To conserve bandwidth, please do not initiate more than a dozen scans against that host per day. If this free scanning target service is abused, it will be taken down and Nmap will report F a i l e d t o r e s o l ve g i ve n ho s t name / I P : s ca nme . nmap . org. 24 http:llwww.ctv.calservlet/A rtic/eNewslstory/CTVNews/1069439746264_64848946/ 18 1 .4. Legal Issues 1 .4.2. Can Port Sca n n i ng Crash the Target Computer/Networks? Nmap does not have any features designed to crash target networks. It usually tries to tread lightly. For example, Nmap detects dropped packets and slows down when they occur in order to avoid overloading the network. Nmap also does not send any corrupt packets. The IP, TCP, UDP, and ICMP headers are always appropriate, though the destination host is not necessarily expecting the packets. For these reasons, no application, host, or network component should ever crash based on an Nmap scan. If they do, that is a bug in the system which should be repaired by the vendor. Reports of systems being crashed by Nmap are rare, but they do happen. Many of these systems were probably unstable in the first place and Nmap either pushed them over the top or they crashed at the same time as an Nmap scan by pure coincidence. In other cases, poorly written applications, TCP/IP stacks, and even operating systems have been demonstrated to crash reproducibly given a certain Nmap command. These are usually older legacy devices, as newer equipment is rarely released with these problems. Smart companies use Nmap and many other common network tools to test devices prior to shipment. Those who omit such pre-release testing often find out about the problem in early beta tests when a box is first deployed on the Internet. It rarely takes long for a given IP to be scanned as part of Internet white noise. Keeping systems and devices up-to-date with the latest vendor patches and firmware should reduce the susceptibility of your machines to these problems, while also improving the security and usability of your network. In many cases, finding that a machine crashes from a certain scan is valuable information. After all, attackers can do anything Nmap can do by using Nmap itself or their own custom scripts. Devices should not crash from being scanned and if they do, vendors should be pressured to provide a patch. In some usage scenarios, detecting fragile machines by crashing them is undesirable. In those cases you may want to perform very light scanning to reduce the risk of adverse effects. Here are a few suggestions: • Use SYN scan ( - s s ) instead of connect scan ( - s T). User-mode applications such as web servers can rarely even detect the former because it is all handled in kernel space (some older Linux kernels are an exception) and thus the services have no excuse to crash. • Version scanning (- sV) risks crashing poorly written applications. Similarly, some pathetic operating systems have been reported to crash when OS fi ngerprinted (-0). Omit these options for particularly sensitive environments or where you do not need the results. • • Using -T2 or slower (-T l , - T O ) timing modes can reduce the chances that a port scan will harm a system, though they slow your scan dramatically. Older Linux boxes had an identd daemon that would block services temporarily if they were accessed too frequently. This could happen in a port scan, as well as during legitimate high-load situations. Slower timing might help here. These slow timing modes should only be used as a last resort as they can slow scans by an order of magnitude or more. Limit the number of ports and machines scanned to the fewest that are required. Every machine scanned has a minuscule chance of crashing, and so cutting the number of machines down improves your odds. Reducing the number of ports scanned reduces the risks to end hosts as well as network devices. Many NAT/firewall devices keep a state entry for every port probe. Most of them expire old entries when the table fills up, but occasional (pathetic) implementations crash instead. Reducing the ports/hosts scanned reduces the number of state entries and thus might help those sorry devices stay up. 1 .4. Legal Issues 19 1 .4.3. N map Copyright While N �ap i s open source, i t still has a copyright license that must be respected. A s free software, Nmap also cames no warranty. These issues are covered in much greater detail in Section 1 5 . 19, "Legal Notices" (4 1 2) . Companies wishing to bundle and use Nmap within proprietary software and appliances are especially encouraged to read this section so they don't inadvertently violate the Nmap license. Fortunately the Nmap Project sells commercial redistribution licenses for companies which need one. 1 .5 . The H istory and Futu re of Nmap Many ancient and well loved security tools, such as Netcat, tcpdump, and John the Ripper, haven't changed much over the years. Others, including Nessus, Wireshark, Cain and Abel, and Snort have been under constant development since the day they were released. Nmap is in that second category. It was released as a simple Linux-only port scanner in 1 997. Over the next 1 0+ years it sprouted a myriad of valuable features, including OS detection, version detection, the Nmap Scripting Engine, a Windows port, a graphical user interface, and more. This section provides a timeline of the most important events over a decade of Nmap history, followed by brief predictions on the future of Nmap. For all significant Nmap changes (thousands of them), read the Nmap Changelog 25 . Old releases of Nmap can be found at http://nmap. org/distl, and ancient versions at http://nmap. org/dist-old!. • • • 26 Nmap is first released in Phrack Magazine Issue 5 1 , article I t . It doesn't have a version number because new releases aren't planned. Nmap is about 2,000 lines long, and compilation is as simple as gee -06 -o nmap nmap.c -Im. September 1, 1997 - September 5, 1997 Due to popular demand, a slightly modified version of the Phrack code is released, calling itself version 1 .25. The gzipped tarball is 28KB. Version 1 .26 (48KB) is released 19 days later. - January 1 1, 1998 DataHaven Project Insecure.Org is registered and Nmap moves there from its previous home at the ISP. - 27 Renaud Deraison writes to i nform me that he is writing a security scanner, and asks if he can use some Nmap source code. Of course I say yes. Nine days later he sends me a pre-release version of Nessus, noting that it "is designed for sysadmi ns, not 3133t H4ck3rZ". March 14, 1998 • - • September 1, 1998 Inspired by Nmap's first anniversary, I begin work on adding remote OS detection for the upcoming Nmap 2.00. On October 7 I release the first private beta version to a handful of top Nmap developers. We quietly work on this for several months. - • fi December 12, 1998 Nmap version 2.00 is publicly released, introducing Nma OS detection for the first time. An article describing the techniques was released in Phrack 54, Article 9 8. By this point Nmap is broken up i nto many files, consists of about 8,000 lines of code, is kept in a private CVS revision control system, and the tarball size is 275KB. The nmap-hackers mailing list is started, and later grows to more than 55,000 members. - 25 http://nmap.org/changelog.html http:llnmap.orglp51-l I.html 27 http://www.dhp.com 28 http://nmap.org/phrack54-09. txt 26 20 1 .5 . The History and Future of Nmap • April 11, 1999 - Nmap 2.1 I BETA 1 is released. This is the first version to contain a graphical user interface as an alternative to the traditional command-line usage. The bundled Unix-only GUI named NmapFE was originally written by Zach Smith. Some people like it, but most prefer command-line execution. Nmap 2.50 is released 29. By this point the tarball has grown to 461 KB. This release includes timing modes such as -T aggre s s ive, direct SunRPC scanning, and Window and ACK scan methods. • April 28, 2000 - 30 to the nmap-dev list describing a new "protocol scan" he has developed for Nmap, and he even incl udes a patch. This is so cool that I release31 Nmap 2.54BETA 1 with his patch less than 12 hours later. • May 28, 2000 - Gerhard Rieger sends a message 32 as the first official version to compile and run on Microsoft Windows. The Windows porting work was done by Ryan Permeh and Andy Lutomirski. • December 7, 2000 - Nmap 2.54BETA 16 is released The Nmap IP ID idle scan is introduced with Nmap 2.54BETA26. A paper describing the technique is released concurrently. This extremely cool (though not always practical) scan technique is described in Section 5. 10, "TCP Idle Scan (-sI)" [ 1 1 7]. • July 9, 2001 - • July 25, 2002 - I quit my job at Netscape/AOL and start my dream job working on Nmap ful l time. 33 • July 31, 2002 - Nmap 3.00 is released . The tarball is 922K. This release includes Mac OS X support, XML output, and uptime detection. • August 28, 2002 - Nmap is converted from C to C++ and IPv6 supported is added as part of the Nmap 34 3.IOALPHA 1 release . • May 15, 2003 - Nmap is featured in the movie The Matrix Reloaded, where Trinity uses it (followed by a real SSH exploit) to hack a power station and save the world. This leads to more publicity for Nmap than it had ever seen before or has seen since then. Details and screen shots are available at http://nmap. o rglmovies. html. • July 21, 2003 - I finish a first implementation of Nmap service/version detection (Chapter 7, Service and Application Version Detection [ 1 45]) and release it to a couple dozen top Nmap developers and users as Nmap 3.40PVT 1 . That is followed up by 16 more private releases over the next couple months as we improve the system and add signatures. • September 16, 2003 - Nmap service detection is finally released 35 publicly as part of Nmap 3.45. A detailed paper is released concurrently. 36 • February 20, 2004 - Nmap 3.50 is released . The tarball is now 1 ,57 1 KB . SCO Corporation is banned from redistributing Nmap because they refuse to comply with the GPL. They have to rebuild their Caldera 29 h11p://seclists.orglnmap-hackers/2000I0/40.html h1tp:l/seclists.orglnmap-hackers/2000/0217.html 31 h1tp://seclists.orglnmap-hackers/2000I02!9.html 32 http:llseclists.orglnmap-dev/2000/q4/00/3.html 33 h11p:/linsec11re.orglstf/Nmap-3.00-Release.html 34 http://seclists.orglnmap-dev/2002/q31004/.html 35 http:llseclists.orglnmap-hackers/200310030.html 36 http://insecure.org/stf/Nmap-3.50-Release.html 30 1 .5 . The History and Future of Nmap 21 release ISOs to remove Nmap. This release includes the packet tracing and UDP ping options. It also includes the OS classification system which classifies each of the hundreds of detected operating systems by vendor name, operating system name, OS generation, and device type. 37 August 31, 2004 - The core Nmap port scanning engine is rewritten for Nmap 3.70 • The new engine, named ul t r a_scan features dramatically improved algorithms and parallelization support to improve both accuracy and speed. The differences are particularly dramatic for hosts behind strict firewalls. • • June 25, 2005 - Google sponsors 10 colle�e and graduate students to work on Nmap full time for the summer as part of Google's Summer of Code3 initiative. Projects include a second generation OS detection system (Zhao Lei), a new cross-platform GUI named Umit (Adriano Monteiro Marques), and many other cool projects described at http:!!seclists.orglnmap-hackers/200510008.html. 39 September 8, 2005 - Nmap gains raw ethernet frame sending support with the release of version 3.90 . This allows for ARP scanning (see Section 3.6.6, "ARP Scan (-PR)" [64]) and MAC address spoofing as well as evading the raw IP packet ban introduced by Microsoft in Windows XP SP2. • • January 31, 2006 - Nmap 4.00 is released40. The tarball is now 2,388KB. This release includes runtime interaction to provide on-demand completion estimates, a Windows executable installer, NmapFE updates to support GTK2, and much more. • • • • May 24, 2006 - Google sponsors 10 more N map summer developers as part of their SoC program. Zhao and Adriano return as part of 2006 SoC to further develop their respective projects. Diman Todorov is sponsored to help develop the Nmap Scripting Engines. These and seven other talented students and their projects are described at http:llseclists.orglnmap-hackers/200610009. html. June 24, 2006 - After two years of development and testing, the 2nd generation OS detection system is 41 integrated into Nmap 4.20ALPHA 1 • This new system is based on everything we've learned and the new ideas we've conceived since the I st generation system debuted 8 years earlier. After a bit of time to grow the DB, the new system proves much more accurate and granular than the old one. It is described in Chapter 8, Remote OS Detection [ 1 7 1 ] . 42 December 10, 2006 The Nmap Scripting Engine is released as part of Nmap 4.21 ALPHA I . NSE allows users to write (and share) simple scripts to automate a wide variety of networking tasks. The system is a huge success, and is described in Chapter 9, Nmap Scripting Engine [205). - 43 December 20, 2006 - Nmap's Subversion source code repository opens to the public . Until this time, only a handful of developers had access to the private source repository. Everyone else had to wait for releases. Now everyone can follow Nmap development day by day. There is even an nmap - s vn mailing list providing real-time change notification by email. Details are provided in Section 2.1.5, "Obtaining Nmap from the Subversion (SYN) Repository" [28). 37 http://seclists.org/nmap-hackers/2004/0010.html http://code.google.com/soc 39 http:llseclists.orglnmap-hackers/200510012.html 40 http://insec11re.org/stf/Nmap-4.00-Release.html 41 http://seclists.orglnmap-dev/2006/q2/0444.html 42 http://seclists.org/nmap-dev/2006/q4/0184.html 43 http://seclists.org/nmap-dev/2006/q4/0253.html 38 22 1 .5 . The History and Future of Nmap • May 28, 2007 - Google sponsors six summer Nmap developers as part of their SoC program. Meanwhile, Adriano's Umit GUI for Nmap is approved as an independent program for SoC sponsorship. Among the sponsored students was David Fifield, who continued long after the summer ended and became one of Nmap's top developers. The Nmap students and their projects are listed at http:/lseclists.org/nmap-hackers/200710003.html. Die Hard 4: Live Free or Die Hard is released in theaters. It includes a brief scene of hacker Matthew Farrel l (Justin Long) demonstrating his Nmap skills. Then he leaves his computer to join Bruce Willis in fighting a diabolical terrorist mastermind. One week later, The Bourne Ultimatum is released and also contains an Nmap scene! The CIA uses Nmap in this movie to hack a newspaper's mail server and read the email of a reporter they assassinated (nice guys)! Screen shots of Nmap movie cameos are all available on the Nmap movies page44 . • June 27, 2007 - • July 8, 2007 - The Umit graphical front end is improved and integrated into the Nmap 4.22SOC 1 release45 for testing. Umit is later renamed to Zenmap, and the venerable NmapFE GUI is removed. Zenmap is covered in Chapter 1 2, Zenmap GUI Users ' Guide [307]. 46 to celebrate Nmap's 10th anniversary ! • December 13, 2007 - Nmap 4.50 is released 47 • June 1, 2008 - Nmap 4.65 is released and includes, for the first time, an executable Mac OS X installer. The Nmap source tarball is now four megabytes. This release includes 41 NSE scripts, 1 ,307 OS fi ngerprints, and 4,706 version detection signatures. The Nmap project completes its fourth Summer of Code, with our highest success percentage ever (six out of seven sponsored students). They greatly improved Zenmap, the Nmap Scripting Engine, OS detection, and Neat, as described at http://seclists.org/nmap-dev/2008/q4/0193. html. • August 18, 2008 - 48 with almost 100 significant improvements over 4.68. These include the Zenmap network topology and scan aggregation features (see Chapter 1 2, Zenmap GUI Users' Guide [307]). It also includes port-frequency data from my Worldscan project, which I presented49 at Black Hat and Defcon in August. • September 8, 2008 - Nmap 4.75 is released While it is easy to catalogue the history of Nmap, the future is uncertain. Nmap didn't start off with any grand development plan, and most of the milestones in the preceding timeline were not planned more than a year in advance. Instead of trying to predict the shape of the Internet and networking way out in the future, I closely study where it is now and decide what will be most useful for Nmap now and in the near future. So 1 have no idea where Nmap will be 10 years from now, though I expect it to be as popular and vibrant as ever. The Nmap community is large enough that we will be able to guide Nmap wherever it needs to go. Nmap has faced curve balls before, such as the sudden removal of raw packet support in Windows XP SP2, dramatic changes in network filtering practices and technology, and the slow emergence of 1Pv6. Each of those required significant changes to Nmap, and we'll have to do the same to embrace or at least cope with networking changes in the future. 44 http://111nap.org/movies.html http://seclists.org/nmap-dev/20071q3/0030.html 46 http:llinsecure.orglstf/Nmap-4.50-Release.html 47 http://seclists.org/111nap-dev/2008/q2/0558.h11nl 48 http:llseclists.org/111nap-hackers/2008/0004.html 49 http://i11secure.org!presef!fations/ 45 1 .5 . The History and Future of Nmap 23 While the IO-year plan is up in the air, the coming year is easier to predict. As exciting as big new features are, they won't be a focus. None of us want to see Nmap get bloated and disorganized. So this will be a year of consolidation. The Zenmap and NSE systems are not as mature as the rest of Nmap, so improving these is a big priority. New NSE scripts are great because they extend Nmap's functionality without the stability risks of incorporating new source code into Nmap proper. Meanwhile, Zenmap needs usability and stability improvements, as well as better results visualization. Another focus is the Nmap web site, which will become more useful and dynamic. A web discussion system, Nmap demo site, and wiki are planned. Nmap may also grow in its ability to handle web scanning. When Nmap was first developed, different services were often provided as separate daemons identified by the port number they listen on. Now, many new services simply run over HTTP and are identified by a URL path name rather than port number. Scanning for known URL paths is similar in many ways to port scanning (and to the SunRPC scanning which Nmap has also done for many years). Nmap already does some web scanning using the Nmap Scripting Engine (see Chapter 9, Nmap Scripting Engine [205]), but it would be faster and more efficient if basic support was built into Nmap itself. Some of the coolest Nmap features in the past, such as OS detection and version scanning, were developed in secret and given a surprise release. You can expect more of these in coming years because they are so much fun ! 24 1 .5. The History and Future of Nmap C hapter 2. O btai n i ng , Comp i l i ng , I n stal ling , and Removing Nmap 2.1 . I ntrod uction Nmap can often be installed or upgraded with a single command, so don't let the length of this chapter scare you. Most readers will use the table of contents to skip directly to sections that concern them. This chapter describes how to install Nmap on many platforms, including both source code compilation and binary installation methods. Graphical and command-line versions of Nmap are described and contrasted. Nmap removal instructions are also provided in case you change your mind. 2.1 .1 . Testi ng Whether N map is Al ready I nstal led The first step toward obtaining Nmap is to check whether you already have it. Many free operating system distributions (including most Linux and BSD systems) come with Nmap packages, although they may not be i nstalled by default. On Unix systems, open a terminal window and try executing the command nmap vers i on If Nmap exists and is in your PATH, you should see output similar to that in Example 2. 1 . -- . Example 2.1. Checking for Nmap and determining its version number felix->nmap --ve r s ion Nmap ver s i on 4 . 76 ( http : / / nmap . or g fel i x - > lf Nmap does not exist on the system (or if your PATH is incorrectly set), an error message such as nmap : Command not found is reported. As the example above shows, Nmap responds to the command by printing its version number (here 4 . 7 6 ). Even if your system already has a copy of Nmap, you should consider upgrading to the latest version available from http://nmap.org/download.html. Newer versions often run faster, fix important bugs, and feature updated operating system and service version detection databases. A list of changes since the version already on your system can be found at http://nmap.org/changelog.html. Nmap output examples in this book may not match the output produced by older versions. 2.1 . 2 . Com mand-l i ne and G raph ical I nterfaces Nmap has traditionally been a command-line tool run from a Unix shell or (more recently) Windows command prompt. This allows experts to quickly execute a command that does exactly what they want without having to maneuver through a bunch of configuration panels and scattered option fields. This also makes Nmap easier to script and enables easy sharing of useful commands among the user community. One downside of the command-line approach is that it can be intimidating for new and infrequent users. Nmap offers more than a hundred command-line options, although many are obscure features or debugging controls that most users can ignore. Many graphical frontends have been created for those users who prefer 2.1. Introduction 25 a GUI interface. Nmap has traditionally included a simple GUI for Unix named NmapFE, but that was replaced in 2007 by Zenmap, which we have been developing since 2005. Zenmap is far more powerful and effective than NmapFE, particularly in results viewing. Zenmap's tab-based interface lets you search and sort results, and also browse them in several ways (host details, raw Nmap output, and ports/hosts). It works on Linux, Windows, Mac OS X, and other platforms. Zenmap is covered in depth in Chapter 1 2, Zenmap GUI Users' Guide [307]. The rest of this book focuses on command-line Nmap invocations. Once you understand how the command-line options work and can interpret the output, using Zenmap or the other available Nmap GUis is easy. Nmap's options work the same way whether you choose them from radio buttons and menus or type them at a command-line. 2.1 .3. Down load i ng N map Nmap.Org i s the official source for downloading Nmap source code and binaries for Nmap and Zenmap. Source code is distributed in bzip2 and gzip compressed tar files, and binaries are available for Linux (RPM format), Windows (NSIS executable installer) and Mac OS X ( dmg disk image). Find all of this at http://nmap.org/download.html. . 2.1 .4. Verifying the I nteg rity of N map Downloads It often pays to be paranoid about the integrity of files downloaded from the Internet. Popular packages such 1 as Sendmail (example ), OpenSSH (example2 ), tcpdump, Libpcap, BitchX, Fragrouter, and many others have been infected with malicious trojans. Software distributions sites at the Free Software Foundation, Debian, and SourceForge have also been successfully compromised. This has never happened to Nmap, but one should always be careful. To verify the authenticity of an Nmap release, consult the PGP detached signatures or cryptographic hashes (including SHA I and MD5) posted for the release in the Nmap signatures directory at http://nmap.org/dist/sigs/?C=M&O=D. The most secure verification mechanism is detached PGP signatures. As the signing key is never stored on production servers, even someone who successfully compromises the web server couldn't forge and properly sign a trojan release. While numerous applications are able to verify PGP signatures, I recommend GNU 3 Privacy Guard (GPG) . Nmap releases are signed with a special Nmap Project Signing Key, which can be obtained from the major keyservers or http://nmap. org/data/nmap_gpgkeys. txt. My key is included in that file too. The keys can be imported with the command gpg --import nmap_gpgkeys.txt. You only need to do this once, then you can verify all future Nmap releases from that machine. Before trusting the keys, verify that the fingerprints match the values shown in Example 2.2. 1 2 hup:/lcert.orgladvisories/CA-2002-28.html http://cert.org/advisories/CA-2002-24.html 3 http://www.gnupg.org/ 26 2. 1 . Introduction Example 2.2. Verifying the Nmap and Fyodor PGP Key Fingerprints flog-> gpg - - fi ngerpr i n t nmap f yodor pub 1 0 2 4D / 3 3 5 9 985F 2 0 0 5 - 0 4 - 2 4 Key f i ngerpr int = 886 1 D 0 5 7 C O D 7 DCEF E 7 3 0 9 9 6 C 1AF6 E C 5 0 3 3 5 9 9 8 5 F uid Fyodor < fyodor @ i nsecure . org> sub 2 0 4 8g / D 3 C 2 2 4 1 C 2 0 0 5 - 0 4 - 2 4 pub 1 0 2 4 D / 6 89 3 5 5 D O 2 0 0 5 - 0 4 - 2 4 Key f ingerprint = 4 3 6 0 6 6A8 9 A 7 9 8 4 2 5 FDAO E 3 F 8 O l AF 9 F 0 3 6 8 9 3 5 5 D O uid Nmap Pro j ec t S ign ing Key ( ht tp : / / in s ecure . or g / ) sub 2 0 4 8g/A50A6A9 4 2 0 0 5 - 0 4 - 2 4 For every Nmap package download file (e.g. nmap- 4 . 7 6 . t a r . b z 2 and nmap- 4 . 7 6 -w i n 3 2 . z i p), there is a corresponding file in the s i gs directory with . gpg . t x t appended to the name (e.g. nmap-4 . 7 6 . t a r . b z 2 . gpg . t x t ). This is the detached signature file. With the proper PGP key in your keyring and the detached signature file downloaded, verifying an Nmap release takes a single GPG command, as shown in Example 2.3. If the fi le has been tampered with, the results will look like Example 2.4. Example 2.3. Verifying PGP key fingerprints (Successful) flog> gpg --ve r i f y nmap- 4 . 76 . t ar . bz 2 . gpg . t x t nmap- 4 . 7 6 . t ar . bz 2 gpg : Signature made F r i 1 2 Sep 2 0 0 8 0 2 : 0 3 : 5 9 AM PDT u s ing DSA key I D 6 8 9 3 5 5D O gpg : Good s ignature from " Nmap Proj ect S i gning Key ( ht t p : / / www . in s ecure . or g / ) " Example 2.4. Detecting a bogus file flog> gpg --ver i f y nmap- 4 . 7 6 . ta r . bz 2 . gpg . txt nmap- 4 . 7 6 -hacked . ta r . bz 2 gpg : Signature made F r i 1 2 Sep 2 0 0 8 0 2 : 0 3 : 5 9 AM PDT u s ing DSA key I D 6 8 9 3 5 50 0 gpg : BAD s i gnature from " Nmap Pro j ect S igning K e y ( ht tp : / / www . insecure . or g / ) " While PGP signatures are the recommended validation technique, SHA l and MD5 (among other) hashes are made available for more casual validation. An attacker who can manipulate your Internet traffic in real time (and is extremely skilled) or who compromises Nmap.Org and replaces both the distribution file and digest file, could defeat this test. However, it can be useful to check the authoritative Nmap.Org hashes if you obtain Nmap from a third party or feel it might have been accidentally corrupted. For every Nmap package download file, there is a corresponding file in the s ig s directory with . d i g e s t . t x t appended to the name (e.g. nmap- 4 . 7 6 . t a r . b z 2 . d i ge s t . t x t ). An example is shown in Example 2.5. This is the detached signature file. The hashes from the digest file can be verified using common tools such as shal sum, md5sum, or gpg, as shown in Example 2.6, "Verifying Nmap hashes" [28]. 2.1. Introduction 27 Example 2.5. A typical Nmap release digest file f l og> cat s i gs / nmap- 4 . 7 6 . t g z . digest . t xt MD5 5 4 BS C9 E3 F 4 4C lA DD El 7D F6 8 1 7 0 EB 7C FE nmap- 4 . 7 6 . t g z : nmap- 4 . 7 6 . t g z : SHAl 4 3 7 4 C F 9 C A 8 8 2 2 C 2 8 5DE9 D O O E 8F6 7 0 6 D O BCFA A 4 0 3 nmap- 4 . 7 6 . tg z : RMD 1 6 0 AE 7 B 8 0EF 4CE6 DBAA 6 E 6 5 7 6 F 9 CA3 8 4 A 2 2 3 B 8 9 BD3A 5 2 4 D 4 7 9 E 7 1 7D 9 8 D O 2 FB OA4 2 B 9A4E6E52 4 0 2 7 C 9 B 6 1 D 8 4 3 F 9 5 nmap-4 . 7 6 . tg z : SHA2 2 4 D 4 1 9F87F nmap- 4 . 7 6 . tg z : SHA2 5 6 O E 9 6 0 E 0 5 5 3 E B 7 6 4 7 O C 8 5 1 7AO 0 3 8 0 9 2A3 9 6 9DB6 5 C BE2 3 C 0 3 F D 6 DAEF 1 A CDCC9 6 5 8 nmap- 4 . 7 6 . t g z : SHA3 8 4 D 5 2 9 1 7FD 9EE6EE6 2 F 5 F 4 5 6 BF E 2 4 5 6 7 5 D B6 EEEBC5 0A2 8 7B 2 7 3 CAA4F 5 0 B l 7 1 DC 2 3 F E 7 8 0 8A8 C5E3A 4 9}l.. 4A7 8ACBE A5AEED33 nmap- 4 . 7 6 . tg z : SHA5 1 2 8 2 6 C D 8 9 F 7 9 3 0A 7 6 5 C 9 F E 9 B 4 1 1 DAFD 1 1 3 2 C 8 8 3 8 5 7 2A3A 9 5 0 3 E 4 C l E 6 9 0 2 0A3 7 F C 8 3 7 5 6 4DC3 4 5 FF O C 9 7 E F 4 5ABE6 6 CEA49FF E 2 6 2B 4 0 3 A 5 2 F 4 ECE C 2 3 3 3 3AO 4 8 DEDA 6 6 Example 2.6. Verifying Nmap hashes f log> s h a l s um nmap- 4 . 7 6 . tg z 4 3 7 4 c f 9 ca 8 8 2 2 c 2 8 5d e 9d 0 0 e 8 f 6 7 0 6 d0bcfaa 4 0 3 nmap-4 . 7 6 . tg z f log> md5 sum nmap- 4 . 7 6 . t g z 5 4 b5c 9 e 3 f 4 4 c l adde l 7 d f 6 8 1 7 0 e b 7 c f e nmap- 4 . 7 6 . t g z f l og> g p g - -pr i n t -md s h a l nmap- 4 . 7 6 . tg z nmap- 4 . 7 6 . tg z : 4 3 7 4 CF9C A 8 8 2 2 C 2 8 5DE9 D O OE 8 F 6 7 0 6 D O BCFA A 4 0 3 While releases from Nmap.Org are signed as described in this section, certain Nmap add-ons, interfaces, and platform-specific binaries are developed and distributed by other parties. They have different mechanisms for establishing the authenticity of their downloads. 2.1 .5. Obta i n i ng N map from the Subversion (SVN) Repository In addition to regular stable and development releases, the latest Nmap source code is always available using the Subversion (SYN) revision control system4 . This delivers new features and version/OS detection database updates immediately as they are developed. The downside is that SYN head revisions aren't always as stable as official releases. So SYN is most useful for Nmap developers and users who need a fix which hasn't yet been formally released. SYN write access is strictly limited to top Nmap developers, but everyone has read access to the repository. Check out the latest code using the command svn co --username guest --password svn://svn.insecure.org/nmap/. Then you can later update your source code by typing svn u p i n your working directory. The "guest" username is required due to an svnserve authorization bug. " " While most users only follow the / nmap directory in svn (which pulls in / nba s e , / n s ock, and I z enmap on its own), there is one other interesting directory: / nmap-exp. This directory contains experimental Nmap branches which Nmap developers create when they wish to try new things without destabilizing Nmap proper. When developers feel that an experimental branch is ready for wider-scale testing, they will generally email the location to the nmap-dev mailing list. 4 http://subversion.tigris.org 28 2. 1 . Introduction Once Nmap is checked out, you can build it from source code just as you would with the Nmap tarball (described later in this chapter). If you would like real-time (or digested) notification and diffs by email when any changes are made to Nmap, sign up for the nmap-svn mailing l ist at http://cgi. insecure. org/mailmanllistinfo!nmap-svn. 2.2. U n ix Com pi lation and Instal lation from Sou rce Code While binary packages (discussed i n later sections) are available for most platforms, compilation and installation from source code is the traditional and most powerful way to install Nmap. This ensures that the latest version is available and allows Nmap to adapt to the library availability and directory structure of your system. For example, Nmap uses the OpenSSL cryptography libraries for version detection when available, but most binary packages do not include this functionality. On the other hand, binary packages are generally quicker and easier to install, and allow for consistent management (installation, removal, upgrading, etc.) of all packaged software on the system. Source installation is usually a painless process-the build system is designed to auto-detect as much as possible. Here are the steps required for a default install : I. Download the latest version o f Nmap in .tar.bz2 (bzip2 compression) or .tgz (gzip compression) format from http://nmap.org/download. html. 2. Decompress the downloaded tarball with a command such as: bzip2 -cd nmap- .tar.bz2 I tar xvf • With GNU tar, the simpler command tar xvjf nmap- .tar.bz2 does the trick. If you downloaded the .tgz version, replace bzip2 with gzip in the decompression command. 3. Change into the newly created directory: cd nmap- 4. Configure the build system: Jconfigure If the configuration succeeds, an ASCII art dragon appears to congratulate you on successful configuration and warn you to be careful, as shown in Example 2.7. 2.2. Unix Compilation and Installation from Source Code 29 Example 2.7. Successful configuration screen f l og - / nmap> . / conf i gure checking bui ld s y stem t ype . . . x 8 6_6 4 -unknown - l i n u x -gnu [ hu ndreds of l ines cut ] c on f i gure : creating . / con f i g . s t a t u s con f i g . s t a tu s : crea t i ng Make f i l e con f i g . s t a t u s : crea t i ng nsock_con f i g . h con f i g . s t a tu s : nsock_co n f i g . h i s unchanged ( ) /\ ( \ < \. < \ I < ) \ \ \ ) \ I ( \ \+ . x ( \ \/ (_' -----------I < o > \ \_ \+ 0 ( \ \ \ I +- . ( - ' . - <. VVVVVVV W V\ ( \I ( : <_ - < ( -- _AAAAAAA_A_/ I / . / . +\ . - I +-/ /_ \ ( _ ' I x I x _/ < \ I \ x I ( ' I I I I \ I I _/ I + I \/ ( _/ I \ NMAP I S A POWERFUL TOOL - - USE CAREFULLY AND RESPON S IBLY Conf igurat ion comp l e t e . Type make ( gmake on s ome * BSD machine s ) t o comp i l e . • __ __ _ _ _ _ _ _ _ __ _ _ _ __ . I 5. Build Nmap (and the Zenmap GUI if its requirements are met): make Note that GNU Make is required. On BSD-derived Unix systems, this is often installed as gmake. So if make returns a bunch of errors such as "Make f i l e , l i ne 1 : Need an ope rat or", try running gmake instead. 6. Become a privileged user for system-wide i nstall : su root This step may be skipped if you only have an unprivileged shell account on the system. In that case, you will likely need to pass the --pref i x option to con f i gure in step four as described in the next section. 7. Install Nmap, support files, docs, etc.: make install Congratulations! Nmap is now installed as / u s r I l o ca l / b i n / nmap ! Run it with no arguments for a quick help screen. As you can see above, a simple source compilation and install consists of little more than running ./configure;make;make install as root. However, there are a number of options available to configure that affect the way Nmap is built. 2.2.1 . Confi g u re Di rectives Most of the Unix build options are controlled by the con f i gure script, as used in step number four above. There are dozens of command-line parameters and environmental variables which affect the way Nmap is built. Run ./configure --help for a huge list with brief descriptions. These are not applicable to building Nmap on Windows. Here are the options which are either specific to Nmap or particularly important: 30 2.2. Unix Compilation and Installation from Source Code --pr e f i x = This option, which is standard to the configure scripts of most software, determines where Nmap and its components are installed. By default, the prefix is / u s r I l o c a l , meaning that nmap is installed in /usr I l o c a l / b i n , the man page (nmap . 1) is installed in / u s r I l o c a l /ma n / ma n l , and the data files (nmap-os -db, nmap- s e r v i c e s , nmap - s e r v i ce -probe s , etc.) are installed under /usr/loca l / s har e / nmap. If you only wish to change the path of certain components, use the options --bi ndir, --da t a d i r , and/or - -mand i r . An example usage of --pr e f i x would be to install Nmap in my account as an unprivileged user. I would run Jconfigure --prefix=< / home / fyodor>. Nmap creates subdirectories like / home / fyodor /ma n /man l in the install stage if they do not already exist. --wit hout - z enmap This option prevents the Zenmap graphical frontend from being installed. Normally the build system checks your system for requirements such as the Python scripting language and then installs Zenmap if they are all available. --wi th-open s s l = The version detection system and Nmap Scripting Engine are able to probe SSL-encrypted services using the free OpenSSL libraries. Normally the Nmap build system looks for these libraries on your system and include this capability if they are found. If they are in a location your compiler does not search for by defa u l t , b u t you s ti l l want t h e m to be u s e d , spec i fy --with-ope n s s l = . Nmap then looks in llibs for the OpenSSL libraries themselves and /include for the necessary header files. Specify --wi thout - ope n s s l to disable SSL entirely. --with - l i bpcap= Nmap uses the Libpcap library5 for capturing raw IP packets. Nmap normally looks for an existing copy of Libpcap on your system and uses that if the version number and platform is appropriate. Otherwise Nmap includes its own recent copy of Libpcap, which has been modified for improved Linux functionality. The specific changes are described in l i bpcap/NMAP_MO D I F I CAT I ON S in the Nmap source directory. Because of these Linux-related changes, Nmap always uses its own Libpcap by default on that platform. If you wish to force Nmap to link with your own Libpcap, pass the option --wit h - l i bpcap= to configure. Nmap then expects the Libpcap library to be in / l i b / l i bpcap . a and the include files to be in / i n c l ude. Nmap will always use the version of Libpcap included in its tarball if you specify --wi t h - l ibp c ap=i n c l uded. --with- l i bpcre = PCRE is a Perl-compatible regular expression library available from http://www.pcre.org. Nmap normally looks for a copy on your system, and then falls back to its own copy if that fails. If your PCRE library is not in your compiler's standard search path, Nmap probably will not find it. In that case you can tell Nmap where it can be found by specifying the option --wi t h - l i bp c r e = to configure. Nmap then expects the library files to be in / l i b and the include fi les to be in / i n c l ud e . In some cases, you may wish to use the PCRE libraries 5 http://www.tcpdwnp.org 2.2. Unix Compilation and Installation from Source Code 31 included with Nmap in preference to those already on your system. In that case, specify --wi t h - l ibpcre=i n c l uded. - -w i t h - l ibdne t = Libdnet is an excellent networking library that Nmap uses for sending raw ethernet frames. The version in the Nmap tree is heavily modified (particularly the Windows code), so the default is to use that included version . If you wish to use a version already installed on your system instead, specify - -wi t h - l i bdne t = . Nmap then expects the library files to be in / l i b and the include files to be in / i n c l ude. - -w i t h - l o c a l d i r s This simple option tells Nmap t o look i n / u s r / l o c a l / l i b and / u s r / l o c a l / i n c l ude for important library and header fi les. This should never be necessary, except that some people put such libraries in / u s r / l oc a l without configuring their compiler to find them. If you are one of those people, use this option. 2.2.2. If You Encou nter Compi lation Problems In a n ideal world, software would always compile perfectly (and quickly) on every system. Unfortunately, society has not yet reached that state of nirvana. Despite all our efforts to make Nmap portable, compilation issues occasionally arise. Here are some suggestions in case the source distribution compilation fails. Upgrade to the latest Nmap Check http://nmap. org/download. html to make sure you are using the latest version of Nmap. The problem may have already been fixed. Read the error message carefully Scroll up in the output screen and examine the error messages given when commands fail. It is often best to find the first error message, as that often causes a cascade of further errors. Read the error message carefully, as it could indicate a system problem such as low disk space or a broken compiler. Users with programming skills may be able to resolve a wider range of problems themselves. If you make code changes to fix the problem, please send a patch (created with diff -uw ) and any details about your problem and platform to nmap-dev as described in Section 15.17, "Bugs" [4 1 1 ] . Integrating the change into the base Nmap distribution allows many other users to benefit, and prevents you from having to make the changes with each new Nmap version. Ask Google and other Internet resources Try searching for the exact error message on Google or other search engines. You might also want to browse recent activity on the Nmap development (nmap-dev) list-archives and a search interface are available at http://seclists.org. Ask nmap-dev If none of your research leads to a solution, try sending a report to the Nmap development (nmap-dev) mailing list, as described in Section 15. 17, "Bugs" [41 1 ] . Consider binary packages Binary packages of Nmap are available on most platforms and are usually easy to install. The downsides are that they may not be as up-to-date and you lose some of the flexibility of self-compilation. Later 32 2.2. Unix Compilation and Installation from Source Code sections of this chapter describe how to find binary packages on many platforms, and even more are available via Internet searching. Obviously you should only install binary packages from reputable sources. 2.3. Linux Distributions Linux is the most popular platform for running Nmap. I n one user survey, 86% said that Linux was at least one of the platforms on which they run Nmap. The first release of Nmap in 1 997 only ran on Linux. Linux users can choose between a source code install or using binary packages provided by their distribution or Insecure.Org. The binary packages are generally quicker and easier to install, and are often slightly customized to use the distribution's standard directory paths and such. These packages also allow for consistent management in terms of upgrading, removing, or surveying software on the system. A downside is that packages created by the distributions are necessarily behind the Nmap.Org source releases. Most Linux distributions (particularly Debian and Gentoo) keep their Nmap package relatively current, though a few are way out of date. Choosing the source install allows for more flexibility in determining how Nmap is built and optimized for your system. To build Nmap from source, see Section 2.2, "Unix Compilation and Installation from Source Code" [29]. Here are simple package instructions for the most common distributions. 2.3.1 . RPM-based Distri butions {Red Hat, Mand rake, SUSE, Fedora) I build RPM packages for every release of Nmap and post them to the Nmap download page at http://nmap.org/download. html. I build two packages : The nmap package contains just the command-line executable and data files, while the z e nmap package contains the optional Zenmap graphical frontend (see Chapter 1 2, Zenmap GUI Users ' Guide [307]). The z e nmap package requires that the nmap package be installed first. One down side to installing the RPMs rather than compiling from source is that the RPMs don't support OpenSSL for version detection and Nmap Scripting Engine probing of SSL services. Installing via RPM is quite easy-it even downloads the package for you when given the proper URLs. The following example downloads and installs Nmap 4.68, including the frontend. Of course you should use the latest version at the download site above instead. Any existing RPM-installed versions are upgraded. Example 2.8 demonstrates this installation process. Example 2.8. Installing Nmap from binary RPMs i rp� -vhU http : / / nmap . or g / d i s t / nmap- 4 . 6 8 - 1 . i 3 8 6 . rpm Retrieving http : / / nmap . or g / di s t / nmap- 4 . 6 8 - 1 . i 3 8 6 . rpm Preparing . . . ########################################### l : nmap ########################################### t rpm -vhU http : / / nmap . or g / di s t / z e nmap- 4 . 6 8 - l . noarch . rpm Retrieving http : / / nmap . or g / d i s t / z enmap- 4 . 6 8 - 1 . no a r ch . rpm Preparing . . . # # # #################### # # # # # # # # # # # # # # # # # # # # l : zenmap ########################################### (100%] ( 100%) ( 100%] ( 100%] As the fi lenames above imply, these binary RPMs were created for normal PCs (x86 architecture). I also distribute x86_64 binaries for 64-bit Linux users. These binaries won't work for the relatively few Linux users on other platforms such as SPARC, Alpha, or PowerPC. They also may refuse to install if your library 2.3. Linux Distributions 33 versions are sufficiently different from what the RPMs were initially built on. One option in these cases would be to find binary RPMs prepared by your Linux vendor for your specific distribution. The original install CDs or DVD are a good place to start. Unfortunately, those may not be current or available. Another option is to install Nmap from source code as described previously, though you lose the binary package maintenance consistency benefits. A third option is to build and install your own binary RPMs from the source RPMs distributed from the download page above. Example 2.9 demonstrates this technique with Nmap 4.68. Example 2.9. Building and installing Nmap from source RPMs rpmbu i ld --rebu i l d http : / / nmap . or g / di s t / nmap- 4 . 6 8 - l . s r c . rpm [ hundreds of l i nes c u t J Wrot e : / h ome / fyodor / rpmd i r / RPMS / i 3 8 6 / nmap- 4 . 6 8 - l . i 3 8 6 . rpm [ cut J > > SU Pas sword : # rpm -vhU / home / fyodor / rpmd i r / RPMS / i 3 8 6 / nmap- 4 . 6 8 - l . i 3 8 6 . rpm Prepa r i n g . . . ########################################### [ 100%] l : nmap ########################################### [ 100%] # I t i s not necessary to rebuild Zenmap i n this fashion because the Zenmap RPM i s architecture-independent ("noarch"). For that reason there are no Zenmap source RPMs. Removing RPM packages is as easy as rpm -e nmap zenmap. 2.3.2. U pdating Red Hat, Fedora, Mandrake, and Yel low Dog Linux with Yu m The. Red Hat, Fedora, Mandrake, and Yellow Dog Linux distributions have an application named Yum which manages software installation and updates from central RPM repositories. This makes software installation and updates trivial. Since distribution-specific Yum repositories are normally used, you know the software has already been tested for compatibility with your particular distribution. Most distributions do maintain Nmap in their Yum repository, but they don't always keep it up to date. This is particularly problematic if you (like most people) don't always quickly update to the latest release of your distribution. If you are running a two-year old Linux release, Yum will often give you a two-year-old version of Nmap. Even the latest version of distributions often take months to update to a new Nmap release. So for the latest version of Nmap on these systems, try the RPMs we distribute as described in the previous section. But if our RPMs aren't compatible with your system or you are in a great hurry, installing Nmap from Yum is usually as simple as executing yum install nmap (run yum install nmap zenmap if you would like the GUI too, though some distributions don't yet package Zenmap). Yum takes care of contacting a repository on the Internet, finding the appropriate package for your architecture, and then installing it along with any necessary dependencies. This is shown (edited for brevity) in Example 2 . 1 0. You can later perform yum update to install available updates to Nmap and other packages in the repository. 34 2.3. Linux Distributions Example 2.10. Installing Nmap from a system Yum repository flog-#yum i n s t a l l nmap Setting up I n s t a l l Proce s s Parsing package i n s t a l l a rgume n t s Resolving Dependencies --> Running transact ion check ---> Package nmap . x 8 6_6 4 2 : 4 . 5 2 - 1 . f c B set to be updated --> Finished Dependency Re s o l u t ion Dependencies Resolved Package Install ing : nmap Arch Ver s ion Repo s i t or y S ize x 8 6_6 4 2 : 4 . 5 2 - l . fc 8 updates 1.0 M Transact ion S ummar y Install Update Remove 1 Package ( s ) 0 Package ( s ) 0 Package ( s ) Total down load s i z e : 1 . 0 M Is this ok [ y / N J : y Downloading Package s : ( 1 / 1 ) : nmap- 4 . 5 2 - 1 . fc 8 . x 8 1 0 0 % 1 ======================== = 1 1 . 0 MB 00 : 02 Running Transact ion T e s t Transaction Test S ucceeded Runn ing Transact i on Inst a l l ing : nmap ######################### ( 1 / 1 ) Instal led : nmap . x 8 6_6 4 2 : 4 . 52 - 1 . f c B Complete ! 2.3.3. Debian Lin ux and Derivatives such as Ubu ntu LaMont Jones does a fabulous job maintaining the Nmap .deb packages, including keeping them reasonably up-to-date. The proper upgrade/install command is apt-get install nmap. This works for Debian derivatives such as Ubuntu too. Information on the latest Debian "stable" Nmap package is available at http://packages.debian.org/stablelnmap and the development ("unstable") Nmap and Zenmap packages are available from http://packages.debian.org/unstable/nmap and http://packages.debian.org!unstable/zenmap. 2.3.4. Other Linux Distri butions There are far too many Linux distributions available to list here, but even many of the obscure ones include Nmap in their package tree. If they don't, you can simply compile from source code as described in Section 2.2, "Unix Compilation and Installation from Source Code" [29]. 2.3. Linux Distributions 35 2.4. Windows While Nmap was once a Unix-only tool, a Windows version was released in 2000 and has since become the second most popular Nmap platform (behind Linux). Because of this popularity and the fact that many Windows users do not have a compiler, binary executables are distributed for each major Nmap release. While it has improved dramatically, the Windows port is not quite as efficient or stable as on Unix. Here are some known limitations: You cannot generally scan your own machine from itself (using a loopback IP such as 1 27.0.0. I or any of its registered IP addresses). This is a Windows limitation that we haven't yet worked around. If you • really want to do this, use a TCP connect scan without pinging (- s T -PN) as that uses the high level socket API rather than sending raw packets. • Nmap only supports ethernet interfaces (including most 802 . I 1 wireless cards and many VPN clients) for raw packet scans. Unless you use the - s T -PN options, RAS connections (such as PPP dialups) and certain VPN clients are not supported. This support was dropped when Microsoft removed raw TCP/IP socket support in Windows XP SP2. Now Nmap must send lower-level ethernet frames instead. Scans speeds on Windows are generally comparable to those on Unix, though the latter often has a slight performance edge. One exception to this is connect scan ( - s T), which is often much slower on Windows because of deficiencies i n the Windows networking APL This is a shame, since that is the one TCP scan that works against localhost and over all networking types (not just ethernet, like the raw packet scans). Connect scan performance can be improved substantially by applying the Registry changes in the nmap_per f orma n ce . reg file i ncluded with Nmap. By default these changes are applied for you by the Nmap executable installer. This registry file is in the nmap- directory of the Windows binary zip file, and nmap- /mswi n 3 2 in the source tarball (where is the version number of the specific release). These changes i ncrease the number of ephemeral ports reserved for user applications (such as Nmap) and reduce the time delay before a closed connection can be reused. Most people simply check the box to apply these changes in the executable Nmap i nstaller, but you can also apply them by double-clicking on nmap_p e r f ormance . reg, or by running the command regedit32 nmap_performance.reg. To make the changes by hand, add these three Registry DWORD values to HKEY_LOCAL_MAC H I N E \ SYSTEM\ CurrentCont r o l Set \ S erv i ce s \ Tcp i p \ Par amet e r s : MaxUserPort Set a large value such as 65534 (OxOOOOfffe). See MS KB Q 1 96271 6 . TCPTimedWai tDelay Set the minimum value (OxOOOOOOl e). See MS KB Q 1495327 . StrictTimeWaitSeqCheck Set to I so TCPTimedWaitDelay is checked. 6 7 http://support.microsoft.comlkb!QJ96271 http://support.microsoft.comlkb!Q 149532 36 2.4. Windows Note I would like to thank Ryan Permeh of eEye, Andy Lutomirski, and Jens Vogt for their hard work on the Nmap Windows port. For many years, Nmap was a Unix-only tool, and it would likely still be that way if not for their efforts. Windows users have three choices for installing Nmap, all of which are available from the download page at http://nmap.org/download.html. 2.4.1 . Windows 2000 Dependencies Nmap supports Windows 2000, but a couple dependencies from Microsoft must be installed first. Those are the Windows Installer 3.1 (v2)8 and the Security Update for Windows 2000 (KB835732)9 . After installing these, follow the general instructions in the following two sections to install Nmap. 2.4.2. Windows Self-i nstal ler Every Nmap release includes a Windows self-i nstaller named nmap- - s e t up . e x e (where is the version number of the specific release). Most Nmap users choose this option since it is so easy. Another advantage of the self-installer is that it provides the option to install the Zenmap GUI. Simply run the installer file and let it walk you through panels for choosing an install path and installing WinPcap. The installer was created with the open-source Null soft Scriptable Install System 10• After it completes, read Section 2.4.5, "Executing Nmap on Windows" [39] for instructions on executing Nmap on the command-line or through Zenmap. 2.4.3. Com mand-line Zip B i naries Note Most users prefer installing Nmap with the self-installer discussed previously. Every stable Nmap release comes with Windows command-line binaries and associated files in a Zip archive. No graphical interface is included, so you need to run nmap . exe from a DOS/command window. Or you can download and install a superior command shell such as those included with the free Cygwin system available from http://www.cygwin.com. Here are the step-by-step instructions for installing and executing the Nmap .zip binaries. Installing the Nmap zip binaries I . Download the .zip binaries from http://nmap. org/download. html. 2. Uncompress the zip file into the directory you want Nmap to reside in. An example would be c : \ Pr ogram F i l e s . A directory called nmap-ve r s i o n should be created, which includes the Nmap executable and data files. Microsoft Windows XP and Vista include zip extraction-just right-click on the file in 8 http://microsoft.comldownloads/details.aspx?Family/D=889482FC-5F56-4A38-8838-DE776FD4138C http:/lmicrosofr.comldownloadsldetails.aspx?FamilylD=0692C27E-F63A-414C-B3EB-D2342FBB6COO IO http://nsis.so11rceforge.11et/Main_Page 9 2.4. Windows 37 Explorer. If you do not have a Zip decompression program, there is one (called unzip) in Cygwin described above, or you can download the open-source and free 7-Zip utility 1 1 • Commercial alternatives are WinZip 12 and PKZIP 1 3 • 3. For improved performance, apply the Nmap Registry changes discussed previously. 4. Nmap requires the free WinPcap packet capture library. We build our own WinPcap installer which is available in the zip file as wi npcap-nmap- . exe, where is the Nmap version rather than the WinPcap version. Alternatively, you can obtain and install the latest version from http://www. winpcap. org. You must install version 4.0 or later. 5. Due to the way Nmap is compiled, it requires the Microsoft Visual C++ 2008 Redistributable Package of runtime components. Many systems already have this i nstalled from other packages, but you should run v c r e d i s t_x 8 6 exe from the zip file just in case you need it. • 6. Instructions for executing your compiled Nmap are given in Section 2.4.5, "Executing Nmap on Windows" [39). 2.4.4. Com pile from Sou rce Code Most Windows users prefer to use the Nmap binary self-installer, but compilation from source code is an option, particularly if you plan to help with Nmap development. Compilation requires Microsoft Visual C++ 2008, which is part of their commercial Visual Studio suite. Any of the Visual Studio editions should work, 14 including the free Visual C++ 2008 Express • Compiling Nmap on Windows from Source I . Download the latest Nmap source distribution from http://nmap.org/download.html. It has the name nmap- .tar.bz2 or nmap- .tgz. Those are the same tar file compressed using gzip or bzip2, respectively. The bzip2-compressed version is smaller. 2. Uncompress the source code file you just downloaded. Recent releases of the free Cygwin distribution 15 can handle both the . t a r . b z 2 and tgz formats. Use the command tar xvjf nmap-version.tar.bz2 or tar xvzf nmap-version.tgz, respectively. Alternatively, the common WinZip application can decompress the .tgz version. . 3. Open Visual Studio and the Nmap solution file ( nmap- /mswi n 3 2 / nmap . s l n). 4. Choose "Build Solution" from the "Build Menu". Nmap should begin compiling, and end with the line - - Done --" saying that all projects built successfully and there were zero failures. " 5 . The executable and . data files can be found in nmap- /mswi n 3 2 / Re l e a s e / . You can copy them to a preferred directory as long as they are all kept together. 1 1 hup://www. 7-zip.org 12 h1tp:llwww. wi11zip.com 13 http://www.pkware.com 14 http:llwww.microsoft.com/expresslvd 15 http://www.cygwin.com/ 38 2.4. Windows 6. Ensure that you have WinPcap installed. You can obtain it by installing our binary self-installer or executing winpcap-nmap- . exe from our zip package. Alternatively, you can obtain the official installer at http://www. winpcap. org. 7. Instructions for executing your compiled Nmap are given in the next section. Many people have asked whether Nmap can be compiled with the gcc/g++ included with Cygwin or other compilers. Some users have reported success with this, but we don't maintain instructions for building Nmap under Cygwin. 2.4.5. Executing N map on Windows Nmap releases now include the Zenmap graphical user interface for Nmap. If you used the Nmap installer and left the Zenmap field checked, there should be a new Zenmap entry on your desktop and Start Menu. Click this to get started. Zenmap is fully documented in Chapter 1 2, Zenmap GUI Users' Guide [307]. While many users love Zenmap, others prefer the traditional command-line approach to executing Nmap. Here are detailed instructions for users who are unfamiliar with command-line interfaces: I. Make sure the user you are logged in as has administrative privileges on the computer (user should be a member of the admi n i s t r a t o r s group). 2. Open a command/DOS Window. Though it can be found in the program menu tree, the simplest approach is to choose "Start" -> "Run" and type cmd . Opening a Cygwin window (if you installed it) by clicking on the Cygwin icon on the desktop works too, although the necessary commands differ slightly from those shown here. 3. Change to the directory you installed Nmap into. Assuming you used the default path, type the following commands. c: cd " \Program F i l e s \Nmap " 4. Execute nmap.exe. Figure 2.1 is a screen shot showing a simple example. 2.4. Windows 39 Figure 2.1. Executing Nmap from a Windows command shell If you execute Nmap frequently, you can add the Nmap directory (c : \ P r ogram F i l e s \Nmap by default) to your command execution path. The exact place to set this varies by Windows platform. On my Windows XP box, I do the following: I. From the desktop, right click on My C omput e r and then click "properties". 2. In the System Properties window, click the "Advanced" tab. 3. Click the "Environment Variables" button. 4. Choose Path from the S y s t em v a r i abl e s section, then hit edit. 5. Add a semi-colon and then your Nmap directory (c : \ P r og r am F i l e s \Nmap by default) to the end of the value. 6. Open a new DOS window and you should be able to execute a command such as nmap scanme.nmap.org from any directory. 2.5. Sun Solaris Solaris has long been well-supported by Nmap. Sun even donated a complete SPARCstation to the project, which is still being used to test new Nmap builds. For this reason, many Solaris users compile and install from source code as described in Section 2.2, "Unix Compilation and Installation from Source Code" [29). Users who prefer native Solaris packages will be pleased to learn that Steven Christensen does an excellent job of maintaining Nmap packages at http://www.sunfreeware.com for all modern Solaris versions and architectures. Instructions are on his site, and are generally very simple: download the appropriate Nmap package for your version of Solaris, decompress it, and then run pkgadd -d . As is generally the case with contributed binary packages, these Solaris packages are simple and quick to install. The advantages of compiling from source are that a newer version may be available and you have more flexibility in the build process. 40 2.5. Sun Solaris 2.6. Apple Mac OS X Thanks to several people graciously donating shell accounts on their Mac OS X boxes, Nmap usually compiles on that platform without problems. Because not everyone has the development tools necessary to compile from source, there is an executable installer as well. Nmap is also available through systems such as MacPorts and Fink which package Unix software for Mac OS X. 2.6.1 . Executable I nsta l ler The easiest way to install Nmap and Zenmap o n Mac OS X is to use our installer. The Mac OS X section of the Nmap download page 16 provides a file named nmap- . dmg, where is the version number of the most recent release. The dmg file is known as a "disk image". Installation instructions follow: . I. Download the file nmap- . dmg. Double-click the icon to open it. (Depending on how you downloaded the file, it may be opened automatically.) 2. The contents of the disk image will be displayed. One of the fi les will be a Mac meta-package file named nmap- . mpkg. Double-click it to start the installer. 3. Follow the instructions in the installer. You will be asked for your password since Nmap installs in a system directory. 4. Once the installer is finished, eject the disk image by control-clicking on its icon and selecting "Eject". The disk image may now be placed in the trash. See the instructions in Section 2 .6.4, "Executing Nmap on Mac OS X" [42] for help on running Nmap and Zenmap after they are installed. The programs installed by the installer are universal binaries that will run on Mac OS X 10.4 (Tiger) or later. Users of earlier versions will have to compile from source or use a third-party package. 2.6.2. Com pile from Sou rce Code Compiling Nmap from source o n Mac OS X is n o more difficult than o n other platforms once a proper build environment is in place. Compile Nmap from sou rce code Compiling Nmap on Mac OS X requires Xcode 17 , Apple's developer tools that include GCC and the rest of the usual build system. Xcode is not installed by default, but is available as an optional install on the Mac OS X installation discs. If you do not have the installation discs or if you want a newer version, you can download Xcode free of charge by following these steps. 16 http://mnap.orgldownload.html#macosx 17 h11p://developer.app/e.comltoolslxcodel 2.6. Apple Mac OS X 41 I . Apple restricts downloads of Xcode to members of the Apple Developer Connection. Browse to http://connect.apple.com and fill out some forms to create an account. Skip to the next step if you already have an account. 2. Return to http://connect.apple.com and log in with your account credentials. 3. Hit the Down l oad link and then choose Deve l oper Tool s . 4 . Download and i nstall the most recent Xcode. These exact steps may change, but it is hoped that this general approach will continue to work. Once you have i nstalled Xcode, follow the compilation instructions found in Section 2.2, "Unix Compilation and Installation from Source Code" (29). Note that on some older versions of Mac OS X, you may have to replace the command Jconfigure with Jconfigure CPP=/usr/bin/cpp. Compile Zenmap from source code Zenmap depends on some external libraries that do not come with Mac OS X, including GTK+ and PyGTK. These libraries have many dependencies of their own. A convenient way to install all of them is to use a third-party packaging system as described in Section 2.6.3. Once the dependencies are installed, follow the instructions in Section 2.2, "Unix Compilation and Installation from Source Code" (29) to install Zenmap as usual. 2.6.3. Th i rd-party Packages Another option for installing Nmap is to use a system which packages Unix software for Mac OS X. The 18 two discussed here are Fink and MacPorts 19 . See the respective projects' web sites for how to install the package managers. To install using Fink, run the command fink install nmap. Nmap will be installed as I sw/bi n / nmap. To uninstall use the command fink remove nmap. To install using MacPorts, run sudo port install nmap. Nmap will be installed as I opt / l o c a l /bin/ nmap. To uninstall, run sudo port uninstall nmap. These systems install the nmap executable outside the global PATH. To enable Zenmap to find it, set the nmap_command_p a t h variable in z e nmap . c o n f to I sw/bi n / nmap or / op t / l o c a l / b i n / nmap as described in Section 1 2. 10. 1, "The nmap Executable" (330). 2.6.4. Executing Nmap on Mac OS X The terminal emulator in Mac OS X is called Terminal, and is located in the directory /App l i c a t i o n s / Ut i l i t i e s . Open it and a terminal window appears. This is where you will type your commands. 18 http://www. fi11kproject.0rg 19 http://www.macports. org 42 2.6. Apple Mac OS X By default the root user is disabled on Mac OS X. To run a scan with root privileges prefix the command name with sudo, as in sudo nmap -sS . You will be asked for a password, which is just your normal login password. Only users with administrator privileges can do this. Zenmap requires the XI I application to be installed. If it was not installed by default it may be available as an optional install on the Mac OS X installation discs. When Zenmap is started, a dialog is displayed requesting that you type your password. Users with administrator privileges may enter their password to allow Zenmap to run as the root user and run more advanced scans. To run Zenmap in unprivileged mode, select the "Cancel" button on this authentication dialog. 2.7. FreeBSD I Open BSD I NetBSD The BSD flavors are well supported by Nmap, so you can simply compile i t from source as described i n Section 2.2, "Unix Compilation and Installation from Source Code" [29). Tills provides the normal advantages of always having the latest version and a flexible build process. If you prefer binary packages, these *BSD variants each maintain their own Nmap packages. Many BSD systems also have a ports tree which standardizes the compilation of popular applications. Instructions for installing Nmap on the most popular *BSD variants follow. 2.7.1 . OpenBS D B i nary Packages and Sou rce Ports Instructions According to the OpenBSD FAQ20, users "are HIGHLY advised t o use packages over building a n application from ports. The OpenBSD ports team considers packages to be the goal of their porting work, not the ports themselves." That same FAQ contains detailed instructions for each method. Here is a summary: Installation using binary packages I. Choose a mirror from http://www. openbsd.org/ftp.html, then FTP in and grab the Nmap package from /pub/OpenB S D / /package s / / nmap - . t g z . Or obtain it from the OpenBSD distribution CD-ROM. 2. As root, execute: pkg_add -v nmap- .tgz Installation using the source ports tree I. If you do not already have a copy of the ports tree, obtain it via CVS using instructions at http://openbsd.org/faq!faq 15.html. 2. As root, execute the following command (replace / u s r /por t s with your local ports directory if it differs): cd /usr/ports/net/nmap 20 && make install clean http://www.openbsd.org/faql 2.7. FreeBSD I OpenBSD I NetBSD 43 2. 7.2. FreeBSD Bina ry Package a nd Source Ports Instructions 21 The FreeBSD project has a whole chapter in their Handbook describing the package and port installation processes. A brief summary of the process follows. Installation of the binary package The easiest way to install the binary Nmap package is to run pkg_add -r nmap. You can then run the same command with the z e nmap argument if you want the X-Window front-end. If you wish to obtain the package manually instead, retrieve it from http://freshports.org!securitylnmap and http://freshports.org/security/zenmap or the CDROM and run pkg_add . Installation using the source ports tree I. The ports tree is often installed with the system itself (usually in / u s r /port s). If you do not already have it, specific installation instructions are provided in the FreeBSD Handbook chapter referenced above. 2. As root, execute the following command (replace / u s r /por t s with your local ports directory if it differs): cd /usr/ports/security/nmap && make install clean 2. 7.3. Net BSD B i nary Package I nstructions NetBSD has packaged Nmap for a n enormous number o f platforms, from the normal i386 to Play Station 2, PowerPC, VAX, SPARC, MIPS, Amiga, ARM, and several platforms that I have never even heard of! Unfortunately they are not very up-to-date. A list of NetBSD Nmap packages is available from ftp://ftp.netbsd. orglpub!NetBSD!packages/pkgsrc/netlnmap!README.html and a description of using their package system to install applications is available at http://netbsd.org/Documentation!pkgsrc!using.html. 2.8. Am iga, HP- UX, I R IX, and Other Platforms One of the wonders of Open Source development is that resources are often directed towards what people find exciting rather than having an exclusive focus on profits as most corporations do. It is along those li nes that the Amiga port came about. Diego Casorran performed most of the work and sent in a clean patch which was integrated into the main Nmap distribution. In general, AmigaOS users should be able to simply follow the source compilation instructions in Section 2.2, "Unix Compilation and Installation from Source Code" [29). You may encounter a few hurdles on some systems, but I presume that must be part of the fun for Amiga fanatics. Nmap supports many proprietary Unix flavors such as HP-UX and SGI IRIX. The Nmap project depends on the user community to help maintain adequate support for these systems. If you have trouble, try sending a report with ful l details to the nmap-dev mailing list, as described in Section 15. 17, "Bugs" [4 1 1 ) . Also let us know if you develop a patch which improves support on your platform so we can incorporate it into Nmap. 21 http://www.freebsd.orgldoden_US.!S08859-!lbooks/handbook/ports.html 44 2.8. Amiga, HP-UX, IRIX, and Other Platforms 2 .9. Removing N map I f your purpose for removing Nmap i s simply to upgrade to the latest version, you can usually use the upgrade option provided by most binary package managers. Similarly, installing the latest source code (as described in Section 2.2, "Unix Compilation and Installation from Source Code" [29]) generally overwrites any previous from-source installations. Removing Nmap is a good idea if you are changing install methods (such as from source to RPM or vice versa) or if you are not using Nmap anymore and you care about the few megabytes of disk space it consumes. How to remove Nmap depends on how you installed it initially (see previous sections). Ease of removal (and other maintenance) is a maj or advantage of most binary packages. For example, when Nmap is installed using the RPM system common on Linux distributions, it can be removed by running the command rpm -e nmap zenmap as root. Analogous options are offered by most other package managers--consult their documentation for further information. If you installed Nmap from the Windows installer, simply open the Control Panel, select "Add or Remove Programs" and select the "Remove" button for Nmap. You can also remove WinPcap unless you need it for other applications such as Wireshark. If you installed Nmap from source code, removal is slightly more difficult. If you still have the build directory available (where you initially ran make install), you can remove Nmap by running make uninstall. If you no longer have that build directory, type nmap -V to obtain the Nmap version number. Then download that source tarball for that version of Nmap from http://nmap. org!dist! or http://nmap.org!dist-old/. Uncompress the tarball and change into the newly created directory (nmap- ). Run ./configure, including any install-path options that you specified the first time (such as --pr e f i x or --datadi r). Then run make uninstall. Alternatively, you can simply delete all the Nmap-related files. If you used a default source install of Nmap versions 4.50 or higher, the following commands remove it. # # # # # cd / u s r / loca l rm -f bin / nmap b i n / nmapfe bin/ xnmap rm -f man/man l / nmap . 1 man/man l / z enmap . 1 rm -rf share/ nmap . /bin /uninstal l_zenmap You may have to adjust the above commands slightly if you specified --pref ix or other install-path option when first installing Nmap. The files relating to zenmap, nmapfe, and xnmap do not exist if you did not install the Zenmap frontend. 2.9. Removing Nmap 45 Chapter 3. Host D iscovery ( " P ing Scanning ") 3.1 . Introd uction One of the very first steps in any network reconnaissance mjssion i s to reduce a (some6mes huge) set of IP ranges into a list of ac6ve or interesting hosts. Scanning every port of every single IP address i s slow and usually unnecessary. Of course what makes a host interesting depends greatly on the scan purposes. Network administrators may only be interested in hosts running a certain service, while security auditors may care about every single device with an IP address. An administrator may be comfortable using j ust an ICMP ping to locate hosts on his internal network, while an external penetration tester may use a diverse set of dozens of probes in an attempt to evade firewall restrictions. Because host discovery needs are so diverse, Nmap offers a wide variety of options for customizing the techniques used. Despite the name ping scan, this goes well beyond the simple ICMP echo request packets associated with the ubiquitous ping tool. Users can skip the ping step entirely with a list scan (- s L) or by disabling ping (-PN), or engage the network with arbitrary combinations of multi-port TCP SYN/ACK, UDP, and ICMP probes. The goal of these probes is to solicit responses which demonstrate that an IP address is actually active (is being used by a host or network device). On many networks, only a small percentage of IP addresses are active at any given time. This is particularly common with private address space such as 10.0.0.0/8. That network has 16.8 million IPs, but I have seen it used by companies with fewer than a thousand machines. Host discovery can find those machines in a sparsely allocated sea of IP addresses. This chapter first discusses how Nmap ping scanning works overall, with high-level control op6ons. Then specific techniques are covered, including how they work and when each is most appropriate. Nmap offers many ping techniques because it often takes carefully crafted combina6ons to get through a series of firewalls and router filters leading to a target network. Effec6ve overal l ping scannjng strategies are discussed, followed by a low-level look at the algorithms used. 3.2. Specifying Target Hosts and Networks Everything on the Nmap command-line that isn't an option (or option argument) is treated as a target host specification. The simplest case is to specify a target IP address or hostname for scanning. Sometimes you wish to scan a whole network of adjacent hosts. For this, Nmap supports CIDR-style addressing. You can append / to an IPv4 address or hostname and Nmap will scan every IP address for which the first are the same as for the reference IP or hostname given. For example, 192. 168.10.0/24 would scan the 256 hosts between 192.168. 10.0 (binary: 1 1000000 1 0 1 0 1000 0000 1010 O O O O O O O O ) and 192. 168. 10.255 (binary: 1 1000000 1 0 1 0 1000 0000 1 0 1 0 1 1 1 1 1 1 1 1 ), inclusive. 192. 168. 10.40/24 would scan exactly the same targets. Given that the host s c anme . nmap . o r g is at the IP address 64. 1 3 . 1 34.52, the specification scanme.nmap.org/ 16 would scan the 65,536 IP addresses between 64. 1 3.0.0 and 64. 13.255.255. The smallest allowed value is /0, which scans the whole Internet. The largest value is /32, which scans just the named host or IP address because all address bits are fixed. 3 . 1 . Introduction 47 CIDR notation is short but not always flexible enough. For example, you might want to scan 192. 168.0.0/16 but skip any IPs ending with .0 or .255 because they may be used as subnet network and broadcast addresses. Nmap supports this through octet range addressing. Rather than specify a normal IP address, you can specify a comma-separated list of numbers or ranges for each octet. For example, 192. 1 68.0-255. 1 -254 wi ll skip all addresses in the range that end in .0 or .255. Ranges need not be limited to the final octets: the specifier 0-255.0-255. 13.37 will perform an Internet-wide scan for all IP addresses ending in I 3.37. This sort of broad sampling can be useful for Internet surveys and research. 1Pv6 addresses can only be specified by their fully qualified 1Pv6 address or hostname. CIDR and octet ranges aren't supported for 1Pv6 because they are rarely useful. Nmap accepts multiple host specifications on the command line, and they don't need to be the same type. The command nmap scanme.nmap.org 192.168.0.0/8 10.0.0,1,3-7.0-255 does what you would expect. 3.2.1 . I n put From List (-iL) Passing a huge list of hosts is often awkward on the command line, yet it is a common need. For example, Or maybe you want to scan all IP addresses except for those ones to locate hosts using unauthorized static IP addresses. Simply generate the list of hosts to scan and pass that fi lename to Nmap as an argument to the - i L option. Entries can be in any of the formats accepted by Nmap on the command line (IP address, hostname, CIDR, 1Pv6, or octet ranges). Each entry must be separated by one or more spaces, tabs, or newlines. You can specify a hyphen (-) as the filename if you want Nmap to read hosts from standard input rather than an actual file. your DHCP server might export a list of 10,000 current /eases that you wish to scan. 3.2.2. Choose Targets at Random (-iR ) For Internet-wide surveys and other research, you may want to choose targets at random. This is done with the - i R option, which takes as an argument the number of IPs to generate. Nmap automatically skips certain undesirable IPs, such as those in private, multicast, or unallocated address ranges. The argument 0 can be specified for a never-ending scan. Keep in mind that some network administrators bristle at unauthorized scans of their networks. Carefully read Section 1 .4, "Legal Issues" [ 1 3] before using -iR. If you find yourself really bored one rainy afternoon, try the command nmap -sS -PS80 -iR 0 -p 80 locate random web servers for browsing. to 3.2.3. Excluding Targets (--exclude, --excludefile ) I t i s common to have machines which you don't want to scan under any circumstances. Machines can be so critical that you won't take any risk of an adverse reaction. You might be blamed for a coincidental outage even if the Nmap scan had nothing to do with it. Or perhaps you have legacy hardware that is known to crash when scanned, but you haven't been able to fix or replace it yet. Or maybe certain IP ranges represent subsidiary companies, customers, or partners that you aren't authorized to scan. Consultants often don't want their own machine included in a scan of their client's networks. Whatever the reason, you can exclude hosts or entire networks with the --exc l ude option. Simply pass the option a comma-separated list of excluded 48 3.2. Specifying Target Hosts and Networks targets and netblocks using the normal Nmap syntax. Alternatively, you can create a file of excluded hosts/networks and pass that to Nmap with the --exc l ude f i l e option. 3.2.4. Practical Examples While some tools have simple interfaces that only allow a list of hosts or maybe let you specify the start and end IP addresses for a range, Nmap is much more powerful and flexible. But Nmap can also be more difficult to learn-and scanning the wrong IP addresses is occasionally disastrous. Fortunately, Nmap offers a dry run using the list scan ( - s L option). Simply execute nmap -sL -n to see which IPs would be scanned before you actually do it. Examples may be the most effective way to teach the Nmap host specification syntax. This section provides some, starting with the simplest. nmap scanme.nmap.org, nmap scanme.nmap.org/32, nmap 64.13.134.52 These commands all do the same thing, assuming that scanme.nmap.org resolves to 64.1 3 . 1 34.52. They scan that one IP and then exit. nmap scanme.nmap.org/24, nmap 64.13.134.52/24, nmap 64.13.134.0-255 These all ask Nmap to scan the 256 IP addresses from 64. 1 3.134.0 through 64. 1 3 . 1 34.255. In other words, they ask to scan the class C sized address space surrounding scanme.nmap.org. nmap 64.13.134.52/24 --exclude scanme.nmap.org,insecure.org Tells Nmap to scan the class C around 64. 13.1 34.52, but to skip scanme.nmap.org and insecure.org if they are found within that address range. nmap 10.0.0.0/8 --exclude 10.6.0.0/16,ultra-sensitive-host.company.com Tells Nmap to scan the whole private 10 range except that it must skip anything starting with 10.6 as well as ultra-sensitive-host.company.com. egrep ' Alease' /var/lib/dhcp/dhcpd.leases I awk ' {print $2}' I nmap -iL Obtain the list of assigned DHCP IP addresses and feed them directly to Nmap for scanning. Note that a hyphen is passed to i L to read from standard i nput. - nmap -6 2001:800:40:2a03::3 Scans the IPv6 host at address 2001 : 800:40:2a03: :3. 3.3. Fi nd ing an Organ ization's I P Add resses Nmap automates many aspects of network scanning, but you stil l must tell it which networks to scan. I suppose you could specify - i R and hope Nmap hits your target company randomly, or you could try the brute force method of specifying 0 . 0 . 0 . 0 I 0 to scan the whole Internet. But either of those options could lake months or years, and possibly get you into trouble. So it is important to carefully research target netblocks before scanning them. Even if you are conducting a legitimate penetration test and the client gave you a list of their netblocks, it is important to double check them. Clients sometimes have out-of-date records or simply write them down wrong. An authorization letter signed by your client won't help if you accidentally break into the wrong company. 3.3. Finding an Organization's IP Addresses 49 In many cases, you start with only a company's domain name. This section demonstrates a few of the most common and effective ways to turn that into a list of netblocks owned, operated by, or affiliated with the target company. Typical Linux command-line utilities are demonstrated, but similar tools are available for other platforms. At the ShmooCon conference in 2006, a fel low came up to me and complained that Nmap documentation specified many example ways to scan t arget . com. He noted that ICANN had reserved the domain name examp l e . com for this purpose, and pressured me to revise the man page accordingly. While he was technically right, it was a strange thing to obsess about. His motivation became clear when he handed me his business card: Figure 3.1. A business card explains everything TARGET CORPO RATION ----- ® -- I nformation Security OpenPGP fingerprint 33 South Sixth Street. Ml1111eap01ts, Minnesota 55402 -- 612-304 -@target .com Apparently, many Nmap users copied examples straight from the man page and ran them without changing the target specifier. So target.com was flooded with scans and corresponding IDS alerts. In honor of that incident, the goal of this section is to determine IP ranges assigned to and used by Target Corporation. 3.3.1 . DNS Tricks The primary purpose of DNS is to resolve domain names into IP addresses, so it is a logical place to start. In Example 3. 1 , I use the Linux host command to query some common DNS record types. 50 3.3. Finding an Organization's IP Addresses Example 3.1. Using the host command to query common DNS record types > host -t ns target . com target . com name s erver ns 4 . t a r ge t . com . target . com name s erver n s 3 . t a r ge t . com . �arget . com name s erver n s l -auth . spr in t l ink . net . target . com name server n s 2 -a u t h . spr i n t l ink . net . target . com name s erver n s 3 -a u t h . spr i n t l i n k . n e t . host -t a t ar g e t . corn arget . com has addr e s s 1 6 1 . 2 2 5 . 1 3 0 . 1 6 3 rget . com has addre s s 1 6 1 . 2 2 5 . 1 3 6 . 0 host -t aaaa target . com rget . com has no AAAA r ecord host -t mx target . com get . com ma i l is handled by 50 smtp0 2 . target . com . get . com ma i l i s handled by 5 smtpO l . target . com . st -t soa target . com rget . com has SOA record extdn s 0 2 . t a r ge t . com . hostma st e r . ta r get . com . Next I resolve the IP addresses for the hostnames above (using host again) and I try a few common subdomain names such as www . target . c om and ftp . t a rget . com. Starting with names like n s 3 . t arget . com and smt p O 1 . target . com, I try changing the digits to fi nd new machines. All of this leaves me with the following target.com names and addresses: Table 3.1. First pass at listing target.com IPs IP Addresses 161 .225 . 1 30. 1 30 161 .225 . 1 36.1 36 161 .225 . 1 30. 1 50 161 .225. 1 36.0, 161 .225 . 1 30. 163 161 .225. 140. 1 20 1 98.70.53.234, 1 98.70.53 .235 172. 17.14.69 207.171. 166.49 While a substantial hostname list can be generated in this manner, the mother lode of hostnames comes from a zone transfer. Most DNS servers now reject zone transfer requests, but it is worth a try because many stil l allow it. Be sure to try every DNS server you have found through domain NS records and port scanning corporate IP ranges. So far we have found seven Target nameservers: n s 3 . t ar get . c om, ns4 . target . com, n s S . t arget . com, n s l -auth . s p r i n t l i nk . net, ns2-aut h . spr i nt l i n k . net, n s 3 -auth . s p r i n t l i nk . net, and extdn s 0 2 . t arget . c om. Unfortunately, all of those servers either refused the transfer or did not support the TCP DNS connections tequired for a zone transfer. Example 3 .2 shows a failed target . c om zone transfer attempt using the mon dig (domain information groper) tool 1, followed by a successful one against an unrelated organization (cpsr . org). �map's zone-transfer NSE script could have been used instead (see Chapter 9, Nmap Scripting Engine (205)). 3.3. Finding an Organization's IP Addresses 51 Example 3.2. Zone transfer failure and success > d i g @ n s 2 -auth . spr i n t l i n k . ne t -t AXFR target . com D i G 9 . 5 . 0b3 < < > > @ n s 2 -auth . sp r i n t l in k . net -t AXFR targe t . com <<>> Trans f e r f a i l ed . > d i g @ n s 2 . eppi . com -t AXFR cpsr . or g < < > > D i G 9 . 5 . 0bl < < > > @ n s 2 . eppi . com - t AXFR cpsr . or g cpsr . or g cpsr . or g . cpsr . or g . cpsr . or g . cpsr . or g . cpsr . or g . d i a c . cp s r . or g . groups . cp s r . or g . l oc a l ho s t . cp s r . or g . ma i l . cp s r . or g . peru . cp s r . or g . www . peru . cp s r . or g . [. . .] 1 0800 1 0800 1 0800 10800 10800 1 0800 10800 10800 10800 1 0800 1 0800 1 0800 IN IN IN IN IN IN IN IN IN IN IN IN SOA NS NS NS A MX A NS A A A A n s l . f indpage . com . root . cpsr . org . n s . s t i mpy . net . n s l . f i ndpage . com . n s 2 . eppi . com . 2 08 . 96 . 55 . 202 0 smtp . e l e c t r i cember s . ne t . 64 . 14 7 . 163 . 10 n s l . e l ec t r i cember s . net . 127 . 0 . 0 . 1 2 0 9 . 2 0 9 . 8 1 . 73 2 08 . 96 . 55 . 202 2 08 . 96 . 5 5 . 202 A common mistake when gathering forward DNS results like these is assuming that all systems found under a domain name must be part of that organization's network and safe to scan. In fact, nothing prevents an organization from adding records pointing anywhere on the Internet. This is commonly done to outsource services to third parties while keeping the source domain name for branding. For example, www . t arget . com resolves to 2 0 7 . 1 7 1 . 1 6 6 . 4 9 . Is this part of Target's network, or is it managed by a third party we might not want to scan? Three quick and easy tests are DNS reverse-resolution, traceroute, and whois against the relevant IP address registry. The first two steps can be done by Nmap, while the Linux whois command works well for the third. These tests against t a r get . com are shown in Example 3.3 and Example 3.4. 52 3.3. Finding an Organization's IP Addresses Example 3.3. Nmap reverse-DNS and traceroute scan against www.target.com i nmap -PN - T 4 - - t r aceroute www . target . com Starting Nmap ( http : / / nmap . or g ) Interesting por t s on 1 6 6 - 4 9 . amazon . com ( 2 0 7 . 1 7 1 . 1 6 6 . 4 9 ) : Not shown : 9 9 8 f i l t ered por t s PORT S TATE S E RV I C E 80 /tcp open http 443 /tcp open https TRACEROUTE HOP RTT [ cut ] 9 84 . 9 4 1 0 87 . 9 1 11 9 4 . 8 0 12 86 . 4 0 13 185 . 1 0 14 84 . 70 15 85 . 7 3 16 85 . 6 8 ( u s ing port 8 0 / tcp ) ADDRES S ae-2 . ebr 4 . NewYork l . Leve l 3 . ne t ( 4 . 6 9 . 1 3 5 . 1 8 6 ) ae-3 . ebr 4 . Wa s h ingtonl . Leve l 3 . net ( 4 . 6 9 . 1 3 2 . 9 3 ) ae- 9 4 - 9 4 . c sw4 . Wa s h ingtonl . Leve l 3 . ne t ( 4 . 6 9 . 1 3 4 . 1 9 0 ) ae- 2 1 -6 9 . carl . Wa s h ington 3 . Leve l 3 . net ( 4 . 6 8 . 1 7 . 7 ) AMAZONCOM . carl . Washington3 . Level 3 . net ( 4 . 7 1 . 2 0 4 . 1 8 ) 72 . 2 1 . 2 0 9 . 3 8 72 . 2 1 . 1 9 3 . 3 7 1 6 6-4 9 . amazon . com ( 2 0 7 . 1 7 1 . 1 6 6 . 4 9 ) Nmap done : 1 I P addr e s s ( 1 host up ) scanned i n 2 0 . 5 7 s econds Example 3.4. Using whois to find owner of www.target.com IP address > whois 2 0 7 . 1 7 1 . 1 6 6 . 4 9 [ Querying whois . ar i n . ne t ] [who is . ar in . net ] OrgName : Org I D : Address : City : StateProv : PostalCode : Country : [...] Ama zon . com, I nc . AMAZON- 4 6 0 5 5th Ave S S EATTLE WA 98104 US In Example 3.3, the reverse DNS (two places) and interesting traceroute results are bolded. The Amazon.com domain name makes it highly likely that the web site is run by Amazon rather than Target itself. Then the whois results showing "Amazon.com, Inc." as the IP space owner removes all doubt. The web site is Target branded, but displays "Powered by Amazon.com" at the bottom. If we were hired by Target to test their security, we would need separate permission from Amazon to touch this address space. Web databases can also be used to find hostnames under a given domain. For example, Netcraft has a web site DNS search feature at http://searchdns.netcraft.com/?host. Typing t arget . com in to the form brings 36 results, as shown in Figure 3.2. Their handy table shows the netblock owner too, which catches cases such as Amazon running www . t arget . com. We a lready knew about some of the discovered hosts, but we would have been unlikely to guess names such as s endasmoochie . t arget . c om. . 3.3. Finding an Organization's IP Addresses 53 Figure 3.2. Netcraft finds 36 Target web servers Fou nd 36 sites Site Site Report First seen Netblock OS October 1 995 Amazon.com , Inc. unknown 1. www.target.com fil 2. weeklyad. target . com fil January 2005 Linux 3. sites . target . corn fil Akarnai Technologies August 2005 F5 Big-IP 4. redcard. target. com fil Target Corporation November Target Corporation FS Big-IP 2005 5. www .target.com.au ii June 2000 APNIC Windows 2000 6. targetrewards . target com fil August 2005 Target Corporation F5 Big-IP 7. cinemared .targel .com fil August 2005 Target Corporation F5 Big- I P 8. recipes.target. com Iii November Allrecipes .com , Inc. Windows 2005 9. bookmarked.target.com ii September Server 2003 l m plex.net Linux Google can also be used for this purpose with queries such as s i t e : t arget . com. 3.3.2. Whois Queries Agai nst I P Reg istries After a set of i nitial "seed" IPs are discovered, they must be researched to ensure they belong to the company you expect and to determine what netblocks they are part of. A small company might have a tiny allocation of 1 - 1 6 IP addresses, while larger corporations often have thousands. This information is kept in regional databases, such as ARIN (American Registry for Internet Numbers) for North America and RIPE for Europe and the Middle East. Modern whois tools take an IP address and automatically query the appropriate registry. Small and mid-sized companies normally don't have IP space allocated by the likes of ARIN. Instead, they are delegated netblocks from their ISPs. Sometimes you get this ISP information from IP queries. This generally leaves you with a big netblock and you don't know which portion of it is allocated to your target. Fortunately, many ISPs now subdelegate customer ranges using Shared Whois (SWIP) or Referral Whois (RWhois). If the ISP has done this, you learn the customer's exact netblock size. One of the IP addresses previously discovered for target.com was 1 6 1 2 2 5 1 3 0 . 1 6 3. Example 3.5 demonstrates a whois query (automatically directed against ARIN) to determine the owner and IP allocation i nformation for this IP. . 54 3.3. Finding an Organization's IP Addresses . Example 3.5. Using whois to find netblock containing 161.225.130.163 erying whoi s . ar i n . net ] hois . arin . net ] Target Corporat ion TARGE T - 1 4 1 0 0 0 Nicollet T P S 3 1 6 5 Minneapo l i s MN 55403 us 161 . 225 . 0 . 0 - 1 6 1 . 22 5 . 255 . 25 5 161 . 225 . 0 . 0/16 TARGETNET NET- 1 6 1 - 2 2 5 - 0 - 0 - 1 NET- 1 6 1 - 0 - 0 - 0 - 0 Direct As s i gnment Server : NS3 . TARGET . COM Server : N S 4 . TARGE T . COM 1993-03-04 2005-11-02 qTechHandle : gTechName : qTechPhone : gTechEmail : DOMAI 4 5 -A RI N Domai nname s admin + 1 -6 1 2 - 6 9 6 -2 5 2 5 Doma i nname s . admi n @ t a r get . com Not surprisingly, Target owns a huge Class B netblock, covering all 65,536 IPs from 161 .225.0.0 through 161 .225.255.255. Since the OrgName is Target, this isn't a case where we are seeing results from their ISP. The next step is to similarly look up all previously discovered IPs which don't fal l within this range. Then you can begin with more advanced queries. The command whois -h whois.arin.net \? gives the ARIN query syntax. It would be nice if you could search for all netblocks matching a given address, OrgID, or OrgTechEmail, but IP registries generally don't allow that. However, many other helpful queries are allowed. For example, whois -h whois.arin.net @target.com shows all the ARIN contacts with email addresses at target.com. The query whois -h whois.arin.net "n target*" shows all the netblock handles starting with target. It is not case sensitive. Similarly, whois -h whois.arin.net "o target*" shows all of the organizational names starting with t ar ge t . You can look up the address, phone number, and contact email associated with each entry to determine whether they are part of the company you wish to scan. Often they are 3rd parties who happen to have a similar name. 3.3.3. Internet Routing I nformation The core routing protocol of the Internet i s the Border Gateway Protocol (BGP). When scanning mid-sized and large organizations, BGP routing tables can help you find their IP subnets all over the world. For example, suppose you want to scan IP addresses belonging to Microsoft Corporation. A DNS lookup for micr o s o f t . com provides the IP address 2 0 7 . 4 6 . 1 9 6 . 1 1 5 . A whois query as discussed in the previous 3.3. Finding an Organization's IP Addresses 55 section shows that the whole 207.46.0.0/ 16 block belongs to Microsoft at their appropriate "One Microsoft Way" address in Redmond. That provides 65,536 IP addresses to scan, but BGP tables expose many more. Entities such as Microsoft are assigned autonomous system (AS) numbers for routing purposes. A handy tool for determining the AS number advertised for a given IP address is available at http://asn. cymru.com/. Typing 2 0 7 . 4 6 . 0 . 0 into this form provides Microsoft's AS number 8075. Next, I want to find all of the IP prefixes which route to this AS. A handy tool for doing so is available at http://www. robtex.com!as/. Typing in AS 8 0 7 5 and hitting Go on that page leads to a summary screen showing 42 prefixes found. Those prefixes represent 339,456 IP addresses and can be enumerated by clicking the BGP tab. While obtaining BGP information from canned web forms such as these is convenient, obtaining routing data from actual routers is more fun and may allow more powerful custom queries. Several organizations provide such a service. For an example, telnet to r o u t e -vi ews . routevi ews . org or visit http://routeviews. org. Of course these services provide read-only access to the data. If you need to manipulate global routing tables as part of a diabolical plan to take over the Internet, that is beyond the scope of this book. 3.4. DNS Resol ution The key focus of Nmap host discovery i s determining which hosts are up and responsive on the network. That narrows down the field of targets, since you can't hack a host which doesn't exist. But don't let discovery end there. You wouldn't date girls (or guys) just because they're breathing, and selecting boxes on the network to penetrate deserves special care too. A great source of information (about networked hosts, not potential dates) is DNS, the domain name system. Even security conscious organizations often assign names which disclose the function of their systems. It's not uncommon to see wireless access points named wap or w i r e l e s s , firewalls named fw, f i rewa l l , or fw- 1 , and development web servers with not-yet-published content named dev, s t ag i ng, www- i nt , or be t a . Locations or department names are also often disclosed, as in the company whose Chicago office firewall is named fw . c h i . By default, Nmap performs reverse-DNS resolution for every IP which responds to host discovery probes (i.e. those that are online). If host discovery is skipped with -PN, resolution is performed for all IPs. Rather than use the slow standard DNS resolution libraries, Nmap uses a custom stub resolver which performs dozens of requests in parallel. While the defaults generally work well, Nmap offers four options for controlling DNS resolution. They can substantially affect scanning speed and the amount of information gathered. -n (No DNS resolution) Tells Nmap to never do reverse DNS resolution on the active IP addresses it finds. Since DNS can be slow even with Nmap's built-in parallel stub resolver, this option reduces scanning times. -R (DNS resolution for all targets) Tells Nmap to always do reverse DNS resolution on the target IP addresses. Normally reverse DNS is only performed against responsive (online) hosts. - - s y s t em-d n s (Use system DNS resolver) By default, Nmap resolves IP addresses by sending queries directly to the name servers configured on your host and then listening for responses. Many requests (often dozens) are performed in parallel to improve performance. Specify this option to use your system resolver instead (one IP at a time via the 56 3.4. DNS Resolution getnameinfo call). This is slow and rarely useful unless you find a bug in the Nmap parallel resolver (please let us know if you do). The system resolver is always used for IPv6 scans. --dns- servers [ , [ , ] ] (Servers to use for reverse DNS queries) By default, Nmap determines your DNS servers (for rDNS resolution) from your resolv.conf fi le (Unix) or the Registry (Win32). Alternatively, you may use this option to specify alternate servers. This option is not honored if you are using - - s y s t em-dn s or an IPv6 scan. Using multiple DNS servers is often faster, especially if you choose authoritative servers for your target IP space. This option can also improve stealth, as your requests can be bounced off just about any recursive DNS server on the Internet. . . . This option also comes in handy when scanning private networks. Sometimes only a few name servers provide proper rDNS information, and you may not even know where they are. You can scan the network for port 53 (perhaps with version detection), then try Nmap list scans ( - s L) specifying each name server one at a time with --dn s - s e r ve r s until you find one which works. 3.5. Host Discovery Controls By default, Nmap will include a ping scanning stage prior to more intrusive probes such as port scans, OS detection, Nmap Scripting Engine, or version detection. Nmap usually only performs i ntrusive scans on machines that are shown to be available during the ping scan stage. This saves substantial time and bandwidth compared to performing ful l scans against every single IP address. Yet this approach is not ideal for all circumstances. There are times when you do want to scan every IP ( -PN), and other times when you want to perform host discovery and nothing more (-sP). There are even times when you want to print out the target hosts and exit prior to even sending ping probes ( - s L). Nmap offers several high-level options to control this behavior. 3.5.1 . List Scan (-sL) List scan is a degenerate form of host discovery that simply lists each host on the network(s) specified, without sending any packets to the target hosts. By default, Nmap still performs reverse-DNS resolution on the hosts to learn their names. Nmap also reports the total number of IP addresses at the end. List scan is a good sanity check to ensure that you have proper IP addresses for your targets. If the hosts sport domain names you do not recognize, it is worth investigating further to prevent scanning the wrong company's network. There are many reasons target IP ranges can be incorrect. Even network administrators can mistype their own netblocks, and pen-testers have even more to worry about.Jn some cases, security consultants are given the wrong addresses. In others, they try to find proper IP ranges through resources such as whois databases and routing tables. The databases can be out of date, or the company could be loaning IP space to other organizations. Whether to scan corporate parents, siblings, service providers, and subsidiaries is an important issue that should be worked out with the customer in advance. A preliminary list scan helps confirm exactly what targets are being scanned. Another reason for an advance list scan is stealth. In some cases, you do not want to begin with a ful l-scale assault on the target network that is likely to trigger IDS alerts and bring unwanted attention. A list scan is unobtrusive and provides information that may be useful in choosing which individual machines to target. It is possible, though highly unlikely, that the target will notice all of the reverse-DNS requests. When that 3.5. Host Discovery Controls 57 - is a concern, you can bounce through anonymous recursive DNS servers using the --dn s - s e rver s option as described in the section called "DNS proxying" [286). A list scan is specified with the - s L command-line option. Since the idea is to simply print a list of target hosts, options for higher level functionality such as port scanning, OS detection, or ping scanning cannot be combined with - s L. If you wish to disable ping scanning while still performing such higher level functionality, read up on the -PN option. Example 3.6 shows list scan being used to enumerate the CIDR /28 network range ( 16 IP addresses) surrounding the main Stanford University web server. Example 3.6. Enumerating hosts surrounding www.stanford.edu with list scan f e l i x - > nmap - s L www . s t anford . edu / 2 8 S t a r t i n g Nmap ( http : / / nmap . or g ) Host www9 . S tan ford . EDU ( 1 7 1 . 6 7 . 1 6 . 8 0 ) not s canned Host wwwl O . Stan ford . EDU ( 1 7 1 . 6 7 . 1 6 . 8 1 ) not s canned Host s c r i p t o r i um . Stan ford . EDU ( 1 7 1 . 6 7 . 1 6 . 8 2 ) not s canned Host cou r s ewor k - a . Stanford . EDU ( 1 7 1 . 6 7 . 1 6 . 8 3 ) not s canned Host cou r s ewor k-e . S t anford . EDU ( 1 7 1 . 6 7 . 1 6 . 8 4 ) not s canned Host www3 . S t a n ford . EDU ( 1 7 1 . 6 7 . 1 6 . 8 5 ) not s canned Host l e l a nd-dev . St an ford . EDU ( 1 7 1 . 6 7 . 1 6 . 8 6 ) not s canned Host cou r s ework -preprod . St a nford . EDU ( 1 7 1 . 6 7 . 1 6 . 8 7 ) not s canned Host s t an fordwho-dev . S tan ford . EDU ( 1 7 1 . 6 7 . 1 6 . 8 8 ) not s canned Host workgroup-dev . St a n ford . EDU ( 1 7 1 . 6 7 . 1 6 . 8 9 ) not s canned Host cour seworkbe t a . Stanford . EDU ( 1 7 1 . 6 7 . 1 6 . 9 0 ) not s canned Rost '«'fl'tl � . Stanfor d . '£.DU \ 1 1 1 . 6 1 . H i . 9 1 ) not s canned Host cou r s ework - i . Stan ford . EDU ( 1 7 1 . 6 7 . 1 6 . 9 2 ) not s canned H o s t l e land2 . S tanford . ED U ( 1 7 1 . 6 7 . 1 6 . 9 3 ) not s canned Host cours ewor k - j . Stan ford . ED U ( 1 7 1 . 6 7 . 1 6 . 9 4 ) not s canned Host 1 7 1 . 6 7 . 1 6 . 9 5 not s canned Nmap done : 16 IP addre s s e s ( 0 h o s t s up ) scanned i n 0 . 3 8 s econds 3.5.2. Ping Scan (-sP) This option tells Nmap to only perform a ping scan, then print out the available hosts that responded to the scan. No further testing (such as port scanning or OS detection) is performed, except for Nmap Scripting Engine (-- s cr ipt) host scripts and traceroute probing ( - - t r aceroute) if you specified those options. This is one step more intrusive than a l ist scan, and can often be used for the same purposes. It performs light reconnaissance of a target network quickly and without attracting much attention. Knowing how many hosts are up i s more valuable to attackers than the list of every single IP and host name provided by list scan. Systems administrators often fi nd this option valuable as well. It can easily be used to count available machines on a network or monitor server availability. This is often called a ping sweep, and is more reliable than pinging the broadcast address because many hosts do not reply to broadcast queries. Example 3.7 shows a quick ping scan against the CIDR /24 (256 IPs) surrounding one of my favorite web sites, Linux Weekly News. F 1 l a SI n SI 1 l u 1 1 d h h A v s 58 3.5. Host Discovery Controls Example 3.7. Discovering hosts surrounding www l wn . net with a ping scan . nmap -sP - T 4 www . lwn . ne t / 2 4 i Starting Nmap ( http : / / nmap . or g Host 66 . 2 1 6 . 6 8 . 0 seems to b e a subnet broadc a s t addr e s s ( returned 1 extra ping ) Host 66 . 2 1 6 . 6 8 . 1 appears to be up . Host 66 . 2 1 6 . 6 8 . 2 appears to be up . Host 66 . 2 1 6 . 6 8 . 3 appears t9 be u p . Host server l . camnetsec . com ( 6 6 . 2 1 6 . 6 8 . 1 0 ) appears to be up . Host akqa . com ( 6 6 . 2 1 6 . 6 8 . 1 5 ) appe a r s to be up . Host asria . or g ( 6 6 . 2 1 6 . 6 8 . 1 8 ) appe a r s to be up . Host webcubic . ne t ( 6 6 . 2 1 6 . 6 8 . 1 9 ) appe a r s to be up . Host dizzy . yel lowdog . com ( 6 6 . 2 1 6 . 6 8 . 2 2 ) appears to be u p . Host www . outdoorwire . com ( 6 6 . 2 1 6 . 6 8 . 2 3 ) appears to be up . Host www . inspectorhos t in g . com ( 6 6 . 2 1 6 . 6 8 . 2 4 ) appear s to be u p . Host jwebmedia . com ( 6 6 . 2 1 6 . 6 8 . 2 5 ) appear s to be up . [ ..] Host rs . lwn . net ( 6 6 . 2 1 6 . 6 8 . 4 8 ) appea r s to be up . Host 66 . 2 1 6 . 6 8 . 5 2 appears to be up . Host cuttlef i sh . l aughingsqu id . ne t ( 6 6 . 2 1 6 . 6 8 . 5 3 ) appear s to be up . [. . . ] Nmap done : 2 5 6 I P addr e s s e s ( 1 0 5 hos t s up ) s canned in 1 2 . 6 9 seconds . This example only took 13 seconds, but provides valuable information. In that class C sized address range, 105 hosts are online. From the unrelated domain names all packed into such a small IP space, it is clear that LWN uses a colocation or dedicated server provider. If the LWN machines turn out to be highly secure, an attacker might go after one of those neighbor machines and then perform a local ethernet attack with tools such as Ettercap or Dsniff. An ethical use of this data would be a network administrator considering moving machines to this provider. He might e-mail a few of the listed organizations and ask their opinion of the service before signing a long-term contract or making the expensive and disruptive datacenter move. The - s P option sends an ICMP echo request and a TCP ACK packet to port 80 by default. Since unprivileged Unix users (or Windows users without WinPcap installed) cannot send these raw packets, a SYN packet is sent instead in those cases. The SYN packet is sent using a TCP conne c t system call to port 80 of the target host. When a privileged user tries to scan targets on a local ethernet network, ARP requests (-PR) are used unless the -- s e nd - i p option is specified. The - s P option can be combi ned with any of the techniques discussed in Section 3.6, "Host Discovery Techniques" [60] for greater flexibility. If any of those probe type and port number options are used, the default probes (ACK and echo request) are overridden. When strict firewalls are in place between the source host running Nmap and the target network, using those advanced techniques is recommended. Otherwise hosts could be missed when the firewall drops probes or their responses. 3.5.3. Disable Ping (-PN) Another option is to skip the Nmap discovery stage altogether. Normally, Nmap uses this stage to determine active machines for heavier scanning. By default, Nmap only performs heavy probing such as port scans, version detection, or OS detection against hosts that are found to be up. Disabling host discovery with the -PN option causes Nmap to attempt the requested scanning functions against every target IP address specified. So if a class B sized target address space (/ 16) is specified on the command line, all 65,536 IP addresses are 3.5. Host Discovery Controls 59 scanned. Proper host discovery is skipped as with a list scan, but instead of stopping and printing the target list, Nmap continues to perform requested functions as if each target IP is active. There are many reasons for disabling the Nmap ping tests. One of the most common is intrusive vulnerability assessments. One can specify dozens of different ping probes in an attempt to elicit a response from all available hosts, but it is still possible that an active yet heavily firewalled machine might not reply to any of those probes. So to avoid missing anything, auditors frequently perform intense scans, such as for all 65,536 TCP ports, against every IP on the target network. It may seem wasteful to send hundreds of thousands of packets to IP addresses that probably have no host listening, and it can slow scan times by an order of magnitude or more. Nmap must send retransmissions to every port in case the original probe was dropped in transit, and Nmap must spend substantial time waiting for responses because it has no round-trip-time (RTT) estimate for these non-responsive IP addresses. But serious penetration testers are willing to pay this price to avoid even a slight risk of missing active machines. They can always do a quick scan as well, leaving the massive -PN scan to run in the background while they work. Chapter 6, Optimizing Nmap Performance [ 1 35 ] provides more performance tuning advice. Another frequent reason given for using -PN is that the tester has a list of machines that are already known to be up. So the user sees no point in wasting time with the host discovery stage. The user creates their own list of active hosts and then passes it to Nmap using the - i L (take input from list) option. This strategy is rarely beneficial from a time-saving perspective. Due to the retransmission and RTT estimate issues discussed in the previous paragraph, even one unresponsive IP address in a large list will often take more time to scan than a whole ping scanning stage would have. In addition, the ping stage allows Nmap to gather RTT samples that can speed up the following port scan, particularly if the target host has strict firewall rules. While specifying -PN is rarely helpful as a time saver, it is important if some of the machines on your list block all of the discovery techniques that would otherwise be specified. Users must strike a balance between scan speed and the possibility of missing heavily cloaked machines. 3.6. Host Discovery Tech n iques There was a day when finding whether an IP address was registered to an active host was easy. Simply send an ICMP echo request (ping) packet and wait for a response. Firewalls rarely blocked these requests, and the vast majority of hosts obediently responded. Such a response has been required since 1989 by RFC 1 1 22, which clearly states that "Every host MUST implement an ICMP Echo server function that receives Echo Requests and sends corresponding Echo Replies". Unfortunately for network explorers, many administrators have decided that security concerns trump RFC requirements and have blocked ICMP ping messages.Example 3.8 uses an ICMP-only Nmap ping scan against six popular Web sites, but receives only two responses. This demonstrates that hosts can no longer be assumed unavailable based on failure to reply to ICMP ping probes. The "-sP -PE " options in this example specify an ICMP-only ping scan. The -R option tells Nmap to perform reverse-DNS resolution against all hosts, even down ones. 60 3.6. Host Discovery Techniques Example 3.8. Attempts to ping popular Internet hosts f nmap -sP -PE -R -v microsof t . com ebay . com c i t ibank . com goog l e . com \ s l a s hdot . or g yahoo . com Starting Nmap ( http : / / nmap . or g ) Host origin2 . microsoft . com ( 2 0 7 . 4 6 . 2 5 0 . 2 5 2 ) appears to be down . Host pages . ebay . com ( 6 6 . 1 3 5 . 1 9 2 . 8 7 ) appears to be down . Host ldl -www . c i t i corp . com ( 1 9 2 . 1 9 3 . 1 9 5 . 1 3 2 ) appears to be down . Host 216 . 2 3 9 . 5 7 . 9 9 appea r s to be up . Host slashdot . org ( 6 6 . 3 5 . 2 5 0 . 1 5 0 ) appears to be down . Host w3 . rc . dcn . yahoo . com ( 2 1 6 . 1 0 9 . 1 2 7 . 3 0 ) appea r s to be up . Nmap done : 6 I P addres s e s ( 2 h o s t s up ) scanned in 3 . 7 6 s e conds Fortunately, Nmap offers a wide variety of host discovery techniques beyond the standard ICMP echo request. They are described in the following sections. Note that if you specify any of the -P options discussed in this section, they replace the default discovery probes rather than adding to them. 3.6.1 . TCP SYN Ping (-P S ) The -PS option sends an empty TCP packet with the SYN flag set. The default destination port is 80 (configurable at compile time by changing DEFAULT_TCP_PROBE_PORT_SPEC in nmap . h), but an alternate port can be specified as a parameter. A list of ports may be specified (e.g. -PS2 2 - 2 5 , 8 0 , 1 1 3 , 1 0 5 0 , 3 5 0 0 O), in which case probes will be attempted against each port in parallel. The SYN flag suggests to the remote system that you are attempting to establish a connection. Normally the destination port will be closed, and a RST (reset) packet will be sent back. If the port happens to be open, the target will take the second step of a TCP three-way-handshake by responding with a SYN/ACK TCP packet. The machine running Nmap then tears down the nascent connection by responding with a RST rather than sending an ACK packet which would complete the three-way-handshake and establish a full connection. 2 Nmap does not care whether the port is open or closed. Either the RST or SYN/ACK response discussed previously tell Nmap that the host is available and responsive. On Unix boxes, only the privileged user root is generally able to send and receive raw TCP packets. For unprivi leged users, a workaround is automatically employed whereby the connect system call is initiated against each target port. This has the effect of sending a SYN packet to the target host, in an attempt to establish a connection. If connect returns with a quick success or an ECONNREFUSED failure, the underlying TCP stack must have received a SYN/ACK or RST and the host is marked available. If the connection attempt is left hanging until a timeout is reached, the host is marked as down. This workaround is also used for 1Pv6 connections, as raw 1Pv6 packet building support is not yet available in Nmap. Example 3.8 failed to detect four out of six machines because they did not respond to ICMP echo requests. Repeating the experiment using a SYN probe to port 80 (HTIP) garners responses from all six, as shown in Example 3.9. 2rhe RST packet is sent by the kernel of the machine running Nmap in response to the unexpected SYN/ACK, not by Nmap itself. 3.6. Host Discovery Techniques 61 Example 3.9. Retry host discovery using port 80 SYN probes # nmap - s P -PS B O -R -v microsoft . com ebay . com c i t ibank . com goog l e . com \ s lashdot . or g y ahoo . com S t a r t i n g Nmap ( http : / / nmap . or g ) Host o r i g i n 2 . microsoft . com ( 2 0 7 . 4 6 . 2 4 9 . 2 5 2 ) appears to be up . Hos t pages . ebay . com Ho s t Host Host Host Nmap ( 6 6 . 1 35 . 1 92 . 8 7) appears t o be up . 1d1 -www . c i t i corp . com ( 1 9 2 . 1 9 3 . 1 9 5 . 1 3 2 ) appears to be up . 2 1 6 . 2 3 9 . 5 7 . 9 9 appea r s to be up . s l as hdot . or g ( 6 6 . 3 5 . 2 5 0 . 1 5 0 ) app e a r s to be up . w3 . rc . dcn . yahoo . com ( 2 1 6 . 1 0 9 . 1 2 7 . 3 0 ) appear s to be up . done : 6 I P addre s se s ( 6 h o s t s up ) s canned in 0 . 4 8 seconds In addition to detecting all six machines, the second run is much faster. It takes less than half a second because the machines are scanned in parallel and the scan never times out waiting for a response. This test is not entirely fair because these are all popular web servers and thus can be expected to listen on TCP port 80. However, it still demonstrates the point that different types of hosts respond to different probe types. Nmap supports the usage of many scan types in parallel to enable effective scanning of diverse networks. 3.6.2. TCP ACK Ping {-PA ) The TCP ACK ping i s quite similar to the SYN ping. The difference, as you could likely guess, i s that the TCP ACK Hag is set instead of the SYN Hag. Such an ACK packet purports to be acknowledging data over an established TCP connection, but no such connection exists. So remote hosts should always respond with a RST packet, disclosing their existence in the process. The -PA option uses the same default port as the SYN probe (80) and can also take a list of destination ports in the same format. If an unprivileged user tries this, or an IPv6 target is specified, the connect workaround discussed previously is used. This workaround is imperfect because connect is actually sending a SYN packet rather than an ACK. The reason for offering both SYN and ACK ping probes is to maximize the chances of bypassing firewalls. Many administrators configure routers and other simple firewalls to block incoming SYN packets except for those destined for public services like the company web site or mail server. This prevents other incoming connections to the organization, while allowing users to make unobstructed outgoing connections to the Internet. This non-stateful approach takes up few resources on the firewall/router and is widely supported by hardware and software filters. As just one example of the prevalence of this method, the Linux Netfilter/iptables firewall software offers the - - s yn convenience option, which the man page describes as follows. Only match TCP packets with the SYN bit set and the ACK and RST bits cleared. Such packets are used to request TCP connection i nitiation; for example, blocking such packets coming in an interface will prevent incoming TCP connections, but outgoing TCP connections will be unaffected. It is equivalent to --tcp-Hags SYN,RST,ACK SYN. When firewall rules such as this are in place, SYN ping probes (-PS) are likely to be blocked when sent to closed target ports. In such cases, the ACK probe excels by cutting right through these rules. 62 3.6. Host Discovery Techniques Another common type of firewall uses stateful rules that drop unexpected packets. This feature was initially found mostly on high-end firewalls, though it has become much more common over the years. The Linux Netfilter/iplables system supports this through the - s t a t e option, which categorizes packets based on connection state as described in the following man page excerpt: - Possible states are INVALID meaning that the packet is associated with no known connection, ESTABLISHED meaning that the packet is associated with a connection which has seen packets in both directions, NEW meaning that the packet has started a new connection, or otherwise associated with a connection which has not seen packets in both directions, and RELATED meaning that the packet is starting a new connection, but is associated with an existing connection, such as an FTP data transfer, or an ICMP error. The ACK probe is unlikely to work against firewalls taking this approach, as such an unexpected packet will be classified in the INVALID state and probably dropped. Example 3.10 shows an attempted ACK ping against Microsoft. Their stateful firewall drops the packet, leading Nmap to wrongly conclude that the host is down. The SYN probe has a much better chance of working in such cases. This raises the question of which technique to use when the firewall rules of the target networks are unknown or inconsistent. The proper answer is usually both. Nmap can send SYN and ACK probes to many ports in parallel, as well as performing other host discovery techniques at the same time. This is further discussed in Section 3.7, "Putting It All Together: Host Discovery Strategies" [66). Example 3.10. Attempted ACK ping against Microsoft J. nmap -sP -PA www . microsoft . com Start i ng Nmap ( http : / / nmap . or g ) Warning : Hostname www . microsoft . com r e solves to 5 I P s . U s i n g 2 0 7 . 4 6 . 1 9 2 . 2 5 4 . Note : Host seems down . I f i t i s rea l l y u p , but blocking p i n g probe s , t r y -PN imap don e : 1 I P addr e s s ( 0 h o s t s up ) s canned i n 2 . 2 2 s econds 3.6.3. UDP Ping (-PU ) Another host discovery option i s the UDP ping, which sends a n empty (unless - -dat a - l e ngth is specified) UDP packet to the given ports. The port list takes the same format as with the previously discussed -PS and -PA options. If no ports are specified, the default is 3 1 ,338. This default can be configured at compile-time by changing DEFAULT_UDP_PROBE_PORT_SPEC in nmap . h. A highly uncommon port is used by default because sending to open ports is often undesirable for this particular scan type. Upon hitting a closed port on the target machine, the UDP probe should elicit an ICMP port unreachable packet in return. This signifies to Nmap that the machine is up and available. Many other types of ICMP errors, such as host/network unreachables or TTL exceeded are indicative of a down or unreachable host. A lack of response is also interpreted this way. If an open port is reached, most services simply ignore the empty packet and fail to return any response. This is why the default probe port is 31 ,338, which is highly unlikely to be in use. A few services, such as the Character Generator (chargen) protocol, will respond to an empty UDP packet, and thus disclose to Nmap that the machine is available. The primary advantage of this scan type is that it bypasses firewalls and filters that only screen TCP. For example, I once owned a Linksys BEFW 1 1 S4 wireless broadband router. The external interface of this device 3.6. Host Discovery Techniques 63 filtered all TCP ports by default, but UDP probes would still elicit port unreachable messages and thus give away the device. 3.6.4. ICMP Ping Types (-PE, -PP , and -PM) In addition to the unusual TCP and UDP host discovery types discussed previously, Nmap can send the standard packets sent by the ubiquitous ping program. Nmap sends an ICMP type 8 (echo request) packet to the target IP addresses, expecting a type 0 (echo reply) in return from available hosts. As noted at the beginning of this chapter, many hosts and firewalls now block these packets, rather than responding as required by RFC 1 1 22. For this reason, ICMP-only scans are rarely reliable enough against unknown targets over the Internet. But for system administrators monitoring an internal network, this can be a practical and efficient approach. Use the -PE option to enable this echo request behavior. While echo request is the standard ICMP ping query, Nmap does not stop there. The ICMP standard (RFC 792) also specifies timestamp request, information request, and address mask request packets as codes 13, 15, and 17, respectively. While the ostensible purpose for these queries is to learn information such as address masks and current times, they can easily be used for host discovery. Nmap does not currently implement information request packets, as they are not widely supported (RFC I 122 insists that "a host SHOULD NOT implement these messages"). Timestamp and address mask queries can be sent with the -PP and -PM options, respectively. A timestamp reply (ICMP code 14) or address mask reply (code 1 8) discloses that the host is avai lable. These two queries can be valuable when administrators specifically block echo request packets, but forget that other ICMP queries can be used for the same purpose. 3.6.5. I P Protocol Ping (-PO ) The newest host discovery option i s the IP protocol ping, which sends IP packets with the specified protocol number set in their IP header. The protocol list takes the same format as do port lists in the previously discussed TCP and UDP host discovery options. If no protocols are specified, the default is to send multiple IP packets for ICMP (protocol 1 ), IGMP (protocol 2), and IP-in-IP (protocol 4). The default protocols can be configured at compile-time by changing DEFAULT_PROTO_PROBE_PORT_SPEC in nmap . h. Note that for the ICMP, IGMP, TCP (protocol 6), and UDP {protocol 17), the packets are sent with the proper protocol headers while other protocols are sent with no additional data beyond the IP header (unless the data l e ngth option is specified). -- - This host discovery method looks for either responses using the same protocol as a probe, or ICMP protocol unreachable messages which signify that the given protocol isn't supported by the destination host. Either type of response signifies that the target host is alive. 3.6.6. ARP Scan (-PR) One of the most common Nmap usage scenarios is to scan an ethernet LAN. On most LANs, especially those using private address ranges granted by RFC 1918, the vast m ajority of IP addresses are unused at any given time. When Nmap tries to send a raw IP packet such as an ICMP echo request, the operating system must determine the destination hardware (ARP) address corresponding to the target IP so that it can address the ethernet frame properly. This requires it to issue a series of ARP requests. This is shown in Example 3.11, where a ping scan is attempted against a local ethernet host. The - - s end-ip option tells Nmap to send IP 64 3.6. Host Discovery Techniques level packets (rather than raw ethernet) even though it is a local network. Wireshark output of the three ARP requests and their timing has been pasted into the session. Example 3.11. Raw IP ping scan of an offiine target t nmap -n -sP - - send-ip 1 9 2 . 1 6 8 . 3 3 . 3 7 Starting Nmap ( http : / / nmap . or g 0 . 000000 00 : 0 1 : 2 9 : f 5 : 2 7 : f2 -> f f : f f : f f : f f : f f : f f ARP Who h a s 1 9 2 . 1 6 8 . 3 3 . 3 7 ? 0 . 999836 0 0 : 0 1 : 2 9 : f 5 : 2 7 : f 2 - > f f : f f : f f : ff : f f : f f ARP Who h a s 1 9 2 . 1 6 8 . 3 3 . 3 7 ? 1 . 9 9 9 6 8 4 0 0 : 0 1 : 2 9 : f 5 : 2 7 : f 2 - > f f : f f : f f : f f : f f : f f ARP Who has 1 9 2 . 1 6 8 . 3 3 . 3 7 ? Note : Host seems down . I f i t i s rea l l y up, but blocking p i ng probe s , t r y -PN Nmap done : 1 IP addr e s s ( 0 h o s t s up ) s canned in 2 . 0 4 seconds This example took more than two seconds to finish because the (Linux) OS sent three ARP requests, one second apart, before giving up on the host. Given that ARP replies usually come within a couple millisecorids, multi-second waits are excessive. Decreasing this timeout period is no priority for OS vendors because the vast majority of packets are sent to hosts that actually exist. Nmap, on the other hand, must send packets to 16 million IPs when given a target such as 10.0.0.0/8. A two second wait for each becomes a huge delay even though many targets are pinged in parallel. There is another problem with raw IP ping scans on LANs. When a destination host is found to be unresponsive as in the previous example, the source host generally adds an incomplete entry for that destination IP in its kernel ARP table. ARP table space is finite, and some operating systems react badly when it fills up. When Nmap is used in raw IP mode ( s e nd i p), Nmap sometimes has to wait several minutes for ARP cache entries to expire before it can continue with host discovery. -- - ARP scanning resolves both problems by putting Nmap in control. Nmap issues the raw ARP requests and handles retransmission and timeout periods at its own discretion. The system ARP cache is bypassed. Example 3.12 shows the difference. This ARP scan takes just over a tenth of the time taken by its IP equivalent. Example 3.12. ARP ping scan of an offtine target # nmap -n -sP -PR - -packet -trace - - s end-eth 1 9 2 . 1 6 8 . 3 3 . 3 7 Starting Nmap ( h t tp : / / nmap . or g ) SENT ( 0 . 0 06 0 s ) ARP who-has 1 9 2 . 1 6 8 . 3 3 . 3 7 t e l l 1 9 2 . 1 6 8 . 0 . 1 0 0 SENT ( 0 . 1 1 8 0 s ) ARP who-has 1 9 2 . 1 6 8 . 3 3 . 3 7 t e l l 1 9 2 . 1 6 8 . 0 . 1 0 0 Note : Host seems down . I f i t i s r e a l l y up , but block ing ping probe s , t r y -PN Nmap done : 1 IP addr e s s ( 0 h o s t s up ) s canned i n 0 . 2 3 s econds In Example 3.12, neither the -PR or - - s e nd-eth options have any effect. This is because ARP is the default scan type when scanning ethernet hosts that Nmap detects are on a local ethernet network. This includes traditional wired ethernet as well as 802. 1 1 wireless networks. Not only is ARP scanning more efficient as discussed above, it is also more accurate. Hosts frequently block IP-based ping packets, but they generally cannot block ARP requests or responses and still communicate on the network. Even if different ping types (such as -PE or -PS) are specified, Nmap uses ARP instead for any of the targets which are on the same LAN. If you absolutely don't want to do an ARP scan, specify - - send-ip as shown in Example 3. 1 1 , "Raw IP ping scan of an offline target" [65]. 3 .6. Host Discovery Techniques 65 Gi ving Nmap control to send raw ethernet frames also allows Nmap to control the source MAC address. If you have the only PowerBook in the room at a security conference and a massive ARP scan is initiated from a MAC address registered to Apple, heads may turn in your direction. You can spoof your MAC address with the - - s po o f -mac option, as discussed in Section 10.4.8, "MAC Address Spoofing" [270). 3.6.7. Default Combi nation I f none of these host discovery techniques are chosen, Nmap uses a default which is equivalent to the -PA -PE arguments for Windows or privileged (root) Unix users. Attentive readers know that this means a TCP ACK packet to port 80 and an ICMP echo request query are sent to each machine. An exception to this is that an ARP scan is used for any targets which are on a local ethernet network. For unprivileged Unix shell users, the default is equivalent to -PS (a TCP connect call against port 80 of the target hosts). For security auditing, I recommend using a more comprehensive set of ping types, such as those discussed in the section called "Designing the ideal combinations of probes" [70] . 3.7. Putting It Al l Together: Host Discovery Strateg ies 3.7.1 . Related Options Previous sections describe the major options used to control the Nmap host discovery phase and customize the techniques used. However, there are many more general Nmap options which are relevant here. This section provides a brief description of how these option flags relate to ping scanning. See Chapter 15, Nmap Reference Guide [373] for complete descriptions of each option. -v (same as - -verbo s e ) By default, Nmap usually only prints active, responsive hosts. Verbose mode causes Nmap to print down hosts, as well as extra information about active ones. - - s ource-port (same as -g) Setting a constant source port works for ping scanning (TCP and UDP) as it does with other Nmap features. Some naive firewall administrators make a ruleset exception in order to keep DNS (port 53) or FTP-DATA (port 20) working. Of course this opens a hole big enough to drive an Nmap ping scan through. Section 10.4.2, "Source Port Manipulation" [266] provides further details on this technique. -n, - R The -n option disables a l l DNS resolution, while the -R option enables DNS queries for a l l hosts, even down ones. The default behavior is to l imit DNS resolution to active hosts. These options are particularly important for ping scanning because DNS resolution can greatly affect scan times. - -dn s - servers [ , [ , ] ] (Servers to use for reverse DNS queries) By default Nmap will try to determine your DNS servers (for rDNS resolution) from your resolv.conf fi le (Unix) or the Registry (Win32). Alternatively, you may use this option to specify alternate servers. This option is not honored if you are using - - s y s t em-dn s or an IPv6 scan. Using multiple DNS servers is often faster and more stealthy than querying just one. The best performance is often obtained by specifying all of the authoritative servers for the target IP space. 66 . . . 3.7. Putting It All Together: Host Discovery Strategies --data-length This option adds random bytes of data to every packet, and works with the TCP, UDP, and ICMP ping scan types (for privileged users scanning IPv4). This helps make the scan less conspicuous and more like the packets generated by the ubiquitous ping diagnostics program. Several intrusion detection systems (IDS), including Snort, have alerts for zero-byte ping packets. This option evades those alerts. An option value of 32 makes an echo request look more like it came from Windows, while 56 simulates the default Linux ping. --ttl Setting the outgoing TTL is supported for privileged users doing 1Pv4 ping scans. This can be useful as a safety precaution to ensure a scan does not propagate beyond the local network. It can also be used to simulate a native ping program even more convincingly. Some enterprise networks suffer from known routing loops which they can't easily fix. Reducing the outgoing TTL with - - t t l helps to reduce router CPU load when loops are encountered. Canned timing options ( - T 3 , - T 4 , - T S , etc.) Higher -T values speed up ping scanning, just as they speed other Nmap features. With a moderately fast and reliable connection between the source and target networks (i.e. anything more than a dial-up modem), the -T4 option is recommended. --max-par a l l e l i sm, --min-pa r a l l e l i sm These affect how many probes may be outstanding at once. With the default ping type (two probes), the parallelism value is roughly the number of machines scanned in parallel. Reducing the ping techniques to one probe per host (e.g. -PE) will double the number of hosts scanned at once for a given parallelism level, while increasing to four probes per host (e.g. -PE - P S 2 2 , 1 1 3 , 50 0 0 O) halves it. Most users simply stick to the canned timing options such as - T 4 . --min-rtt-t imeout, - -ma x - r t t - t ime out , - - i n i t i a l - r t t - t ime out These options control how long Nmap waits for a ping response. Input options (- i L , - i R ) Host input options are supported as in the rest of Nmap. Users often combine the input-from-list (- i L) option with -PN to avoid ping-scanning hosts that are already known to be up. Before doing this in an attempt to save time, read Section 3 .5.3, "Disable Ping (-PN)" [59]. The - i R option chooses hosts at random from allocated Internet IP space. It takes as an argument the number of random hosts you wish to scan. Use zero for a never-ending (until you abort or kill the Nmap process) scan. Output options (-oA, -oN, -oG, -ox, etc.) All of the Nmap output types (normal, grepable, and XML) support ping scanning. Chapter 13, Nmap Output Formats [337] further describes how they work. --randomi ze-ho s t s Shuffling the host scan order with this option may make the scan less conspicuous, though i t also can make the scan output a bit more difficult to follow. --reason The normal Nmap output indicates whether a host is up or not, but does not describe which discovery test(s) the host responded to. For this detail, add the --reason option. The results can be confusing for host discovery since Nmap does not always try every probe. It stops as soon as it gets a first response. 3.7. Putting It All Together: Host Discovery Strategies 67 So Nmap might report an ICMP echo response from a host during the run, but then a RST response might be received first during a second run and lead Nmap to report that. --pack e t - t r a c e When you want even more details than - - r e a s o n provides, try --pack e t - t r a c e . This option shows every packet send and received by Nmap, including details such as sequence numbers, TTL values, and TCP flags. -D Decoys are fully supported fo r privileged 1Pv4 ping scans, camouflaging the true attacker. -6 The TCP connect-based ping scans (-PS) support the 1Pv6 protocol, including multi-port mode (such as -P S 2 2 , 8 0 , 1 1 3 . - s , - eAs with other functions of Nmap, the source address and sending device can be specified with these options. General options By default, unless - s P or - s L are specified, Nmap moves onto more intrusive scanning after the host discovery stage. Thus many dozens of general port scanning, OS detection, and version detection options can be used. See the reference guide or relevant chapters for further information. 3. 7.2. C hoosing and Com b i n i ng Ping Options Effective scanning requires more than knowing all o f the options described i n this and previous sections. Users must understand how and when to use them to suit the target network topology and scanning goals. TCP probe and port selection The TCP ping options are some o f the most powerful discovery techniques in Nmap. A n administrator may be able to get away with blocking ICMP echo request packets without affecting most users, but a server absolutely must respond to SYN packets sent to the public services it provides. Meanwhile, ACK packets often get through non-stateful firewal ls. I would recommend using both of SYN and ACK probes, using lists of ports based on any knowledge you might have of the target networks as well as more generally popular ports. A quick scan of more than 10,000 IP addresses across the Internet showed the ports in Table 3.2 to be particularly valuable. Of hosts with a default-drop fi lter (the hardest type to reach), these are the 14 ports most likely to be accessible (open or closed). Table 3.2. Most valuable TCP probe ports, in descending order of accessibility. Port number I Service Reasoning 8 0 /http The prevalence o f Web servers o n the Internet leads many newbies t o believe that the Web is the Internet. 2 5 / smtp Mail is another Internet "killer app" that companies allow through their firewalls. 22/ssh SSH seems to have finally surpassed Telnet as the standard for remote terminal administration. 68 3.7. Putting It All Together: Host Discovery Strategies Port number I Service Reasoning 4 4 3 /https SSL is a popular way for web sites to protect confidential directory information. 2 1 / ftp This file transfer protocol lives on, though many firewall administrators would not mourn its passing. 1 1 3 / aut h The auth (identd) service allows servers (usually mail or IRC) to request the username of clients connected to them. Administrators often leave this port unfiltered to avoid long timeouts that can occur when firewal l rules prevent servers from connecting back to port 1 1 3. Using this port for ping scanning can sometimes lead to false positives, as some administrators have been known to configure their firewalls to forge RST packets back in response to auth queries to any IP on their network, even when no machine exists at that IP. Administrators do this to avoid server timeouts while still preventing the ports from being accessed. 23 /telnet Many devices still offer this administrative interface, though it is a security nightmare. 5 3 / doma i n Domain name servers are extremely widespread. 5 5 4 / rtsp Real Time Stream Control Protocol is used by media servers, including QuickTime and RealServer. 3389/ms-term-server Microsoft Terminal Services allows users (and sometimes hackers) to access applications and data on a remote computer. 1 7 2 3 /pptp Point-to-Point Tunneling Protocol is often used to implement VPN solutions on Microsoft Windows. 3 8 9 / ldap The Lightweight Directory Access Protocol is often used to store contact directories and the like. 6 3 6 / ldap s s l LDAP over SSL i s popular for accessing confidential information. 2 5 6 /FWl-securemote Checkpoint Firewall- I devices often have this administration port open. In addition to popular ports such as the ones in the list above, choosing at least one high-numbered port is recommended. Many poorly configured firewalls only have default-deny for the privileged ports, meaning those below 1,024. I usually pick a high numbered port out of the air, such as 40,000 or 10,042, to catch machines behind this sort of firewall. In choosing the ports to probe, remember to emphasize platform diversity. If you are limiting your ping scan to two ports, HTTP (80) and SSH (22) are probably better than HTTP (80) and HTTPS (443) because the latter two are related web services, and many machines that have HTTPS will often have HTTP available anyway. Finding two accessible ports on the same machine is no better for ping scanning purposes than finding one. The goal is to choose ports so that a broad set of hosts will match at least one of them. Note that the valuable port table does not include many client-oriented ports such as the ubiquitous Windows SMB port 135. The primary reason is that this table only looked at hosts behind default-deny firewalls, where the vast m ajority of ports are filtered. In those situations, Windows ports such as 1 35- 139 and 445 are usually blocked. When these machines are not behind a firewall, the open ports are unimportant for ping scanning because the thousands of closed ports work just as well. 3. Z Putting It All Together: Host Discovery Strategies 69 UDP port selection In selecting UDP ports, remember that an open port is unlikely to respond to the probes. Unfiltered ports are desired. To avoid open ports, you might consider excluding common UDP services like DNS (port 53) and SNMP ( 161 ). On the other hand, firewal l rules are often so broad that those probes (particularly to port 53) might get through and hit a closed port. So I would recommend choosing at least port 53 and an arbitrarily selected high-numbered port such as 37,452. ICMP probe selection For ICMP, the standard ping (echo request) is usually worth trying. Many administrators specifically allow this because it is useful for debugging or because RFC 1 1 22 requires it. I would also use at least one of the address mask or timestamp requests. These are valuable for networks where administrators intentionally block echo request packets, but forget about other ICMP queries. Designing the ideal com binations of probes How all of these ping types are combined into a ping scan strategy depends on characteristics of the target network and on the scan goals. For internal networks, the default ping type usually works well . The default is also fi ne for most casual scanning, where missing an occasional host is no big deal. Adding more probes can help catch those occasional stealthy machines, at the expense of making the ping scan take a bit longer. Time taken is roughly proportional to the number of probes sent to each machine. For security scans of target networks over the Internet, adding more probes is usually advisable. Try to include a diverse set of the techniques discussed previously. Here is a set of ping options that should catch the vast majority of hosts: -PE -PA -PS2 1 , 2 2 , 2 3 , 2 5 , 8 0 , 1 1 3 , 3 1 3 3 9 -PAS O , 1 1 3 , 4 4 3 , 1 0 0 4 2 . Adding in - - s our ce-port 53 might be worthwhile as well. How much better will the results be, and how much longer will it take? That depends on the target network, of course, but the Nmap random target selection option (- i R) makes it easy to perform a quick test. Example 3 . 1 3 shows Nmap generating 50,000 random IP addresses and then performing a default ping scan. You should remember that the default is a TCP ACK packet to port 80, and an ICMP echo request packet. 70 3.7. Putting It All Together: Host Discovery Strategies Example 3.13. Generating 50,000 IP addresses, then ping scanning with default options runap -n -sL - i R 5 0 0 0 0 -oN - I grep " not s canned " I awk ' { p r i n t $ 2 ) ' \ I sort -n > 50K_I P s head - 5 50K_IPs � 10 0 . 1 4 7 . 9 ) , 1 00 . 1 4 8 . 1 1 9 3 . 1 0 . 1 60 . 3 3 3 . 1 0 . 2 01 . 1 1 3 . 1 01 . 154 . 1 3 9 t runap -SP - T 4 - i L 5 0K_I P s ' runap -SP -T4 - i L 5 0K_I P s - s -oA 5 0KHos t s_Def a u l t Pi n g ft;arting Nmap ( http : / / nmap . or g ) Rost dialup- 4 . 1 7 7 . 9 . 7 5 . SanDiego l . Leve l 3 . ne t ( 4 . 1 7 7 . 9 . 7 5 ) appe a r s to be up . Host dialup-4 . 1 8 1 . 1 0 0 . 9 7 . SanJose l . Leve l 3 . net ( 4 . 1 8 1 . 1 0 0 . 9 7 ) appears to be up . Bost firewal l 2 . baymoun ta i n . com ( 8 . 7 . 9 7 . 2 ) appea r s to be up . (thousands of l i nes cut J Host 22 2 . 9 1 . 1 2 1 . 2 2 appears to be up . Nmap done : 5 0 0 0 0 IP addr e s s e s ( 3 3 4 8 h o s t s up ) scanned in 1 5 9 8 . 0 7 s e conds t Scanning the 50,000 address took just under 27 minutes, and 3,348 hosts were detected. Most of the DNS names were already in cache due to a previous scratch run, though it still would have likely been faster had DNS resolution been disabled with - n . To determine the effects of using a wider range of ping techniques, the same 50K hosts were rescanned with 13 probes per port rather than the default of two. As shown in Example 3. 14, Nmap was able to detect 1 , 1 25 (34%) more hosts. It took about 7 1 minutes, which is more than 2.5 times as long. Given all the new hosts detected, that extra time was well spent. Note that not all of the new hosts wil l be legitimate. Increasing the number of ping probes increases the chances that Nmap will hit network artifacts that make a non-existent host appear to be active. Firewalls that return a RST for SYN or ACK packets to port 1 1 3 are one example of this. Example 3.14. Repeating ping scan with extra probes nmap -SP -PE -PP -PS2 1 , 2 2 , 2 3 , 2 5 , 8 0 , 1 1 3 , 3 1 3 3 9 -PA 8 0 , 1 1 3 , 4 4 3 , 1 0 0 4 2 \ -T4 --source-port 53 - i L 5 0K_I Ps -oA 5 0KHo s t s_ExtendedP ing Starting Nmap ( http : / / nmap . or g ) Bost sim71 2 4 . agni . l i nden lab . com ( 8 . 1 0 . 1 4 4 . 1 2 6 ) appears to be up . Bost firewa l l 2 . baymoun t a i n . com ( 8 . 7 . 9 7 . 2 ) appears to be up . Host 12 . 1 . 6 . 2 0 1 appea r s to be up . Host psor . inshea l t h . com ( 1 2 . 1 3 0 . 1 4 3 . 4 3 ) appea r s to be up . [thousands of hos t s cut ) Bost ZM0 8 8 0 1 9 . ppp . di on . ne . j p ( 2 2 2 . 8 . 8 8 . 1 9 ) appea r s t o be u p . Host 22 2 . 9 2 . 1 3 6 . 1 0 2 appears to be up . Nmap done : 5 0 0 0 0 IP addres se s ( 4 4 7 3 hos t s up ) s canned in 4 2 5 9 . 2 8 s e conds t When performing security audits for clients, I normally start TCP analysis with a port scan against the most common 1000 ports (the default) with comprehensive ping scan options like those shown in Example 3.14, "Repeating ping scan with extra probes" [7 1 ] . Such a scan does not take particularly long, allowing me to quickly start working. I also launch - PN (ping disabled) scans against all 65K TCP ports in the background while I work. When they finish, which may be days later, I compare them to my initial quick scan and investigate any new ports or machines found. 3.7. Putting It All Together: Host Discovery Strategies 71 3.8. Host Discovery Code Algorith ms One of the greatest benefits of Open Source software like Nmap is that curious users are always able to study the source code when they want answers about its operation. The highest level ping scanning function is next h o s t (in t arget s . cc, which calls ma s s p i ng to initialize a list of targets. Ma s sp i ng in turn passes the list off to u l t r a_s can (in s can_engine . cc). Ultra_scan is Nmap's general-purpose scanning function and does all the hard work of sending, receiving, and interpreting packets. For more on ul t ra_scan see Section 5 . 13, "Scan Code and Algorithms" [ 1 28). While source code analysis is the only way to truly get the complete picture of Nmap operation down to every trivial detail, it is not always the easiest approach to understanding Nmap. In many cases, the most effective way to explore Nmap's behavior given a set of command-line options is to add the --pa c k e t - t r a c e option, which prints out all of the packets sent and received by Nmap. Because the source code and the --pack e t - t r ace option are excellent resources for learning the nitty-gritty details of Nmap operation, I'll only discuss how host discovery works at a high level here. When Nmap is executed, it may be passed networks containing hundreds of thousands or even millions of hosts. So Nmap breaks them into blocks that are small enough to deal with at one time (dozens up to a few thousand hosts). u l t r a_scan then works its way through the block, sending packets as fast as its congestion controls allow. Rather than sending all the probes requested by the user to each host all at once, Nmap sends the first probe to all the targets, then the second probe, and so on. When a conclusive response to a probe is received, that host is marked as up or down as appropriate and no further probes are sent to it. A target host which fails to respond to any probes, even after retransmissions, is marked as down. Nmap waits until every host has either received a conclusive response or has timed out. Eventually, Nmap runs out of new hosts in the block and the number of outstanding probes dwindles to zero as retransmissions complete. The ping scanning subsystem returns the results so that Nmap can begin port scanning or any other requested probing of the target machines. When Nmap finishes completely with a block of hosts, it prints the results and passes the next block to the ping scanner. Multiple hosts, usually with multiple probes per host, are handled in parallel . The number of outstanding probes and timeout periods are modified in real-time based on network latency and reliability. The u l t r a_scan performance algorithms are further· described in Section 5 . 13, "Scan Code and Algorithms" [ 1 28). 72 3.8. Host Discovery Code Algorithms C hapter 4. Port Scanning Overview 4.1 . Introd uction to Port Sca n n i ng While Nmap has grown in functionality over the years, it began as an efficient port scanner, and that remains its core function. The simple command nmap scans the most commonly used 1 ,000 TCP ports on the host , classifying each port into the state open, c l o s ed, f i l t e red, u n f i l tered, open l f i l ter ed, or c l osed l f i l t e red. 4.1 .1 . What Exactly is a Port? Ports are simply a software abstraction, used to distinguish between communication channels. Similar to the way IP addresses are used to identify machines on networks, ports identify specific applications in use on a single machine. For example, your web browser will by default connect to TCP port 80 of machines in HTIP URLs. If you specify the secure HTIPS protocol instead, the browser will try port 443 by default. Nmap works with two protocols that use ports: TCP and UDP. A connection for each protocol is uniquely identified by four elements: source and destination IP addresses and corresponding source and destination ports. All of these elements are simply numbers placed in the headers of each packet sent between hosts. The protocol is an eight-bit field, which specifies what type of packet is contained in the IP data (payload) section. For example, TCP is protocol number six, and UDP is 17. 1Pv4 addresses have a length of 32-bits, while ports are 16-bits long. IPv6 addresses are 128-bits in length. Further IP, TCP, and UDP header layout details can be found in Section 7, "TCP/IP Reference" [xxvi]. Because most popular services are registered to a well-known port number, one can often guess what services open ports represent. Nmap includes an nmap- s e r v i c e s file, containing the well-known service for registered port and protocol numbers, as well as common ports for trojan backdoors and other applications that don't bother registering with the Internet Assigned Numbers Authority (IANA). Nmap prints this service name for reference along with the port number. Because the port number field is 16-bits wide, values can reach 65,535. The lowest possible value, zero, is invalid. The Berkeley sockets API, which defines how programs are usually written for network communication, does not allow port zero to be used as such. Instead, it interprets a port zero request as a wildcard, meaning that the programmer does not care which is used. The system then chooses an available port number. For example, programmers rarely care what source port number is used for an outgoing connection. So they set it to zero and let the operating system choose one. While port zero is invalid, nothing stops someone from specifying it in the header field. Some malicious trojan backdoors listen on port zero of compromised systems as a stealthy way to offer illegitimate access without appearing on most port scans. To combat this, Nmap does allow scanning of port zero when it is specified explicitly (e.g. -p 0 - 6 5 5 3 5 ). The first class of valid ports, numbers one through 1,023, are known as reserved ports. Unix systems (unlike Windows) require that applications have special (root) privileges in order to bind to and listen on these ports. The idea is to allow remote users to trust that they are connecting to a valid service started by an administrator and not by some wicked, unprivileged user. If the registered port for SSH was 2,222 instead of 22, a malicious 4.1. Introduction to Port Scanning 73 user could start up a rogue SSH daemon on that port, collecting passwords from anyone who connects. As most common server applications listen on reserved ports, these are often the most fruitful to scan. The ephemeral port range is another class of ports. This pool of ports is made available by the system for allocation as needed. When an application specifies port zero (meaning "any port"), the system chooses a port from this range. The range varies by operating system, and is usually configurable. It should contain at least a couple thousand ports to avoid running out when many concurrent connections are open. The Nmap connect scan can use hundreds at a time as it scans every specified port on each target machine. On Linux, you can view or set the range using the file /pr o c / s y s / net / ipv4 / ip_l oca l_port_r ange. Example 4.1 shows that on my Linux system, the range is 32,768 to 61,000 . Such a large range should be sufficient in almost all cases, but I expand it just to demonstrate how to do so. Example 4.1. Viewing and increasing the ephemeral port range on Linux fel i x / # 32768 fel i x / # fel i x / # · 10000 fel i x / # cat /proc / s y s / ne t / i pv 4 / ip_l oca l_port_range 6 1000 echo " 1 0 0 0 0 6 5 0 0 0 " > /proc / s y s / net / ipv 4 / ip_l oca l_port_range cat /proc / s y s /net / i pv 4 / ip_loca l_port_range 65000 SunRPC ports are often found i n the ephemeral range. Other applications open ephemeral ports temporarily for a file transfer or other event. FTP clients often do this when requesting an active mode transfer. Some P2P and instant messaging clients do so as well. The IANA has their own port classification scheme, which differs slightly from the vernacular of this book. Their authoritative port list at http://www. iana.org!assignmentslport-numbers divides the space into the following three classes: well-known ports These are reserved ports (within the range of I to 1 ,023, as discussed above) which have been registered with the IANA for a certain service. Familiar examples are ports 22, 25, and 80 for the services SSH, SMTP, and HTTP, respectively. registered ports These ports fall within the range 1 ,024 to 49, 151 and have been registered with the IANA in the same way the well known ports have. Most of these are not as commonly used as the well-known ports. The key difference is that unprivileged users can bind to these ports and thus run the services on their registered port. Users cannot do so on most platforms for well-known ports, since they reside in the reserved port range. dynamic and/or private ports The IANA reserves the port numbers from 49152 through 65535 for dynamic uses such as those discussed in the ephemeral ports section. Proprietary services that are only used within a company may also use these ports. When this book mentions registered or well-known ports without any reference to the IANA, it usually means ports registered with Nmap in the nmap- s e r v i c e s file, regardless of whether they fall in the reserved port range. 74 4.1. Introduction to Port Scanning Nmap's port registration file (nmap- s e r v i c e s ) contains empirical data about how frequently each TCP or UDP port is found to be open. By default, Nmap scans the 1 ,000 most popular ports of each protocol it is asked to scan. There are many options for specifying an alternate set of ports (by frequency or by listing them explicitly), as described in Section 4.3.2, "Selecting Ports to Scan" [83]. 4.1 .2. What Are the Most Popular Ports? I spe nt the Summer of 2008 scanning tens of millions of Internet hosts and collecting data from enterprises to determine how frequently each port number is found open. It is important to be familiar with the most common service ports, and also interesting to see which ones made the list. The following two lists provide the top TCP and UDP ports as determined by our empirical scan data. The l isted service is the one found in our nmap- servi ces file. We try to list the most common service for each port there, though of course it is possible for a port to be used for different things. Top 20 (most commonly open) TCP ports 1. Port 80 (HTTP)-If you don't even know this service, you're reading the wrong book. This accounted for more than 14% of the open ports we discovered. 2. Port 23 (Telnet)-Telnet lives on (particularly as an administration port on devices such as routers and smart switches) even though it is insecure (unencrypted). 3. Port 443 (HTTPS)-SSL-encrypted web servers use this port by default. 4. Port 21 (FfP)-FfP, like Tel net, is another insecure protocol which should die. Even with anonymous FTP (avoiding the authentication sniffing worry), data transfer is still subject to tampering. S. Port 22 (SSH)-Secure Shell, an encrypted replacement for Telnet (and, in some cases, FTP). 6. Port 25 (SMTP)-The Standard Mail Transfer Protocol (also insecure). 7. Port 3389 (ms-term-server)-Microsoft Terminal Services administration port. 8. Port 1 1 0 (POP3)-Post Office Protocol version 3 for email retrieval (insecure). 9. Port 445 (Microsoft-DS)-For SMB communication over IP with MS Windows services (such as file/printer sharing). n Port 139 (NetBIOS-SSN)-NetBIOS Session Service for communication with MS Windows services (such as file/printer sharing). This has been supported on Windows machines longer than 445 has. I L Port 143 (IMAP)-lnternet Message Access Protocol version 2. An insecure email retrieval protocol. 12. Port 53 (Domain)-Domain Name System (DNS), an insecure system for conversion between host/domain names and IP addresses. 13. Port 135 (MSRPC)-Another common port for MS Windows services. 14. Port 3306 (MySQL)-For communication with MySQL databases. 4. l . Introduction to Port Scanning 75 15. Port 8080 (HTIP-Proxy)-Commonly used for HTIP proxies or as an alternate port for normal web servers (e.g. when another server is already listening on port 80, or when run by unprivileged UNIX users who can only bind to high ports). 16. Port 1723 (PPTP)-Point-to-point tunneling protocol (a method of implementing VPNs which is often required for broadband connections to ISPs). 17. Port 1 1 l (RPCBind)-Maps SunRPC program numbers to their current TCP or UDP port numbers. 18. Port 995 (POP3S)-POP3 with SSL added for security. 19. Port 993 (IMAPS)-IMAPv2 with SSL added for security. � Port 5900 (VNC)-A graphical desktop sharing system (insecure). Top 20 (most commonly open) UDP ports I . Port 631 (IPP)-Internet Printing Protocol. 2. Port 161 (SNMP)-Simple Network Management Protocol. 3. Port 137 (NETBIOS-NS)-One of many UDP ports for Windows services such as file and printer sharing. 4. Port 1 23 (NTP)-Network Time Protocol. 5 . Port 138 (NETBIOS-DGM)-Another Windows service. 6. Port 1434 (MS-SQL-DS)-Microsoft SQL Server. 7. Port 445 (Microsoft-DS)-Another Windows Services port. 8. Port 135 (MSRPC)-Yet Another Windows Services port. 9. Port 67 (DHCPS)-Dynamic Host Configuration Protocol Server (gives out IP addresses to clients when they join the network). 10. Port 53 (Domain)-Domain Name System (DNS) server. 1 1. Port 1 39 (NETBIOS-SSN)-Another Windows Services port. 12 Port 500 (ISAKMP)-The Internet Security Association and Key Management Protocol is used to set up IPsec VPNs. 13. Port 68 (DHCPC)-DHCP cl ient port. 14. Port 520 (Route)-Routing Information Protocol (RIP). 15. Port 1900 (UPNP)-Microsoft Simple Service Discovery Protocol, which enables discovery of Universal plug-and-play devices. 16. Port 4500 (nat-t-ike)-For negotiating Network Address Translation traversal while initiating IPsec connections (during Internet Key Exchange). 76 4.1. Introduction to Port Scanning 17. Port 514 (Syslog)-The standard UNIX log daemon. 18. Port 491 52 (Varies)-The first of the !ANA-specified dynamic/private ports. No official ports may be registered from here up until the end of the port range (65536). Some systems use this range for their ephemeral ports, so services which bind a port without requesting a specific number are often allocated 49152 if they are the first program to do so. 19. Port 162 (SNMPTrap)-Simple Network Management Protocol trap port (An SNMP agent typically uses 161 while an SNMP manager typically uses 1 62). ll Port 69 (TFTP)-Tri vial 4.1 .3. File Transfer Protocol . What is Port Scanning ? fort scanning is the act of remotely testing numerous ports to determine what state they are in. The most :111teresting state is usually open, meaning that an application is listening and accepting connections on the port. Many techniques are available for conducting such a scan. Chapter 5, Port Scanning Techniques and Algorithms [95] explains the circumstances under which each is most appropriate. While many port scanners have traditionally lumped all ports into the open or closed states, Nmap is much more granular. It divides ports into six states. These states are not intrinsic properties of the port itself, but describe how Nmap sees them. For example, an Nmap scan from the same network as the target may show port 1 3 5 /tcp as open, while a scan at the same time with the same options from across the Internet might show that port as fiI tered. The six port states recognized by Nmap open An application is actively accepting TCP connections or UDP packets on this port. Finding these is often the primary goal of port scanning. Security-minded people know that each open port is an avenue for attack. Attackers and pen-testers want to exploit the open ports, while administrators try to close or protect them with firewalls without thwarting legitimate users. Open ports are also interesting for non-security scans because they show services available for use on the network. Before you get too excited about an open port, note that it is possible that the application is protected with a TCP wrapper (tcpd) or that the application itself is configured to only service approved client IP addresses. Such cases still leave more attack surface than a closed port. A closed port is accessible (it receives and responds to Nmap probe packets), but there is no application listening on it. They can be helpful in showing that a host is on line and using an IP address (host discovery, or ping scanning), and as part of OS detection. Because closed ports are reachable, they may be worth scanning later in case some open up. Administrators may want to consider blocking such ports with a firewall so they appear in the filtered state, discussed next. Nmap cannot determine whether the port is open because packet filtering prevents its probes from reaching the port. The filtering could be from a dedicated firewall device, router rules, or host-based firewall software. These ports frustrate attackers because they provide so little information. Sometimes they respond with ICMP error messages such as type 3 code 13 (destination unreachable: communication 4.1. Introduction to Port Scanning 77 administratively prohibited), but filters that simply drop probes without responding are far more common. This forces Nmap to retry several times just in case the probe was dropped due to network congestion rather than filtering. This sort of filtering slows scans down dramatically. unfiltered The unfiltered state means that a port is accessible, but Nmap is unable to determine whether it is open or closed. Only the ACK scan, which is used to map firewall rulesets, classifies ports into this state. Scanning unfiltered ports with other scan types such as Window scan, SYN scan, or FIN scan, may help resolve whether the port is open. openlfiltered Nmap places ports in this state when it is unable to determine whether a port is open or filtered. This occurs for scan types in which open ports give no response. The lack of response could also mean that a packet filter dropped the probe or any response it elicited. So Nmap does not know for sure whether the port is open or being filtered. The UDP, IP protocol, FIN, NULL, and Xmas scans classify ports this way. closedlfiltered This state is used when Nmap is unable to determine whether a port is closed or filtered. It is only used for the IP ID Idle scan discussed in Section 5 . 1 0, "TCP Idle Scan (-sl)" [ 1 1 7] . While Nmap attempts to produce accurate results, keep in mind that a l l o f its insights are based o n packets returned by the target machines (or firewalls in front of them). Such hosts may be untrustworthy and send responses intended to confuse or mislead Nmap. Much more common are non-RFC-compliant hosts that do not respond as they should to Nmap probes. FIN, NULL, and Xmas scans are particularly susceptible to this problem. Such issues are specific to certain scan types and so are discussed in the relevant sections of Chapter 5, Port Scanning Techniques and Algorithms [95]. 4.1 .4. Why Scan Ports? Port scanning is not only performed for fun and amusement. There are numerous practical benefits to regularly scanning your networks. Foremost among these is security. One of the central tenets of network security is that reducing the number and complexity of services offered reduces the opportunity for attackers to break in. Most remote network compromises come from exploiting a server application listening on a TCP or UDP port. In many cases, the exploited application is not even used by the targeted organization, but was enabled by default when the machine was set up. Had that service been disabled, or protected by a firewall, the attack would have been thwarted. Realizing that every open port is an opportunity for compromise, attackers regularly scan targets, taking an inventory of all open ports. They compare this list of listening services with their list of favorite exploits for vulnerable software. It takes just one match to compromise a machine, creating a foothold that is often used to infest the whole network. Attackers who are less discriminate about who they target will often scan for just the default port of an exploitable application. This is much faster than scanning every port, though the service will be missed when running on a non-default port. Such attackers are often derided as "script kiddies", because they often know little more about security than how to run an exploit script written by someone more skilled. Across many organizations, such attackers are bound to find vulnerable hosts. They can be quite a nuisance, though their sheer numbers and relentless pounding against Internet-accessible machines often drive people to patch systems quickly. This reduces the likelihood of more serious, targeted attacks succeeding. 78 4.1. Introduction to Port Scanning An important defense against these crackers is for systems administrators to scan their own networks regularly with tools such as Nmap. Take the list of open ports, and shut down any services that aren't used. Ensure that those which must remain available are fully patched and that you are on the vendor's security notification list. Firewall rules should be added where possible, limiting access to only legitimate users. Hardening instructions are available on the Web for most popular applications, reducing the cracker's opportunity even further. Nmap cannot do most of this for you, but it creates the list of available services to start out with. Some administrators try to use netstat instead, but that doesn't scale well. It requires access to every machine, and some mobile machines are easy to miss. Plus, you can't run netstat on your average wireless access point, VoIP phone, or printer. In addition, there is always the risk that a compromised machine will have a trojaned netstat which gives out false information. Most of the modern rootkits i nstalled by attackers include this functionality. Relying solely on Nmap is a mistake too. A combination of careful design, configuration auditing, and regular scanning is well advised. While security is the most common reason for port scanning, administrators often find that it suits other purposes as well . Creating an inventory of machines and the services they offer can be useful for asset tracking, network design, policy compliance checks, software license tracking, availability testing, network debugging, and more. 4.2. A Quick Port Sca n n i ng Tutorial One of my goals in developing Nmap is to keep the most common usage simple, while retaining the flexibility for custom and advanced scans. This is accomplished with the command-line interface by offering dozens . of options, but choosing sane defaults when they are not specified. A newbie can start out with a command as simple as nmap . Meanwhile, advanced users sometimes specify so many options that their terminal line wraps around. A similar balance must be struck with command output. The most i mportant results should stick out even to the occasional user who hasn't even read the man page. Yet the output should be comprehensive and concise enough to suit professional penetration testers who run Nmap against thousands of machines daily. Users smart enough to read this book or the Nmap source code benefit from greater control of the scanner and insights into what Nmap output really means. This tutorial demonstrates some common Nmap port scanning scenarios and explains the output. Rather than attempt to be comprehensive, the goal is simply to acquaint new users well enough to understand the rest of this chapter. The simplest Nmap command is just nmap by itself. This prints a cheat sheet of common Nmap options and syntax. A more interesting command is nmap , which does the following: I. Converts from a hostname into an IPv4 address using DNS . If an IP address is specified instead of a hostname this lookup is skipped. 2. Pings the host, by default with an ICMP echo request packet and a TCP ACK packet to port 80, to determine whether it is up and running. If not, Nmap reports that fact and exits. I could have specified -PN to skip this test. See Chapter 3, Host Discovery (Ping Scanning) [47). 3. Converts the target IP address back to the name using a reverse-DNS query. Because of the way DNS works, the reverse name may not be the same as the < t a rge t > specified on the command-line. This query can be skipped with the -n option to improve speed and stealthiness. 4.2. A Quick Port Scanning Tutorial 79 4. Launches a TCP port scan of the most popular 1 ,000 ports listed in nmap- s e r v i c e s . A SYN stealth scan is usually used, but connect scan is substituted instead for non-root Unix users who lack the privileges necessary to send raw packets. 5. Prints the results to standard output in normal human-readable format, and exits. Other output formats and locations (files) can be specified, as described in Chapter 1 3, Nmap Output Formats [337). Example 4.2 displays the results when scanme.nmap.org is used as . Example 4.2. Simple scan: nmap scanme.nmap.org # nmap scanme . nmap . org S t a r t i n g Nmap ( h t tp : / / nmap . or g I nt e r e s t i n g por t s on s canme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) : Not s hown : 9 9 4 f i l t ered por t s PORT STATE SERV I CE 2 2 / t cp open ssh 2 5 / t cp c l osed smtp 5 3 / t cp open doma i n 7 0 / t cp c l o sed gopher 8 0 / t cp open http 1 1 3 / tcp c l osed auth Nmap done : 1 I P add r e s s ( 1 host u p ) scanned i n 4 . 9 9 seconds The first output line in Example 4.2 simply gives the URL for downloading Nmap. The time Nmap started and version number are normally provided as well, though these were are generally removed from this book for consistency and to avoid line wrapping. The next line provides the target IP address (1Pv4 in this case), and reverse DNS name (also known as the PTR record) if it is available. Nmap promises to show the "interesting ports", though all ports scanned are accounted for. The ports considered most interesting because they are open or in a rarely-seen state for that host are itemized individually. When many ports are in a single non-open state, they are considered a default state, and aggregated onto a single line to avoid diluting the results with thousands of uni nteresting entries. In this case, Nmap notes that 994 ports are filtered. The interesting ports table comes next, and provides the key scan results. The columns vary depending on options used, but in this case provide the port number and protocol, state, and service protocol for each port. The service here is j ust a guess made by looking up the port in nmap - s e r v i ce s . The service would be listed as unk n own if any of the ports had no name registered in that file. Three of these ports are open and three are closed. Finally, Nmap reports some basic timing stats before it exits. These stats are the number of targets specified, the number of those that the ping scan found to be up, and the total time taken. While this simple command is often all that is needed, advanced users often go much further. In Example 4.3, the scan is modified with four options. -p O - asks Nmap to scan every possible TCP port, -v asks Nmap to be verbose about it, -A enables aggressive tests such as remote OS detection, service/version detection, and the Nmap Scripting Engine (NSE). Finally, - T 4 enables a more aggressive timing policy to speed up the scan. 80 4.2. A Quick Port Scanning Tutorial Example 4.3. More complex: nmap -pO- -v -A -T4 scanme.nmap.org # nmap -pO- -v -A - T 4 s canme . nmap . org Starting Nmap ( http : / /nmap . or g ) Completed Ping Scan at 0 0 : 0 3 , O . O l s e lapsed ( 1 t o t a l hos t s ) Scanning scanme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) ( 6 5 5 3 6 por t s ] Discovered open port 2 2 / t cp o n 6 4 . 1 3 . 1 3 4 . 5 2 Di scovered open port 5 3 / tcp on 6 4 . 1 3 . 1 3 4 . 5 2 Discovered open port 8 0 / tcp on 6 4 . 1 3 . 1 3 4 . 5 2 SYN Stea lth Scan T iming : About 6 . 2 0 % done ; ETC : 0 0 : 1 1 ( 0 : 0 7 : 3 3 rema i n i n g ) Completed SYN Stealth Scan at 0 0 : 1 0 , 4 6 3 . 5 5 s elapsed ( 6 5 5 3 6 t o t a l por t s ) Completed Service scan at 0 0 : 1 0 , 6 . 0 3 s elapsed ( 3 services on 1 ho s t ) Initiat ing OS det ect i on ( t ry # 1 ) against scanme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) Initiat ing Traceroute at 0 0 : 1 0 64 . 1 3 . 1 3 4 . 5 2 : gue s s i n g hop d i s tance at 9 Completed SCRIPT ENG I NE at 0 0 : 1 0 , 4 . 0 4 s e lapsed Host s canme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) appea r s to be up . . . good . Interest ing por t s on scanme . nmap . org ( 6 4 . 1 3 . 1 3 4 . 5 2 ) : Not shown : 6 5 5 3 0 f i l tered por t s PORT STATE SERV I CE VER S I ON OpenSSH 4 . 3 ( prot oco l 2 . 0 ) 2 2 /tcp open ssh 2 5/tcp closed smtp 53/tcp open doma i n I SC B I ND 9 . 3 . 4 70 /tcp closed gopher 80/tcp open http Apache ht tpd 2 . 2 . 2 ( ( Fedora ) ) I _ HTML t i t l e : Go ahead and S canMe ! 113/ tcp closed auth Device type : gene r a l purpose Running : Li nux 2 . 6 . X OS details : L i nux 2 . 6 . 2 0 - 1 ( Fedora Core 5 ) Upt ime gue s s : 2 . 4 5 7 days ( s ince Thu Sep 1 8 1 3 : 1 3 : 2 4 2 0 0 8 ) TCP Sequence Predict ion : D i f f i c u l t y= 2 0 4 ( Good luck ! ) IP ID Sequence Generat ion : A l l zeros TRACEROUTE ( us i ng port 8 0 / t cp ) HOP RTT ADDRESS • [First eight hops cut for brev i t y ) 9 1 0 . 3 6 metroO . sv . svcol o . com ( 2 0 8 . 1 8 5 . 1 6 8 . 1 7 3 ) 10 1 0 . 2 9 scanme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) Nmap done : 1 I P addre s s ( 1 host up ) s canned in 4 7 7 . 2 3 s econds Raw packet s sent : 1 3 1 4 3 2 ( 5 . 7 8 3 MB ) I Rcvd : 3 5 9 ( 1 4 . 9 6 4KB ) Nmap certainly provided the requested verbosity in Example 4.3! Fortunately the extra output is easy to understand. The first 1 3 new lines are runtime information letting the user know what is happening as she stares expectantly at the terminal, hoping for good news. What constitutes good news depends on whether she is a systems administrator who has to fix problems, a pen-tester who needs some issues to report on, or a black-hat cracker trying to exploit them. About a dozen similar lines were removed for brevity. The "discovered open port" lines provide as-it-happens notification of open ports so that she can start banging on them before the scan even finishes. The "scan timing" line provides a completion time estimate, so she knows whether to keep staring at the screen or have lunch. Since network conditions (latency, congestion, bandwidth, etc.) and packet filtering rules vary so much, the same scan options may take 30 seconds to 4.2. A Quick Port Scanning Tutorial 81 complete against one host and 45 minutes against another. If you want the current time estimate while scanning, just press enter. The port table shows no new ports. All the extra ports scanned are in the fil tered state, raising the fil tered port total from 994 to 65,530. While there are no new itemized ports, the entries have changed. A new VERS I ON column provides the application name and version details for the listening service. This comes from service detection, one of the features enabled by the -A option. Another feature of service detection is that all of the service protocols in the SERV I C E column have actually been verified. In the previous scan, they were based on the relatively flimsy heuristic of an nmap- s e r v i c e s port number lookup. That table lookup happened to be correct this time, but it won't always be. Another feature added by -A is the Nmap Scripting Engine, which is discussed in depth in Chapter 9, Nmap Scripting Engine [205]. The only script shown here is HTML t i t l e. Dozens of other scripts exist, but none found useful output for this machine. The traceroute results were also added by -A. This option is more efficient and more powerful than most traceroute programs since probes are performed in parallel and Nmap uses scan results to determine a favorable probe type (TCP packets to port 80 in this case). Most of the remaining new lines come from OS detection (also enabled by -A), which is discussed in depth in Chapter 8, Remote OS Detection [ 1 7 1 ] . The final line shows that all this extra info came at a price-the scan took almost 100 times longer than Example 4.2, "Simple scan: nmap scanme.nmap.org" [80] to complete (477 seconds compared to 5). 4.3. Command- l i ne Flags While the tutorial showed how simple executing an Nmap port scan can be, dozens of command-line flags are available to make the system more powerful and flexible. This section covers only options that relate to port scans, and often describes only the port-scanning-related functionality of those options. See Chapter 15, Nmap Reference Guide [373] for a comprehensive list of option flags and everything they do. 4.3.1 . Selecting Scan Techniques One of the first considerations when contemplating a port scan is deciding what techniques to use. Nmap offers about a dozen such methods and this section provides a brief summary of them. Full coverage comes in the next chapter. Only one scan method may be used at a time, except that UDP scan ( - sU) may be combined with any one of the TCP scan types. As a memory aid, port scan type options are of the form - s , where is a prominent character in the scan name, usually the first. The one exception to this is the deprecated FTP bounce scan (-b). By default, Nmap performs a SYN Scan, though it substitutes a connect scan if the user does not have proper privileges to send raw packets (requires root access on Unix) or if 1Pv6 targets were specified. Port scanning methods supported by Nmap TCP SYN Stealth ( - s s ) This is far and away the most popular scan type because it the fastest way to scan ports of the most popular protocol (TCP). It is stealthier than connect scan, and it works against all functional TCP stacks (unlike some special-purpose scans such as FIN scan). 82 4.3. Command-line Flags TCP Connect (-s T) Connect scan uses the system call of the same name to scan machines, rather than relying on raw packets as most of the other methods do. It is usually used by unprivileged Unix users and against IPv6 targets because SYN scan doesn't work in those cases. UDP (-sU) Don't forget UDP ports-they offer plenty of security holes too. TCP FIN, Xmas, and Null (- sF, -sX, - sN) These special purpose scan types are adept at sneaking past firewalls to explore the systems behind them. Unfortunately they rely on target behavior that some systems (particularly Windows variants) don't exhibit. TCP ACK ( - sA) ACK scan is commonly used to map out firewall rulesets. In particular, it helps understand whether firewall rules are stateful or not. The downside is that it cannot distinguish open from closed ports. TCP Window (- sW) Window scan is like ACK scan, except that it is able to detect open versus closed ports against certain machines. TCP Maimon (- sM) This obscure firewall-evading scan type is similar to a FIN scan, but i ncludes the ACK flag as well . This allows i t to get by more packet filtering firewalls, with the downside that it works against even fewer systems than FIN scan does. TCP Idle (-s I ) Idle scan is the stealthiest scan type of all, and can sometimes exploit trusted IP address relationships. Unfortunately, it is also slow and complex. IP protocol (- so) Protocol scan determines which IP protocols (TCP, ICMP, IGMP, etc.) are supported by the target machine. This isn't technically a port scan, since it cycles through IP protocol numbers rather than TCP or UDP port numbers. Yet it still uses the -p option to select scanned protocol numbers, reports its results with the normal port table format, and even uses the same underlying scan engine as the true port scanning methods. So it is close enough to a port scan that it belongs here. TCP FfP bounce (-b ) This deprecated scan type tricks FTP servers into performing port scans by proxy. Most FTP servers are now patched to prevent this, but it is a good way to sneak through restrictive firewalls when it works. 4.3.2. Selecting Ports to Scan Nmap's port registration file (nmap - s e r v i c e s ) contains empirical data about how frequently each TCP « UDP port is found to be open. This data was collected by scanning tens of millions of Internet addresses, ncombining those results with internal scan data contributed by large enterprises. By default, Nmap scans 1,000 most popular ports of each protocol it is asked to scan. Alternatively, you can specify the -F (fast) "on to scan only the J OO most common ports in each protocol or - - t op-po r t s to specify an arbitrary ber of ports to scan. 4.3. Command-line Flags 83 · When none of these canned port sets suit your needs, an arbitrary list of port numbers can be specified on the command-line with the -p option. The syntax of the -p option can be complex, and is best described with examples. Port selection examples with the -p option -p 2 2 Scan a single port (in this case port 22) by specifying just that number as the -p argument. -p s s h Port names may be specified rather than numbers. Note that a name may match multiple ports. -p 2 2 , 2 5 , 8 0 Multiple ports may be separated with commas. Note that no protocol is specified, so these same port numbers will be used for whatever scan methods are specified on the command-line. If a TCP scan such as SYN scan ( - s s) is specified, TCP ports 22, 25, and 80 are scanned. Those correspond to the services SSH, SMTP, and HTTP, respectively. If a UDP scan is selected (- s U), those three UDP ports are scanned. If both are specified, those three ports are scanned for each protocol, for a total of six scanned ports. With IP protocol scan (- so), those three IP protocols (corresponding to XNS IDP, Leaf- I , and ISO-IP) are scanned. -p 8 0 - 8 5 , 4 4 3 , 8 0 0 0 - 8 0 0 5 , 8 0 8 0 - 8 0 8 5 Port ranges may be specified by separating the beginning and end port with a hyphen. Multiple ranges or individual ports can be specified with commas. This option scans ports 80, 8 1 , 82, 83, 84, 85, 443, 8000, etc. Based on the port numbers, this user is probably scanning TCP and looking for web servers. -p- 1 0 0 , 6 0 0 0 0 - You can omit the beginning of a range to imply port one, or the end to imply the last port possible (65535 for TCP and UDP, 255 for protocol scan). This example scans ports one through 100, and all ports greater or equal to 60,000. -p- Omit beginning and end numbers to scan the whole range (excluding zero). -pT : 2 1 , 2 3 , 1 1 0 , U : 5 3 , 1 1 1 , 1 3 7 , 1 6 1 Separate lists of TCP and UDP ports can be given by preceding the lists with T: (for TCP) or U:. This example scans three TCP ports (FTP, Telnet, and POP3), and four UDP services (DNS, rpcbind, NetBIOS, and SNMP). Specifying both TCP and UDP ports only matters if you also tell Nmap to do a UDP scan (- sU) and one of the TCP scan methods, such as - s s, - sA, or - s F . -p h t t p * Wildcards may b e used to match ports with similar names. This expression matches eight port numbers, including http (80), http-mgmt (280), https (443), and http-proxy (8080). Depending on your command shell, you may need to escape the asterisk so it isn't treated as a filename glob. -p 1 - 1 0 2 3 , [ 1 0 2 4 - J Enclosing a range in brackets causes those port numbers to be scanned only if they are registered in nmap- s er v i c e s . In this example, all the reserved ports ( 1 - 1 ,023), plus all the higher ports registered in nmap- s e r v i c e s . That was Nmap's default behavior before nmap- s e r v i c e s was augmented with open port frequency data for more precise selection. 84 4.3. Command-line Flags 4.3.3. Ti mi ng-related Options Port scanning i s often the most time consuming part of an Nmap scan (which might also include OS detection, version detection, and NSE scripts). While Nmap tries to be quick and efficient by default, manual optimization often helps. Nmap offers dozens of options for tailoring scan intensity and speed to match your exact needs. This section lists the most important options for optimizing port scan times. Options which take an amount of time are given in milliseconds unless you append s (seconds), m (minutes), or h (hours) to the value. For further details on any of these options, see Section 1 5. 1 1 , "Timing and Performance" [394]. A much more thorough treatment, with examples and best-practices for improving Nmap performance is available in Chapter 6, Optimizing Nmap Performance [ 1 35]. Top port scan performance options -TO through -TS These timing templates affect many variables, offering a simple way to adjust overal l Nmap speed from very slow (-T 0) to extremely aggressive ( -T 5 ). A timing template may be combined with the more granular options describe below, and the most granular option takes precedence. --min-rtt-t imeout, - -max - r t t - t i meout, - - i n i t i a l - r t t -t imeout The minimum, maximum, and initial amount of time that Nmap will wait for a port scan probe response. --ho st-t imeout Asks Nmap to give up on hosts that take more than the given amount of time to scan. --min-rate, --max -rate Sets the floor and ceiling, respectively, to the number of probe packets Nmap sends per second. --max -retr i e s Specifies the maximum number o f port scan probe retransmissions to a single port. --min-hostgroup, --max-ho s t group Sets the minimum and maximum number of hosts that Nmap will port scan in parallel. --mi n-para l l e l i sm, - -max -par a l l e l i sm Limits the minimum or maximum number of port scan probes (across all hosts scanned concurrently) that Nmap may have outstanding. --scan-delay, --max - s can-de l ay Asks Nmap to wait at least the given amount of time between sending probes to any individual host. The scan delay can grow as Nmap detects packet loss, so a maximum may be specified with --max-scan-de l ay. 4.3.4. Output Format and Verbosity Options Nmap offers the ability to write its reports i n its standard format, a simple line-oriented "grepable" format, or XML. These reports are enabled with the -oN (normal), - oG (grepable), and -ox (XML) options. Each option takes a filename, and they may be combined to output in several formats at once. Several options are available to increase output verbosity. This section lists the most important output-related options and they apply to port scanning. For further details on any of these options, see Section 15.13, "Output" [403]. 4.3. Command-line Flags 85 A much more thorough treatment of output options and formats, with many examples, is available ii Chapter 1 3, Nmap Output Formats [337]. Top Nmap output options applicable to port scans -v Increases the verbosity level, causing Nmap to print more information about the scan in progress. Open ports are shown as they are found and completion time estimates are provided when Nmap thinks a scan will take more than a few minutes. Use it twice or more for even greater verbosity. -d Increases the debugging level, causing Nmap to print out details about i ts operation that can be useful for tracking down bugs or simply understanding how it works. Higher levels result in massive amounts of data. Using the option once sets the debugging level to one, and it is incremented for each additional -d. Or you may follow the -d with the desired level, as in -d5. If you don't see enough information, try a higher level. The maximum effective level is nine. If your screen is flooded with too much debugging data, reduce the level. Reducing scan intensity, such as the number of ports or targets scanned and the features used, can also help to isolate only the debug messages you want. --pa c k e t - t r a c e Causes Nmap to print a summary of every packet sent or received. This is often used for debugging, but is also a valuable way for new users to understand exactly what Nmap is doing under the covers. To avoid printing thousands of lines, you may want to specify a limited number of ports to scan, such as -p2 0 - 3 0 . -oN (normal output) Write output in Nmap's normal format to . This format is roughly the same as the standard interactive output printed by Nmap at runtime. -ox (XML output) Write output in Nmap's XML format to . Normal (human readable) output will still be printed to stdout unless you ask for XML to be directed there by specifying - as . This is the preferred format for use by scripts and programs that process Nmap results. -oG (grepable format output) Write output in Nmap's so-called grepable format to . This tabular format fits the output of each host on a single line, making it easy to grep for open ports, certain operating systems, application names, or other data. Normal output will still be printed to stdout unless you ask for the grepable output to be directed there by specifying - as . While this format works well for parsing with simple grep and awk command-lines, significant scripts and programs should use the XML output instead. The XML format contains substantial information that grepable format has no place for, and extensibility makes XML easier to update with new information without breaking tools that rely on it. -oA (output to all formats) As a convenience, you may specify - oA to store scan results in normal, XML, and grepable formats at once. They are stored in .nmap, .xml, and .gnmap, respectively. As with most programs, you can prefix the filenames with a directory path, such as - / nmap l og s / f oocorp/ on Unix or c : \ hack i n g \ s co on Windows. 86 4.3. Command-line Flags --re sume Resume an aborted scan by specifying the normal (-oN) or grepable (-oG) output fi le which was created during the ill-fated scan. Don't use any options other than - - r e sume, as Nmap will use the ones specified in the output file. It then parses the file and resumes scanning (and logging to the file) at the host which the previous Nmap execution was working on when it ceased. --append-output Tells Nmap to append scan results to any output files specified (with arguments such as -oN or - ox) rather than overwriting them. --open Only show open ports in the Nmap interesting port tables. Firewal l and I DS Evasion Options 4.3.5. Nmap offers many options for sneaking past IDSs undetected or evading firewall rules. For a n overview, see Section 15.12, "Firewall/IDS Evasion and Spoofing" [399). For a comprehensive look at firewall and IDS evasion techniques, along with practical examples, see Chapter 10, Detecting and Subverting Firewalls and Intrusion Detection Systems [257) . Specifying Targets 4.3.6. To scan a single host (or a few of them), simply add their names or IP addresses to the end of your Nmap command line. Nmap also has a structured syntax to make scanning large networks easy. You can give Nmap a file listing targets, or even ask Nmap to generate them randomly. This is all described in Section 3.2, "Specifying Target Hosts and Networks" (47). 4.3.7. Miscel laneous Options Here are some options that can be quite handy even though they don't fit into specific categories. The descriptions focus on how each option relates to port scanning. See the Chapter 1 5, Nmap Reference Guide (373) for more comprehensive coverage of each option. -6 Asks Nmap to scan the target using the 1Pv6 protocol. This process is described i n Section 4.4, "1Pv6 Scanning (-6)" [88). -r Nmap randomizes the port scan order by default to make detection slightly harder. The - r option causes them to be scanned in numerical order instead. Tells Nmap to skip the ping test and simply scan every target host provided. Other options for controlling host discovery are described in Chapter 3, Host Discovery (Ping Scanning) [ 47) . --reason Adds a column to the interesting ports table which describes why Nmap classified a port as it did. 4.3. Command-line Flags 87 4.4. 1 Pv6 Sca n n i ng (- 6) Since 2002, Nmap has offered 1Pv6 support for its most popular features. In particular, ping scanning (TCP-only), connect scanning, and version detection all support IPv6. The command syntax is the same as usual except that you also add the -6 option. Of course, you must use IPv6 syntax if you specify an address rather than a hostname. An address might look like 3 f fe : 7 5 0 1 : 4 8 1 9 : 2 O 0 0 : 2 1 O : f 3 f f : f e 0 3 : 1 4 d0, so hostnames are recommended. Example 4.4 shows a typical port scanning session. The output looks the same as it usually does, with the 1Pv6 address on the "interesting ports" line being the only 1Pv6 gi ve away. Example 4.4. A simple 1Pv6 scan # nmap -6 -sV www . eurov6 . or g Start ing Nmap ( http : / / nmap . or g I nt e r e s t i n g por t s on n s l . euro6 i x . com ( 2 0 0 1 : 8 0 0 : 4 0 : 2 a 0 3 : : 3 ) : Not s hown : 9 9 6 c l osed por t s PORT STATE SERV I C E VERS I ON Pure-FTPd 2 1 / t cp open ftp 2 2 / t cp open s s h OpenSSH 3 . 5p l ( protocol 2 . 0 ) 5 3 / t cp open doma i n ! SC B IND 9 . 2 . 1 8 0 / t c p open http Apache httpd Nmap done : 1 I P addr e s s ( 1 host up ) s canned in 5 6 . 7 8 s econds While IPv6 hasn't exactly taken the world by storm, it gets significant use in some countries and most modern operating systems support it. To use Nmap with 1Pv6, both the source and target of your scan must be configured for 1Pv6. If your ISP (like most of them) does not allocate IPv6 addresses to you, free tunnel brokers are widely available and work fine with Nmap. I use the free 1Pv6 tunnel broker service at http://www. tunnelbroker. net. Other tunnel brokers are listed at Wikipedia 1 • 6to4 tunnels are another popular, free approach. Systems that support 1Pv6 don't always have their IPv4 and IPv6 firewall rules in sync. Section 10.4.3, "1Pv6 Attacks" [267) shows a real-life example of reaching ports through IPv6 that are filtered in 1Pv4. 4.5. SOLUTION : Scan a Large Network for a Certai n Open TCP Port 4.5.1 . Problem You wish to quickly find all machines o n a network that have a certain TCP port open. For example, after a new Microsoft ITS vul nerability is found, you might want to scan for all machines with TCP port 80 open and ensure that they aren't running a vulnerable version of that software. Or if you investigate a compromised box and fi nd that the attacker left a backdoor running on port 3 1 337, scanning your whole network for that port might quickly identify other compromised systems. A full (all ports) scan would be done later. 1 http://en.wikipedia.org/wiki/List_of_/Pv6_1u1111el_brokers 88 4.4. IPv6 Scanning (-6) 4.5.2. Sol ution The straightforward way i s to run: nmap -PN -p -oG < logfi lenarne . grunap> Here is a concrete example of searching 4096 IPs for web servers (port 80 open): nmap -PN -p80 -oG logs/pb-port80scan- % D.gnmap 216.163.128.0/20 The "%0'' in the filename is replaced with the numeric date on which the scan was run (e.g. "090107" on September I , 2007). While this scan command works, a little effort choosing appropriate timing values for the network being scanned reduces scan time substantially. The scan above took 1 ,236 seconds, while the optimized version below provided the same results in 869 seconds: IDlap -T4 -PN -p80 --max-rtt-timeout 200 --initial-rtt-timeout 150 --min-hostgroup 512 -oG logs/pb-port80scan2- % D.gnmap 216.163.128.0/20 And much of that time is spent doing reverse-DNS resolution. Excluding that by adding -n to the command-line above reduces the 4096-host scan time to 193 seconds. Being patient for three minutes is far easier than for the 21 minutes taken before. The commands above store grepable-format results in the specified fi le. A simple egrep command will then find the machines with port 80 open: · egrep '["0-9)80/open' logs/pb-port80scan2-*.gnmap The egrep pattern is preceded with ["0-9) to avoid bogus matching ports such as 3 1 80. Of course that can't pen since we are only scanning port 80, but it is a good practice to remember for many-port scans. If you tnly want the IP addresses and nothing else, pipe the egrep output to awk '{print $2}' . . 5.3. Discussion etimes a story i s the best way to understand decisions, such as how I decided upon the command lines the solution section. I was bored at home, and started exploring the network of a popular magazine named 'layboy. Their main site includes a huge trove of images, but most are locked away behind a paid subscription entication system. I was curious as to whether I could find any other systems on their network which up images for free. I figured that they might have staging or development servers which rely on obscurity r than password authentication. While such servers could theoretically listen on any port number, the likely is TCP port 80. So I decide to scan their whole network for that open port as quickly as possible. first step is determining which IP addresses to scan. I perform a whois search of the American Registry Internet Numbers (ARIN) for organizations named Playboy. The results are shown in Example 4.5. 4.5. SOLUTION: Scan a Large Network for a Certain Open TCP Port 89 Example 4.5. Discovering Playboy's IP space core-> whoi s -h who i s . ar i n . ne t n p l a yboy [ Querying who i s . ar in . ne t ] [ wh o i s . ar in . ne t ] OrgName : OrgI D : Addre s s : City : S t at eProv : P o s t a l Code : Countr y : P layboy PLAYBO 6 8 0 N . Lake Shore Drive C h i c ago IL 60611 US NetRange : C I DR : NetName : Net Hand l e : Paren t : NetType : NameServe r : NameServe r : [. . .] 216 . 163 . 128 . 0 - 216 . 163 . 143 . 255 216 . 163 . 128 . 0/20 PLAYBOY-BLK- 1 NET- 2 1 6 - 1 6 3 - 1 2 8 - 0 - 1 NET- 2 1 6 - 0 - 0 - 0 - 0 D i rect A s s i gnment NS l -CH I . PLAYBOY . COM N S 2 -CH I . PLAYBOY . COM This shows 4096 IPs {the net range 216.163.1 28.0/20) registered to Playboy. Using techniques discussed in Section 3.3, "Finding an Organization's IP Addresses" [49) I could have found many more netblocks they control, but 4096 IPs are sufficient for this example. Next I want to estimate latency to these machines, so that Nmap will know what to expect. This isn't required, but feeding Nmap appropriate timing values can speed it up. This is particularly true for single-port -PN scans, such as this one. Nmap does not receive enough responses from each host to accurately estimate latency and packet drop rate, so I will help it out on the command line. My first thought is to ping their main web server, as shown in Example 4.6. Example 4.6. Pinging Playboy's web server for a latency estimate t ping - c s www . playboy . com P ING www . phat . p l a yboy . com ( 2 0 9 . 2 4 7 . 2 2 8 . 2 0 1 ) from 2 0 5 . 2 1 7 . 1 5 3 . 5 6 6 4 bytes f rom free-ch i . p l a yboy . com ( 2 0 9 . 2 4 7 . 2 2 8 . 2 0 1 ) : i cmp_seq=l 6 4 bytes f rom free-ch i . playboy . com ( 2 0 9 . 2 4 7 . 2 2 8 . 2 0 1 ) : i cmp_seq=2 6 4 bytes from free-ch i . p l a yboy . com ( 2 0 9 . 2 4 7 . 2 2 8 . 2 0 1 ) : i cmp_seq=3 6 4 bytes f r om free-ch i . p l ayboy . com ( 2 0 9 . 2 4 7 . 2 2 8 . 2 0 1 ) : i cmp_seq=4 6 4 bytes f rom free-ch i . p l a yboy . com ( 2 0 9 . 2 4 7 . 2 2 8 . 2 0 1 ) : i cmp_seq=5 t ime=5 7 . 5 t ime=5 6 . 7 t ime= 5 6 . 9 t ime=5 7 . 0 t ime= 5 6 . 6 ms ms ms ms ms --- www . phat . p layboy . com ping s t a t i s t i c s --5 packe t s t r ansmitted, 5 received, 0% l os s , t ime 4 0 4 7ms rtt m i n / avg/ma x /mdev = 5 6 . 6 5 2 / 5 7 . 0 0 4 / 5 7 . 5 2 2 / 0 . 3 3 3 ms The maximum round trip time is 58 milliseconds. Unfortunately, this IP address (209.247.228.201 ) is not within the 2 16. 1 63 . 1 28.0/20 netblock I wish to scan. I would normally add this new netblock to the target list, but have already decided to limit my scan to the original 4096 IPs. These times are probably perfectly fi ne to use, but fi nding actual values from IPs on the target network would be even better. I use dig to obtain 90 4.5. SOLUTION: Scan a Large Network for a Certain Open TCP Port Playboy's public DNS records from a nameserver shown in the previous whois query. The output is shown in Example 4.7. Example 4.7. Digging through Playboy's DNS records e�>dig @ n s l - ch i . playboy . com p layboy . com . any <>> DiG 8 . 3 <<>> @ n s l -c h i . p l ayboy . com p l ayboy . com . any yboy . com . yboy . com . yboy . com . yboy . com . ayboy . com . ayboy . com . ayboy . com . lyboy . com . ayboy . com . 10 10 10 lD 10 10 lD 10 10 IN IN IN IN IN IN IN IN IN A MX MX NS NS NS NS NS SOA 2 09 . 24 7 . 22 8 . 2 0 1 1 0 mx . l a . p l a yboy . com . 5 mx . ch i . p l a yboy . com . n s l 5 . c u s t omer . leve l 3 . net . n s 2 1 . c u s t omer . leve l 3 . ne t . n s 2 9 . c u s t omer . leve l 3 . net . n s l - c h i . p layboy . com . n s 2 - c h i . p l ayboy . com . n s l -c h i . p l a yboy . com . d ns . p l a yboy . com . serial 20040920 1 0 refresh 12H retry 2 h 3 0m e xp i r y 2wld m i n i mum lD ) ADDITIONAL SECT I ON : chi . playboy . com . . la . playboy . com . 1-chi . playboy . com . 2-chi . playboy . com . 10 10 lD 10 IN IN IN IN A A A A 2 1 6 . 163 . 1 43 . 4 2 1 6 . 16 3 . 12 8 . 15 2 09 . 247 . 22 8 . 135 6 4 . 2 0 2 . 1 05 . 3 6 Total query t ime : 1 0 7 msec The DNS query reveals two MX (mail) servers within the target 216. 163. 128.0/20 netblock. Since the names mx . chi and mx . la imply that they are in different regions (Chicago and Los Angeles), I decide to test diem both for latency. The ping results are shown in Example 4.8. 4.5. SOLUTION: Scan a Large Network for a Certain Open TCP Port 91 Example 4.8. Pinging the MX servers core - > p i ng - c s mx . chi . p layboy . com P ING mx . ch i . p l a yboy . com ( 2 1 6 . 1 6 3 . 1 4 3 . 4 ) S 6 ( 8 4 ) bytes of dat a . - - - mx . ch i . p l a yboy . com ping s t a t i s t i c s - - S packet s transmitted, 0 received, 1 0 0 % p a c k e t l os s , t ime 4 0 0 0ms core - > ping -cs mx . l a . p l a yboy . com P I NG mx . la . playboy . com ( 2 1 6 . 1 6 3 . 1 2 8 . l S ) S 6 ( 8 4 ) bytes of data . - - - mx . l a . p layboy . com ping stat i s t i c s --S packe t s t ransmitted, 0 received, 1 0 0 % packet l os s , t ime 4 0 l lms Well, that attempt was a miserable failure! The hosts seem to be blocking ICMP ping packets. Since they are mail servers, they must have TCP port 25 open, so I try again using hping22 to perform a TCP ping against port 25, as demonstrated in Example 4.9. Example 4.9. TCP pinging the MX servers cor e # hping2 - - s yn -p 2 S -c S mx . ch i . playboy . com e t h O default rout i n g i n t er face sel ected ( according to /proc ) HPING mx . ch i . p l ayboy . com ( et h O 2 1 6 . 1 6 3 . 1 4 3 . 4 ) : S s e t , 4 0 headers + O 46 byt e s from 2 1 6 . 1 6 3 . 1 4 3 . 4 : f lags=SA seq=O t t l = S l id=l 4 2 2 1 r t t = S 6 . 8 46 byt e s from 2 1 6 . 1 6 3 . 1 4 3 . 4 : f l a g s=SA seq=l t t l = S l id=1 4 2 4 4 r t t = S 6 . 9 4 6 byt e s f rom 2 1 6 . 1 6 3 . 1 4 3 . 4 : flags =SA seq=2 t t l = S l id=l 4 2 7 4 r t t =S 6 . 9 46 bytes from 2 1 6 . 1 6 3 . 1 4 3 . 4 : f l ags =SA seq=3 t t l = S l id= l 4 3 8 3 r t t =6 1 . 8 46 bytes from 2 1 6 . 1 6 3 . 1 4 3 . 4 : f l ags =SA seq= 4 t t l = S l id=l 4 3 8 7 rtt= S 7 . S data bytes ms ms ms ms ms - - - mx . ch i . p l ayboy . com hping s t a t i s t i c - - S pack e t s transmi t t e d , S p a c k e t s rece i ved , 0 % packet l o s s round- t r i p m i n / avg/max S6 . 8 / S 8 . 0 / 6 1 . 8 ms core# hping2 --syn -p 2 S - c S mx . la . p l ayboy . com ethO default rout i n g i n t er face s e lected ( according to /pro c ) H P I NG mx . l a . playboy . com ( et h O 2 1 6 . 1 6 3 . 1 2 8 . l S ) : S set , 4 0 heade r s + 0 data bytes 46 bytes from 2 1 6 . 1 6 3 . 1 2 8 . l S : f l ag s =SA seq=O t t l = S 2 id=S 8 7 2 8 r t t = l 6 . 0 ms 46 bytes from 2 1 6 . 1 6 3 . 1 2 8 . l S : f l ags =SA seq=l t t l = S 2 i d= S 8 7 S 3 r t t = l S . 4 ms 46 byt e s from 2 1 6 . 1 6 3 . 1 2 8 . l S : f l ags=SA seq=2 t t l = S 2 id= S 8 7 9 0 r t t = l 5 . 5 ms 46 bytes from 2 1 6 . 1 6 3 . 1 2 8 . 1 5 : f l ags=SA s eq=3 t t l = 5 2 id=5 8 8 7 0 r t t = l 6 . 4 ms 4 6 bytes from 2 1 6 . 1 6 3 . 1 2 8 . 1 5 : f lags=SA s eq= 4 t t l= 5 2 id=S 8 9 0 7 r t t = l S . 5 ms - - - mx . l a . p l ayboy . com hping s t at i s t i c --S pack e t s t ransmi t ted, S packets rece ived , 0 % packet l o s s r ound- t r i p m i n / avg/max = 1 5 . 4 / 1 5 . 8 / 1 6 . 4 ms These are the results I was looking for. The LA host never takes more than 16 mill iseconds to respond, while the Chicago one takes up to 62 milliseconds. This is not surprising, given that I am probing from a machine in California. It pays to be cautious, and latency can increase during heavy scanning, so I decide to let Nmap wait up to 200 milliseconds for responses. I'll have it start with a timeout of 150 ms. So I pass it the options 2 http://www.hpi11g.org 92 4.5. SOLUTION: Scan a Large Network for a Certain Open TCP Port --max-rtt-t ime out 2 0 0 - - i n i t i a l - r t t -t ime out 1 5 0 . To set a generally aggressive timjng mode, I specify -T4 at the beginning of the line. Since I value minimizing completion time of the whole scan over minimizing the amount of time before the first batch of host results is returned, I specify a large scan group size. The option - -mi n-ho s t gr o up 5 1 2 is specified so that at least 5 1 2 IPs will be scanned in parallel (when possible). Using an exact factor of the target network size (4096) prevents the small and less efficient 96-host block which would occur at the end if l speci fied - -mi n-ho s t gr o up 5 0 0. All of these timing issues are explained in much more depth in Chapter 6, Optimizing Nmap Performance [ 1 35]. There is no need to waste time with a prior ping stage, since a ping would take as long as the single-port scan itself. So -PN is specified to disable that stage. Substantial time is saved by skipping reverse-DNS resolution with the -n argument. Otherwise, with ping scanning disabled, Nmap would try to look up all 4096 IPs. I am searching for web servers, so I request port 80 with - p 8 0 . Of course I will miss any HITP servers running on non-standard ports such as 81 or 8080. SSL servers on port 443 won't be found either. One could add them to the -p option, but even one more port would double the scan time, which is roughly proportional to the number of ports scanned. The final option is -oG followed by the filename in which I want grepable results stored. I append the target network to the command, then press enter to execute Nmap. The output is shown in Example 4.10. Example 4.10. Launching the scan nmap -T4 -p8 0 -PN - -max - r t t - t imeout 2 0 0 - - i n i t i a l - r t t - t imeout 1 5 0 \ mi n-hos t g roup 5 1 2 -n -oG pb-port 8 0 scan-% D . gnmap 2 1 6 . 1 6 3 . 1 2 8 . 0 / 2 0 -- ning : You spe c i f ied a h i g h l y aggr e s s ive - -m in-hostgr oup . rting Nmap ( http : / / nmap . org ) terest i n g por t s on 2 1 6 . 1 6 3 . 1 2 8 . 0 : T STATE SERVICE terest ing por t s on 2 1 6 . 1 6 3 . 1 2 8 . 1 : T STATE S E RV I C E /tcp fi ltered h t t p ere s t ing por t s o n 2 1 6 . 1 6 3 . 1 2 8 . 2 : T STATE SERVICE erest i ng port s on 2 1 6 . 1 6 3 . 1 2 8 . 3 : T STATE SERVICE erest i ng por t s on 2 1 6 . 1 6 3 . 1 4 3 . 2 5 5 : T STATE S E RV I CE http I P addr e s s e s ( 4 0 9 6 hos t s up ) s canned i n 1 9 2 . 9 7 s e conds ap scans all 4096 1Ps in about three minutes. The normal output shows a bunch of ports in the f i l t e r e d Most of those IPs are probably not active hosts-the port simply appears fi ltered because Nmap receives . 4.5. SOLUTION: Scan a Large Network for a Certain Open TCP Port 93 no response to its SYN probes. I obtain the list of web servers with a simple egrep on the output file, as shown i n Example 4. 1 1 . Example 4.11. Egrep for open ports # egrep ' [ A 0 - 9 ) 8 0 / ope n ' pb-por t 8 0 s can- * . gnmap Host : 2 1 6 . 1 6 3 . 1 4 0 . 2 0 ( ) Port s : 8 0 / open / t c p / /http / / / Host : 2 1 6 . 1 6 3 . 1 4 2 . 1 3 5 ( ) Port s : 8 0 / open / t cp/ /http/ / / After all that effort, only two accessible web servers are found out of 4096 IPs ! Sometimes that happens. The first one, 216. 163. 140.20 (no reverse DNS name) brings me to a Microsoft Outlook Web Access (webmail) server. That might excite me if I was trying to compromise their network, but it isn't gratifying now. The next server (reverse name mirrors.playboy.com) is much better. It offers those gigabytes of free images I was hoping for! In particular it offers Linux ISO images as well as substantial FreeBSD, CPAN, and Apache archives ! I download the latest Fedora Core ISOs at a respectable 6 Mbps. The abundance of bandwidth at Playboy is not surprising. Later I scan other Playboy netblocks, finding dozens more web servers, though some of their content is inappropriate for this book. While this is an unusual reason for port scanning, single port sweeps are common for many other purposes expressed previously. The techniques described here can be easily applied to any single-port TCP sweep. 4.5.4. See Also Version detection can be used to fi nd specific applications listening on a network. For example, you could seek a certain vulnerable version of OpenSSH rather than find all hosts with port 22 open. This is also useful for single-port UDP scans, as the techniques in this solution only work well for TCP. Instructions are provided in Section 7.8, "SOLUTION: Find All Servers Running an Insecure or Nonstandard Application Version" [ 1 66]. Chapter 6, Optimizing Nmap Performance [ 1 35] looks at scan speed optimization in much more depth. 94 4.5. SOLUTION: Scan a Large Network for a Certain Open TCP Port Chapter 5. Port Scanning Techniq ues and Algorit hms 5.1 . Introduction As a novice performing automotive repair, I can struggle for hours trying to fit my rudimentary tools (hammer, duct tape, wrench, etc.) to the task at hand. When I fail miserably and tow my jalopy to a real mechanic, he ilvariably fishes around in a huge tool chest until pulling out the perfect gizmo which makes the job seem effortless. The art of port scanning is similar. Experts understand the dozens of scan techniques and choose the appropriate one (or combination) for a given task. Inexperienced users and script kiddies, on the other lland, try to solve every problem with the default SYN scan. Since Nmap is free, the only barrier to port ICIJlning mastery is knowledge. That certainly beats the automotive world, where it may take great skill to determine that you need a strut spring compressor, then you still have to pay thousands of dollars for it. 1be previous chapter described port scanning with Nmap in general terms, including a brief summary of Nmap's supported scan types in Section 4.3 . 1 , "Selecting Scan Techniques" [82). This chapter describes each of those scan types in depth. T}'pical usage scenarios and instructions are given for each scan type, as me on-the-wire packet traces illustrating how they work. Then the ul t r a_scan algorithm (which most IC8ll methods use) is discussed, with an emphasis on aspects that can be tweaked to improve performance . . Most of the scan types are only available to privileged users. This is because they send and receive raw IP packets, (or even ethernet frames) which requires root access on Unix systems. Using an administrator account on Windows is recommended, though Nmap sometimes works for unprivileged users on that platform when WinPcap has already been loaded into the OS. Requiring root privileges was a serious limitation when Nmap was released in 1997, as many users only had access to shared shell accounts. Now, the world is different. Computers are cheaper, far more people have always-on direct Internet access, and desktop Unix systems (including Linux and Mac OS X) are prevalent. A Windows version of Nmap is now available, allowing it to run on even more desktops. For all these reasons, users rarely need to run Nmap from limited llhared shell accounts. This is fortunate, as the privileged options make Nmap far more powerful and flexible. When discussing how Nmap handles probe responses, many sections discuss ICMP error messages by their type and code numbers. The type and code are each eight-bit fields in ICMP headers that describe the message's purpose. Nmap port scanning techniques are concerned only with ICMP type 3, which are destination anreachable messages. Figure 5 . 1 shows the ICMP header layout of such a packet (it is encapsulated in the data section of an IP packet, as shown in Figure I, "1Pv4 header" [xxvii]). 5 . 1 . Introduction 95 Figure 5.1. ICMPv4 destination unreachable header layout o�� o 2 3 0 ,_...� ���_.__.._.___.__'--'l �h�- ��_,._�_,_,_._C ��(� _._-+-'--y���(� . T od C� pe 3 ) e 0 1 5) e c ksu m un ( t b O) u_ u_ s_ s __e_�_,..._...r-T"'__""""_,_ ed,,_,m a----�--....__.._____ �"' _...� . 2 1 3 -r _j_ D ata: Origi nal (received) I P header, p l u s at least the fi rst 8 data bytes There are sixteen codes representing different destination unreachable messages. They are all shown in Table 5. 1 , though Nmap only cares about codes 0-3, 9, J O, and 1 3, which are marked with an asterisk. Table 5.1. ICMP destination unreachable (type 3) code values Code Description O* Network unreachable l* Host unreachable 2* Protocol unreachable 3* Port unreachable 4 Fragmentation needed but don't-fragment bit set 5 Source route failed 6 Destination network unknown 7 Destination host unknown 8 Source host isolated (obsolete) 9* Destination network administratively prohibited 10* Destination host administratively prohibited 11 Network unreachable for type of service (TOS) 12 Host unreachable fo r TOS 13* Communication administratively prohibited by filtering 14 Host precedence violation 15 Precedence cutoff in effect 5.2. TCP SYN (Stealth) Scan (- s s) SYN scan is the default and most popular scan option for good reason. It can be performed quickly, scanning thousands of ports per second on a fast network not hampered by intrusive firewalls. SYN scan is relatively unobtrusive and stealthy, since it never completes TCP connections. It also works against any compliant TCP stack rather than depending on idiosyncrasies of specific platforms as Nmap's FIN/NULIJXmas, Maimon and idle scans do. It also allows clear, reliable differentiation between open, c l o s ed, and f i l t e red states. 96 5.2. TCP SYN (Stealth) Scan (-sS) SYN scan may be requested by passing the - s s option to Nmap. It requires raw-packet privileges, and is the default TCP scan when they are avai lable. So when runn i ng Nmap as root or Administrator, - s s is usually omitted. This default SYN scan behavior is shown in Example 5. 1 , which fi nds a port in each of the three major states. Example 5.1. A SYN scan showing three port states krad# nmap -p2 2 , 1 1 3 , 1 3 9 s canme . nmap . or g Start ing Nmap ( h t tp : / / nmap . or g ) Intere s t ing por t s on scanme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) : PORT STATE SERV I CE 22/tcp open ssh 1 1 3 / tcp closed auth 1 3 9 / tcp f i l t ered netbios - s s n Nmap done : 1 IP addr e s s ( 1 h o s t up ) s canned i n 1 . 3 5 s econds While SYN scan is pretty easy to use without any low-level TCP knowledge, u nderstanding the technique helps when interpreti ng unusual results. Fortunately for us, the fearsome black-hat cracker Ereet Hagiwara has taken a break from terrorizing Japanese Windows users 1 to illustrate the Example 5 . 1 SYN scan for us at the packet level. First, the behavior against open port 22 is shown in Figure 5.2. Figure 5.2. SYN scan of open port 22 As this example shows, Nmap starts by sending a TCP packet with the SYN flag set (see Figure 2, "TCP header" [xxviii] if you have forgotten what packet headers look like) to port 22. This is the first step in the TCP three-way handshake that any legitimate connection attempt takes. Since the target port is open, Scanme takes the second step by sending a response with the SYN and ACK flags back. In a normal connection, Ereet's machine (named krad) would complete the three-way handshake by sending an ACK packet acknowledging the SYN/ACK. Nmap does not need to do this, since the SYN/ACK response already told it that the port is open. If Nmap completed the connection, it would then have to worry about closing it. This usually involves another handshake, using FIN packets rather than SYN. So an ACK is a bad idea, yet something still has to be done. If the SYN/ACK is ignored completely, Scanme will assume it was dropped and keep re-sendi ng it. The proper response, since we don't want to m ake a ful l connection, is a RST packet IS shown in the diagram. This tells Scanme to forget about (reset) the attempted connection. Nmap could nd this RST packet easily enough, but it actually doesn't need to. The OS running on krad also receives SYN/ACK, which it doesn't expect because Nmap crafted the SYN probe itself. So the OS responds to lbe unexpected SYN/ACK with a RST packet. All RST packets described i n this chapter also have the ACK /tnp:l/www.microsoft.comljapanlsecuritylbulleti11s!MS04-003e.mspx 5.2. TCP SYN (Stealth) Scan (-sS) 97 bit set because they are always sent in response to (and acknowledge) a received packet. So that bit is not shown explicitly for RST packets. Because the three-way handshake is never completed, SYN scan is sometimes called half-open scanning. Figure 5.3 shows how Nmap determines that port 1 1 3 is closed. This is even simpler than the open case. The first step i s always the same-Nmap sends the SYN probe to Scanme. But instead of receiving a SYN/ACK back, a RST is returned. That settles it-the port is closed. No more communication regarding this port is necessary. Figure 5.3. SYN scan of closed port 1 13 Finally, Ereet shows us how a fil tered port appears to Nmap in Figure 5.4. The initial SYN is sent first, as usual, but Nmap sees no reply. The response could simply be slow. From previous responses (or timing defaults), Nmap knows how long to wait and eventually gives up on receiving one. A non-responsive port is usually filtered (blocked by a firewall device, or perhaps the host is down), but this one test is not conclusive. Perhaps the port is open but the probe or response were simply dropped. Networks can be flaky. So Nmap tries again by resending the SYN probe. After yet another timeout period, Nmap gives up and marks the port f i l t e red. In this case, only one retransmission was attempted. As described in Section 5. 1 3, "Scan Code and Algorithms" [ 1 28], Nmap keeps careful packet loss statistics and will attempt more retransmissions when scanning less reliable networks. Figure 5.4. SYN scan of filtered port 139 Nmap will also consider a port f i l t e r ed if it receives certain ICMP error messages back. Table 5.2 shows how Nmap assigns port states based on responses to a SYN probe. Table 5.2. How Nmap interprets responses to a SYN probe Probe Response Assigned State TCP SYN/ACK response open TCP RST response c l o s ed 98 5.2. TCP SYN (Stealth) Scan (-sS) Probe Response Assigned State No response received (even after retransmissions) f i l tered ICMP unreachable error (type 3, code 1, 2, 3, 9, 10, or 13) f i l t ered While the pretty ill ustrations in this section are useful when you have them, Nmap reports exactly what it is doing at the packet level when you specify the --pack et-trace option in addition to any other desired command-line flags. This is a great way for newbies to understand Nmap's behavior when Ereet is not around to help. Even advanced users fi nd it handy when Nmap produces results they don't expect. You may want to increase the debug level with -d (or even -d5) as well. Then scan the minimum number of ports and hosts necessary for your purpose or you could end up with literally millions of output lines. Example 5.2 repeats Ereet's three-port SYN scan with packet tracing enabled (output edited for brevity). Read the command-line, then test yourself by figuring out what packets will be sent before reading on. Then once you read the trace up to "The SYN Stealth Scan took l .25s", you should know from the RCVD lines what the port state table will look like before continuing on to read it. Example 5.2. Using --packet-trace to understand a SYN scan krad# nmap -d --packet - t race -p2 2 , 1 1 3 , 1 3 9 s canme . nmap . or g Starting Nmap ( http : / / nmap . or g ) SENT ( 0 . 0 1 3 0 s ) I CMP krad > s canme echo r eque s t ( t ype= 8 / code=0 ) t t l = 5 2 i d= l 8 2 9 SENT ( 0 . 0 1 6 0 s ) TCP krad : 6 3 5 4 1 > s c a nme : 8 0 A iplen= 4 0 seq=9 1 9 1 1 0 7 0 ack=9 9 8 5 0 9 1 0 RCVD ( 0 . 0 2 8 0 s ) I CMP scanme > krad echo reply ( t ype = O / code=O ) i p l en= 2 8 We got a ping packet back from scanme : i d = 4 8 8 2 1 seq = 7 1 4 check s um = 1 6 0 0 0 mas sping done : num_host s : 1 num_respon s e s : 1 Initiat ing SYN Stea l t h Scan a ga i n s t s canme . nmap . or g ( sc a nme ) ( 3 p o r t s ] at 0 0 : 53 SENT ( 0 . 1 3 4 0 s ) TCP krad : 6 3 5 1 7 > s canme : l l 3 S iplen= 4 0 seq= 1 0 4 3 8 6 3 5 SENT ( 0 . 1 3 7 0 s ) TCP krad : 6 3 5 1 7 > s canme : 2 2 S i p l en= 4 0 seq= 1 0 4 3 8 6 3 5 SENT ( 0 . 1 4 0 0 s ) TCP krad : 6 3 5 1 7 > s canme ! l 3 9 S i p l e n = 4 0 seq= l 0 4 3 8 6 3 5 RCVD ( 0 . 1 4 6 0 s ) TCP s canme : 1 1 3 > k r ad : 6 3 5 1 7 RA iplen= 4 0 seq=O ack=1 0 4 3 8 6 3 6 RCVD ( 0 . 1 5 1 0 s ) TCP s c a nme : 2 2 > krad : 6 3 5 1 7 SA iplen= 4 4 seq=7 5 8 9 7 1 0 8 a c k = l 0 4 3 8 6 3 6 SENT ( 1 . 2 5 5 0 s ) TCP krad : 6 3 5 1 8 > scanme : 1 3 9 S iplen= 4 0 seq= l 0 3 7 3 0 9 8 win=3 0 7 2 The SYN Stealth Scan took l . 2 5 s t o scan 3 t ot a l por t s . Interesting port s on scanme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) : PORT STATE SERV I CE ssh 2 2 /tcp open auth 113/tcp closed 139/tcp f i ltered netbio s - s s n Nmap done : 1 I P add r e s s ( 1 h o s t up ) s canned in 1 . 4 0 seconds SYN scan has long been called the stealth scan because it is subtler than TCP connect scan (discussed next), which was the most common scan type before Nmap was released. Despite that moniker, don't count on a default SYN scan slipping undetected through sensitive networks. Widely deployed intrusion detection systems and even personal firewalls are quite capable of detecting default SYN scans. More effective techniques for stealthy scanning are demonstrated in Chapter 1 0, Detecting and Subverting Firewalls and Intrusion Detection Systems [257]. 5.2. TCP SYN (Stealth) Scan (-sS) 99 5.3. TCP Con nect Scan (-sT) TCP connect scan is the default TCP scan type when SYN scan is not an option. This is the case when a user does not have raw packet privileges or is scanning 1Pv6 networks. Instead of writing raw packets as most other scan types do, Nmap asks the underlying operating system to establish a connection with the target machine and port by issuing the connect system call. This is the same high-level system call that web browsers, P2P clients, and most other network-enabled applications use to establish a connection. It is part of a programming interface known as the Berkeley Sockets APL Rather than read raw packet responses off the wire, Nmap uses this API to obtain status information on each connection attempt. This and the FrP bounce scan (Section 5 . 1 2, "TCP FTP Bounce Scan (-b)" [ 1 27]) are the. only scan types available to unprivileged users. When SYN scan is available, it is usually a better choice. Nmap has less control over the high level connect call than with raw packets, making it less efficient. The system call completes connections to open target ports rather than performing the half-open reset that SYN scan does. Not only does this take longer and require more packets to obtain the same information, but target machines are more likely to log the connection. A decent IDS will catch either, but most machines have no such alarm system. Many services on your average Unix system will add a note to syslog, and sometimes a cryptic error message, when Nmap connects and then closes the connection without sending data. Truly pathetic services crash when this happens, though that is uncommon. An administrator who sees a bunch of connection attempts in her logs from a single system should know that she has been connect scanned. Figure 5.5 shows a connect scan in action against open port 22 of scanme.nmap.org. Recall that this only required three packets in Figure 5.2, "SYN scan of open port 22" [97]. The exact behavior against an open port depends on the platform Nmap runs on and the service listening at the other end, but this six packet example is typical. Figure S.S. Connect scan of open port 22 (nmap -sT -p22 scanme.nmap.org) The first two steps (SYN and SYN/ACK) are exactly the same as with a SYN scan. Then, instead of aborting the half-open connection with a RST packet, krad acknowledges the SYN/ACK with its own ACK packet, completing the connection. In this case, Scanme even had time to send its SSH banner string ( S S H - 1 . 9 9 -0pe n S S H_3 . l p l \ n ) through the now-open connection. As soon as Nmap hears from its host OS that the connection was successful, it terminates the connection. TCP connections usually end with another handshake involving the FIN flag, but Nmap asks the host OS to terminate the connection immediately with a RST packet. While this connect scan example took twice as many packets as a SYN scan, the bandwidth di fferences are rarely so substantial. The vast majority of ports in a large scan will be c l o s ed or f i 1 t ered. The packet 100 5.3. TCP Connect Scan (-sT) ttaces for those are the same as described for SYN scan in Figure 5.3, "SYN scan of closed port 1 1 3" [98] and Figure 5.4, "SYN scan of filtered port 1 39" [98]. Only open ports generate more network traffic. The output of a connect scan doesn't differ significantly from a SYN scan. Example 5.3 shows a connect acan of Scanme. The - s T option could have been omitted since Nmap is being run from a non-privileged 1eeount so connect scan is the default type. Example 5.3. Connect scan example - s T scanme . nmap . or g rt ing Nmap ( http : / / nmap . or g ) eresting por t s on scanme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) : shown : 9 9 4 f i l t ered por t s T STATE SERV I CE open ssh closed smtp open doma in closed gopher http au th p done : 1 IP addre s s ( 1 host up ) scanned in 4 . 7 4 seconds 5.4. U D P Scan (- su) While most popular services on the Internet run over the TCP protocol, UDP services are widely deployed . DNS, SNMP, and DHCP (registered ports 53, 161/ 162, and 67/68) are three of the most common. Because UDP scanning is generally slower and more difficult than TCP, some security auditors ignore these ports. This is a mistake, as exploitable UDP services are quite common and attackers certainly don't ignore the whole protocol. Fortunately, Nmap can help inventory UDP ports. UDP scan is activated with the - s U option. It can be combined with a TCP scan type such as SYN scan (-sS) to check both protocols during the same run. UDP scan works by sending an empty (no data) UDP header to every targeted port. Based on the response, thereof, the port is assigned to one of four states, as shown in Table 5.3. or lack Table 5.3. How Nmap interprets responses to a UDP probe Assigned State Any UDP response from target port (unusual) open No response received (even after retransmissions) open I f i l tered CMP port unreachable error (type 3, code 3) c l o sed Other ICMP unreachable errors (type 3, code I , 2, 9, 10, or 1 3) f i l t ered most curious element of this table may be the open I f i 1 tered state. It is a symptom of the biggest Henges with UDP scanning: open ports rarely respond to these probes. The target TCP/IP stack simply 5.4. UDP Scan (-sU) 101 passes the (empty) packet up to the listening application, which usually discards it immediately as invalid. If ports in all other states would respond, then open ports could all be deduced by elimination. Unfortunately, firewalls and filtering devices are also known to drop packets without responding. So when Nmap receives no response after several attempts, it cannot determine whether the port is open or f i 1 te red. When Nmap was released, filtering devices were rare enough that Nmap could (and did) simply assume that the port was open. The Internet is better guarded now, so Nmap changed in 2004 (version 3.70) to report non-responsive UDP ports as open I f i l tered instead. We can see that in Example 5.4, which shows Ereet scanning a Linux box named Felix. Example 5.4. UDP scan example krad# nmap - s U -v f e l i x S t a r t i n g Nmap ( http : / / nmap . or g I nt e r e s t i n g por t s on f e l i x . nmap . or g ( 1 9 2 . 1 6 8 . 0 . 4 2 ) : ( The 9 9 7 port s s canned but not shown be l ow are in s t a t e : c losed ) PORT STATE SERV I CE 5 3 / udp open l f i l t ered doma in 6 7 / udp open l f i l t ered dhcpserver 1 1 1 / udp open l f i l tered rpcbind MAC Addre s s : 0 0 : 0 2 : E3 : 1 4 : 1 1 : 0 2 ( Li te -on Commu n i c a t ions ) Nmap done : 1 I P addr e s s ( 1 host up ) s canned in 9 9 9 . 2 5 seconds This scan of Felix demonstrates the open I f i 1 t e r ed ambiguity issue as well as another problem: UDP scanning can be slow. Scanning a thousand ports took almost 17 minutes in this case due to ICMP response rate limiting performed by Felix and most other Linux systems. Nmap provides ways to work around both problems, as described by the following two sections. 5.4.1 . Disambiguating Open from Fi ltered U D P Ports In the case of the Felix scan, all but the three open I f i l t ered ports were c l o s ed. So the scan was still successful in narrowing down potentially open ports to a handful. That is not always the case. Example 5.5 shows a UDP scan against the heavily filtered site Scanme. Example 5.5. UDP scan example krad# nmap - s u - T 4 s canme . nmap . or g Start ing Nmap ( http : / / nmap . or g ) Al l 1 0 0 0 s canned por t s on scanme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) are open l f i l tered Nmap done : 1 IP addre s s ( 1 h o s t up ) s canned i n 5 . 5 0 seconds In this case, the scan didn't narrow down the open ports at all. All 1000 are open I f i l tered. A new strategy is called for. Table 5 .3, "How Nmap interprets responses to a UDP probe" [ 1 0 1 ) shows that the open I f i l tered state occurs when Nmap fails to receive any responses from its UDP probes to a particular port. Yet it also shows that, on rare occasions, the UDP service listening on a port will respond in kind, proving that the port is open. 102 5.4. UDP Scan (-sU) The reason these services don't respond often is that the empty packets Nmap sends are considered invalid. Unfortunately, UDP services generally define their own packet structure rather than adhering to some common general format that Nmap could always send. An SNMP packet looks completely different than a SunRPC, DHCP, or DNS request packet. To send the proper packet for every popular UDP service, Nmap would need a large database defining their probe formats. Fortunately, Nmap has that in the form of nmap - s e r v i c e -pr obe s , which is part of the service and version detection subsystem described in Chapter 7, Service and Application Version Detection [ 1 45]. When version scanning is enabled with -sv (or -A), it will send UDP probes to every ope n I f i l t e r ed pon (as well as known open ones). If any of the probes elicit a response from an open I f i l t ered port, the state is changed to ope n. The results of adding - sV to the Fel ix scan are shown in Example 5.6. Example 5.6. Improving Felix's UDP scan results with version detection �ad# nmap -sUV -F f e l i x . nmap . or g http : / / nmap . or g ) teresting por t s on f e l i x . nmap . or g ( 1 9 2 . 1 6 8 . 0 . 4 2 ) : t shown : 9 9 7 c lo s ed port s RT STATE SERV I C E VERS I ON open doma in I SC B I ND 9 . 2 . 1 open l f i l tered dhcpserver open rpcbind 2 ( rpc # 1 0 0 0 0 0 ) Address : 0 0 : 0 2 : E 3 : 1 4 : 1 1 : 0 2 ( Li te-on Communicat ions ) IP add r e s s ( 1 host up ) s canned i n 1 0 3 7 . 5 7 seconds This new scan shows that port 1 1 1 and 53 are definitely open. The system isn't perfect though-port 67 is llill open I f i l te red. In this particular case, the port is open but Nmap does not have a working version probe for DHCP. Another tough service is SNMP, which usually only responds when the correct community Siring is given. Many devices are configured with the community string pub l i c, but not all are. While these results aren't perfect, learning the true state of two out of three tested ports is still helpful. After the success in disambiguating Felix results, Ereet turns his attention back to Scanme, which listed all ports as open I f i l tered last time. He tries again with version detection, as shown in Example 5.7. 5.4. UDP Scan (-sU) 103 Example 5.7. Improving Scanme's UDP scan results with version detection krad# nmap - s UV -T4 s canme . nmap . or g S t a r t i n g Nmap ( http : / / nmap . or g ) I nt e r e s t i n g por t s on scanme . nmap . org ( 6 4 . 1 3 . 1 3 4 . 5 2 ) : Not shown : 9 9 9 open l f i l tered por t s PORT STATE SERV I CE VERS I ON 5 3 / udp open doma i n I SC B I ND 9 . 3 . 4 Nmap done : 1 I P addr e s s ( 1 host up ) s canned in 3 6 9 1 . 8 9 s e conds This result took an hour, versus five seconds for the previous Scanme scari, but these results are actually useful. Ereet's smile widens and eyes sparkle at this evidence of an open ISC BIND nameserver on a machine he wants to compromise. That software has a Jong history of security holes, so perhaps he can find a flaw in this recent version . While Ereet will focus his UDP attacks on port 53 since it is confirmed open, he does not forget about the other ports. Those I 007 are listed as open I f i l t er ed. As we witnessed with the dhcpserver port on Felix, certain open UDP services can hide even from Nmap version detection. He has also only scanned the default ports so far, there are 64529 others that could possibly be open. For the record, 53 is the only open UDP port on Scanme. While this version detection technique is the only way for Nmap to automatically disambiguate ope n I f i l t ered ports, there are a couple tricks that can be tried manually. Sometimes a specialized traceroute can help. You could do a traceroute against a known-open TCP or UDP port with a tool such as 2 hping2 . Then try the same against the questionable UDP port. Differences in hop counts can differentiate open from filtered ports. Ereel attempts this against Scanme in Example 5.8. The first hping2 command does a UDP traceroute against known-open port 53. The -t 8 option tells hping2 to start at hop eight and is only used here to save space. The second command does the same thing against presumed-closed port 54. 2 htlp:llwww.hping.org 104 5.4. UDP Scan (-sU) Example 5.8. Attempting to disambiguate UDP ports with TTL discrepancies ad# hping2 --udp - - t r a ceroute -t 8 -p 5 3 scanme . nmap . org ING scanme . nmap . or g ( pppO ) : udp mode set , 28 heade r s + 0 data bytes p = 8 TTL 0 dur ing trans i t from 2 0 6 . 2 4 . 2 1 1 . 7 7 ( dcr 2 . SanFranc i s c o s fo . s avvi s . net ) •9 TTL 0 during t r a n s i t from 2 0 8 . 1 7 2 . 1 4 7 . 9 4 ( bpr 2 . Pa l oA l t oPa i x . s avvi s . ne t ) •10 TTL 0 dur ing t r a n s i t from 2 0 6 . 2 4 . 2 4 0 . 1 9 4 ( meer . Pa loAltoPa i x . s avvi s . net ) •11 TTL 0 dur ing tran s i t from 2 0 5 . 2 1 7 . 1 5 2 . 2 1 ( vl a n 2 1 . s v . meer . ne t ) - scanme . nmap . org hping s t at i s t i c - - packets transmi tted, 4 packe t s rece i ved , 6 7 % packet l o s s und-trip min/ avg /max = 1 3 . 4 / 1 3 . 8 / 1 4 . l m s ad# hping2 --udp - - t raceroute -t 8 - p 5 4 scanme . nmap . org ING scanme . nmap . org ( pppO ) : udp mode s e t , 28 heade r s + 0 da t a byt e s p=S T T L 0 during t r a n s i t from 2 0 6 . 2 4 . 2 1 1 . 7 7 ( dc r 2 . SanFran c i s c o s f o . s avvi s . ne t ) p=9 TTL 0 dur ing t r an s i t from 2 0 8 . 1 7 2 . 1 4 7 . 9 4 ( bpr 2 . P a l oAl toPa i x . s avvi s . ne t ) p=l O TTL 0 dur ing t r a n s i t from 2 0 6 . 2 4 . 2 4 0 . 1 9 4 ( meer . Pa l oAltoPa i x . s avvi s . ne t ) p=l l TTL 0 dur ing t r an s i t from 2 0 5 . 2 1 7 . 1 5 2 . 2 1 ( vl a n 2 1 . sv . meer . ne t ) - scanme . nmap . or g hping s t a t i s t i c - - packets tran smi tted, 4 pack e t s rece ived , und-t r i p min/ avg/max = 1 2 . 5 / 1 3 . 6 / 1 4 . 7 ms 6 7 % packet l o s s In this example, Ereet was only able to reach hop eleven of both the open and closed ports. So these results can't be used to distinguish port states against this host. It was worth a try, and does work in ·a significant number of cases. It is more likely to work in situations where the screening firewal l is at least a hop or two before the target host. Scanme, on the other hand, is running its own Linux iptables host-based firewall. So there is no difference in hop count between filtered and open ports. Another technique is to try application-specific tools against common ports. For example, a brute force SNMP community string cracker could be tried against port 161. As Nmap's version detection probe database grows, the need to augment its results with external specialized tools is reduced. They will sti ll be useful for special cases, such as SNMP devices with a custom community string. 5.4.2. Speed ing U p U D P Scans The other big challenge with UDP scanning is doing it quickly. Open and filtered ports rarely send any response, leaving Nmap to time out and then conduct retransmissions just in case the probe or response were lost. Closed ports are often an even bigger problem. They usually send back an ICMP port unreachable error. But unlike the RST packets sent by closed TCP ports in response to a SYN or connect scan, many hosts rate limit ICMP port unreachable messages by default. Linux and Solaris are particularly strict about this. For example, the Linux 2.4.20 kernel on Felix limits destination unreachable messages to one per second (in net / ipv4 I i cmp . c). This explains why the scan in Example 5.4, "UDP scan example" [ 1 02] is so slow. Nmap detects rate limiting and slows down accordingly to avoid flooding the network with useless packets that the target machine will drop. Unfortunately, a Linux-style limit of one packet per second makes a 65,536-port scan take more than 1 8 hours. Here are some suggestions for improving UDP scan performance. Also read Chapter 6, Optimizing Nmap Performance [ 1 35) for more detailed discussion and general advice. 5.4. UDP Scan (-sU) 1 05 Increase host parallelism If Nmap receives just one port unreachable error from a single target host per second, it could receive 1 00/second j ust by scanning 100 such hosts at once. Implement this by passing a large value (such as 1 00) to - -m i n - h o s t gr oup. Scan popular ports first Very few UDP port numbers are commonly used. A scan of the most common 100 UDP ports (using the -F option) will finish quickly. You can then investigate those results while you launch a multi-day 65K-port sweep of the network in the background. Add - - ve r s i on - i nt e n s i t y 0 to version detection scans As mentioned in the previous section, version detection (- sv) is often needed to differentiate open from filtered UDP ports. Version detection is relatively slow since it involves sending a large number of application protocol-specific probes to every open or open I f i l t e red port found on the target machines. Specifying --ve r s i on - i n t e n s i t y 0 directs Nmap to try only the probes most likely to be effective against a given port number. It does this by using data from the nmap-service-probes fi le. The performance impact of this option is substantial, as will be demonstrated later in this section. Scan from behind the firewall As with TCP, packet filters can slow down scans dramatically. Many modern firewalls make setting packet rate limits easy. If you can bypass that problem by launching the scan from behind the firewall rather than across it, do so. Use - - ho s t - t imeout to skip slow hosts ICMP-rate-limited hosts can take orders of magnitude more time to scan than those that respond to every probe with a quick destination unreachable packet. Specifying a maximum scan time (such as 900000 for 15 minutes) causes Nmap to give up on individual hosts if it hasn't completed scanning them in that much time. This allows you to scan all of the responsive hosts quickly. You can then work on the slow hosts in the background. Use -v and chill out With verbosity (-v) enabled, Nmap provides estimated time for scan completion of each host. There is no need to watch it closely. Get some sleep, head to your favorite pub, read a book, finish other work, or otherwise amuse yourself while Nmap tirelessly scans on your behalf. A perfect example of the need to optimize UDP scans is Example 5.7, "Improving Scanme's UDP scan results . with version detection" [ 1 04). The scan obtained the desired data, but it took more than an hour to scan this one host! In Example 5.9, I run that scan again. This time I add the -F --ver s i o n - i n t e n s i t y O options and the hour long scan is reduced to 1 3 seconds ! Yet the same key information (an ISC Bind daemon running on port 53) is detected. 106 5.4. UDP Scan (-sU) Example 5.9. Optimizing UDP Scan Time adt nmap -sUV - T 4 - F --ve r s ion-inten s i t y 0 scanme . nmap . or g rting Nmap ( http : / / nmap . or g ) teresting por t s on scanme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) : shown : 99 open l f i l tered port s T STATE S ERV I CE VERS I ON udp open doma in ISC B IND 9 . 3 . 4 IP addr e s s ( 1 host up ) s canned in 1 2 . 9 2 s e conds 5.5. TCP FI N , N U LL, and Xmas Scans (-sF, - sN, -sx) These three scan types (even more are possible with the - - s ca n f l a g s option described in the next section) Clploita subtle loophole in the TCP RFC to differentiate between ope n and c l o sed ports. Page 65 of RFC 793 says that "if the [destination] port state i s CLOSED .... an incoming segment not containing a RST causes a RST to be sent in response." Then the next page discusses packets sent to open ports without the SYN, RST, or ACK bits set, stating that: "you are unlikely to get here, but if you do, drop the segment, and return." When scanning systems compliant with this RFC text, any packet not containing SYN, RST, or ACK bits will result in a returned RST if the port is closed and no response at all if the port is open. As long as none of those three bits are included, any combination of the other three (FIN, PSH, and URG) are OK. Nmap Clploits this with three scan types: Null scan (- sN) Does not set any bits (TCP flag header is 0) FIN scan (-sF) Sets just the TCP FIN bit. Xmas scan (-sX) Sets the FIN, PSH, and URG flags, lighting the packet up like a Christmas tree. These three scan types are exactly the same in behavior except for the TCP flags set in probe packets. Responses are treated as shown in Table 5.4. Table 5.4. How Nmap interprets responses to a NULL, FIN, or Xmas scan probe Assigned State open I f i l t e red c l o sed P unreachable error (type 3, code I , 2, 3, 9, IO, or 1 3) f i l tered key advantage to these scan types is that they can sneak through certain non-stateful firewalls and packet ltering routers. Such firewalls try to prevent incoming TCP connections (while allowing outbound ones) 5.5. TCP FIN, NULL, and Xmas Scans (-sF, -sN, -sX) 107 · by blocking any TCP packets with the SYN bit set and ACK cleared. This configuration is common enough that the Linux iptables firewall command offers a special - - s yn option to implement it. The NULL, FIN, and Xmas scans clear the SYN bit and thus Hy right through those rules. Another advantage is that these scan types are a little more stealthy than even a SYN scan. Don't count on this though-most modern IDS products can be configured to detect them. The big downside is that not all systems follow RFC 793 to the letter. A number of systems send RST responses to the probes regardless of whether the port is open or not. This causes all of the ports to be labeled c l o sed. Major operating systems that do this are Microsoft Windows, many Cisco devices, and IBM OS/400. This scan does work against most Unix-based systems though. Since Nmap OS detection tests for this quirk, you can learn whether the scan works against a particular type of system by examining the nmap - o s -db file. Test T2 sends a NULL packet to an open port. So if you see a line like T 2 ( R=N ) , that system seems to support the RFC and one of these scans should work against it. If the T2 line is longer, the system violated the RFC by sending a response and these scans won't work. Chapter 8, Remote OS Detection [ 1 7 1 ] explains OS fingerprinting in further detai l. Another downside of these scans is that they can't distinguish open ports from certain filtered ones. If the packet filter sends an ICMP destination prohibited error, Nmap knows that a port is filtered. But most filters simply drop banned probes without any response, making the ports appear open. Since Nmap cannot be sure which is the case, it marks non-responsive ports as open I f i l t e red. Adding version detection (- sV) can disambiguate as it does with UDP scans, but that defeats much of the stealthy nature of this scan. If you are willing and able to connect to the ports anyway, you might as well use a SYN scan. Using these scan methods is simple. Just add the - sN, - s F , or - s x options to specify the scan type. Example 5.10 shows two examples. The first one, a FIN scan against Para, identifies all five open ports (as open I f i 1 t e r ed). The next execution, an Xmas scan against scanme.nmap.org doesn't work so well. It detects the closed port, but is unable to differentiate the 995 filtered ports from the four open ones, all 999 are listed as open I f i l tered. This demonstrates why Nmap offers so many scan methods. No single technique is preferable in all cases. Ereet will simply have to try another method to learn more about Scanme. 108 5.5. TCP FIN, NULL, and Xmas Scans (-sF, -sN, -sX) Example 5.10. Example FIN and Xmas scans krad# nmap - s F - T 4 p a r a Starting Nmap ( http : / /nmap . or g Interesting por t s on p a r a ( 1 9 2 . 1 6 8 . 1 0 . 1 9 1 ) : Not shown : 9 9 5 c l osed por t s PORT STATE SERV I CE 22/tcp open l f i l t ered s s h 53 /tcp open l f i l tered doma i n 111 /tcp open j f i l tered rpcbi n d 5 1 5 /tcp open ! f i l t ered p r i n t e r 6000 / t cp open l f i l t ered X l l MAC Address : 0 0 : 6 0 : 1 0 : 3 8 : 3 2 : 9 0 ( Lucent Technol ogie s ) Nmap done : 1 IP addr e s s ( 1 host up ) s c anned i n 4 . 6 4 s e conds krad# nmap -sx -T4 s c a nme . nmap . or g Starting Nmap ( http : / / nmap . or g ) Interesting por t s on s c a nme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) : Not shown : 9 9 9 open j f i ltered port s STATE SERV I CE PORT 113 /tcp closed a u t h Nmap done : 1 I P addr e s s ( 1 host up ) s canned i n 2 3 . 1 1 seconds Demonstra ting the full, firewall-bypassing power of these scans requires a rather lame target firewall configuration. Unfortunately, those are easy to find. Example 5. 1 1 shows a SYN scan of a SCO/Caldera machine named Docsrv. Example 5.1 1. SYN scan of Docsrv I nmap -ss -T4 doc srv . ca lder a . com Starting Nmap ( http : / / nmap . or g ) Jnteresting por t s on docsrv . ca ld e r a . com ( 2 1 6 . 2 5 0 . 1 2 8 . 2 4 7 ) : The 997 por t s s c anned but not shown be l ow a r e in s t a t e : f i l tered ) fORT STATE SERVICE http 80/tcp open l l 3 /tcp closed auth 507 /tcp open er s !fniap done : 1 IP addr e s s ( 1 host up ) s c anned i n 2 8 . 6 2 seconds This example looks OK. Only two ports are open and the rest (except for 1 1 3) are filtered. With a modern stateful firewall, a FIN scan should not produce any extra information. Yet Ereet tries it anyway, obtaining the output in Example 5 . 1 2. 5.5. TCP FIN, NULL, and Xmas Scans (-sF, -sN, -sX) 109 Example 5.12. FIN scan of Docsrv # nmap - s F - T 4 docsrv . caldera . com S t a r t i n g Nmap ( http : / / nmap . or g ) I nt e r e s t i n g por t s on doc s r v . ca ldera . com ( 2 1 6 . 2 5 0 . 1 2 8 . 2 4 7 ) : Not shown : 9 6 1 c losed por t s PORT STATE SERVICE 7 / tcp open J f i l tered echo 9 / tcp open J f i ltered d i s card 1 1 / t cp open J f i l tered s y s t at 1 3 / t cp open J f i l tered dayt ime 1 5 / tcp open J f i l tered net s t a t 1 9 /tcp open J f i l t ered chargen 2 1 / t cp open J f i l tered ftp 2 2 / t cp open J f i l tered s s h 2 3 / t cp open f i l t ered te lnet 2 5 / t cp open f i l t ered smtp 3 7 / t cp open f i l t ered t ime 7 9 / t cp open f i l t ered f i nger open f i l tered http 8 0 / t cp 1 1 0 /tcp open f i l t ered pop3 1 1 1 / tcp open f i l t ered rpcbind 1 3 5 / t cp open f i l t ered msrpc 1 4 3 / t cp open f i ltered imap 3 6 0 / t cp open f i ltered s c o i 2 od i a l og 3 8 9 / t cp open f i l t ered l dap 4 6 5 / t cp open f i l t ered smtps 5 0 7 / t cp open f i l tered c r s 5 1 2 / t cp open f i l t ered e xe c 5 1 3 / t cp open J f i l t ered login 5 1 4 / t cp open J f i l tered she l l 51 5 / tcp open J f i l tered printer 6 3 6 / t cp open f i l t ered l daps s l 7 1 2 /tcp open f i l t ered unknown 9 5 5 / t cp open f i l t ered unknown 9 9 3 / t cp open f i l t ered imaps 9 9 5 / t cp open f i l t ered pop 3 s 1 4 3 4 / tcp open f i l t ered m s - s q l -m 2 0 0 0 / tcp open f i l t ered c a l lbook 2 7 6 6 / tcp open f i l t ered l i s t en 3 0 0 0 / t cp open f i ltered PPP 3 3 0 6 / t cp open f i l t ered mysql 6 1 1 2 / tcp open f i ltered dt spc 3 2 7 7 0 / tcp open f i l t ered s omet ime s -rpc3 3 2 7 7 1 / tcp open f i l t ered s omet i me s -rpc5 3 2 7 7 2 / t c p open f i l t ered s omet ime s -rpc 7 Nmap done : 1 I P addr e s s ( 1 host up ) s canned in 7 . 6 4 seconds Wow ! That is a lot of apparently open ports. Most of them are probably open, because having just these 39 filtered and the other 961 closed (sending a RST packet) would be unusual. Yet it is still possible that some or all are filtered instead of open. FIN scan cannot determine for sure. We will revisit this case and learn more about Docsrv later in this chapter. 1 10 5.5. TCP FIN, NULL, and Xmas Scans (-sF, -sN, -sX) 5.6. Custom Scan Types with - - s canflags Truly advanced Nmap users need not limit themselves to the canned scanned types. The - - s c a n f l a g s option allows you to design your own scan by specifying arbitrary TCP flags. Let your creative juices flow, while evading intrusion detection systems whose vendors simply paged through the Nmap man page adding specific rules! The --scanf lags argument can be a numerical flag value such as 9 (PSH and FIN), but using symbolic names is easier. Just mash together any combination of URG, ACK, PSH, RST, S YN, and F I N. For example, --scanflags URGACKP SHRS T S YNF I N sets everything, though it's not very useful for scanning. The order these are specified in is irrelevant. In addition to specifying the desired flags, you can specify a TCP scan type (such as - s A or - s F). That base type tells Nmap how to interpret responses. For example, a SYN scan considers no-response indicative of a filtered port, while a FIN scan treats the same as open I f i l t e r ed. Nmap will behave the same way it does for the base scan type, except that it will use the TCP flags you specify instead. If you don't specify a base type, SYN scan is used. 5.6.1 . Custom SYN/FIN Scan One interesting custom scan type is SYN/FIN. Sometimes a firewall administrator or device manufacturer will attempt to block incoming connections with a rule such as "drop any incoming packets with only the · SYN flag set". They limit it to only the SYN flag because they don't want to block the SYN/ACK packets which are returned as the second step of an outgoing connection. The problem with this approach is that most end systems will accept initial SYN packets which contain other (non-ACK) flags as well. For example, the Nmap OS fi ngerprinting system sends a SYNIFIN/URG/PSH packet to an open port. More than half of the fingerprints in the database respond with a SYN/ACK. Thus they allow port scanning with this packet and generally allow making a full TCP connection too. Some systems have even been known to respond with SYN/ACK to a SYN/RST packet ! The TCP RFC is ambiguous IS to which flags are acceptable in an initial SYN packet, though SYN/RST certainly seems bogus. Example 5.13 shows Ereet conducting a successful SYN/FIN scan of Google. He is apparently getting bored with scanme.nmap.org. 5.6. Custom Scan Types with --scanflags l ll Example 5.13. A SYN/FIN scan of Google krad# nmap - s s - - s canflags SYNF I N - T 4 www . google . com S t a r t i n g Nmap ( http : / / nmap . or g ) War n i ng : Host name www . google . com r e s o l ve s to 4 I P s . U s ing 7 4 . 1 2 5 . 1 9 . 9 9 . I nt e r e s t ing por t s on c f - i n - f 9 9 . google . com ( 7 4 . 1 2 5 . 1 9 . 9 9 ) : No t sh own : 9 9 6 f i l t ered por t s PORT STATE SERV I CE 8 0 / t cp open http 1 1 3 / t cp c losed auth 1 7 9 / t cp c l osed bgp 4 4 3 / t cp open https Nmap done : 1 I P addre s s ( 1 host up ) s canned in 7 . 5 8 s econds Similar scan types, such as SYN/URG or SYN/PSH/URG/FIN will generally work as wel l. If you aren't getting through, don't forget the already mentioned SYN/RST option. 5.6.2. PSH Scan Section 5.5, "TCP FIN, NULL, and Xmas Scans (-sF, -sN, -sX)" [ 1 07] noted that RFC-compliant systems allow one to scan ports using any combination of the FIN, PSH, and URG flags. While there are eight possible permutations, Nmap only offers three canned modes (NULL, FIN, and Xmas). Show some personal flair by trying a PSH/URG or FIN/PS H scan i nstead. Results rarely differ from the three canned modes, but there is a small chance of evading scan detection systems. To perform such a scan, just specify your desired flags with - - s ca n f l ags and specify FIN scan (-sF) as the base type (choosi ng NULL or Xmas would make no difference). Example 5.14 demonstrates a PSH scan against a Linux machine on my local network. Example 5.14. A custom PSH scan krad# nmap - s F - - s c a n f l ags PSH para Start ing Nmap ( http : / / nmap . or g I nt e r e s t i n g por t s on para ( 1 9 2 . 1 6 8 . 1 0 . 1 9 1 ) : ( The 9 9 5 ports s canned but not shown be low are in s t ate : c lo s e d ) PORT STATE SERVI CE 2 2 / t cp open l f i l tered s sh 5 3 / t cp open l f i l tered doma i n 1 1 1 / t cp open l f i l t ered rpcbind 5 1 5 / tcp open l f i l tered printer 6 0 0 0 / t cp open l f i l tered X l l MAC Addr e s s : 0 0 : 6 0 : 1 0 : 3 8 : 3 2 : 9 0 ( Lucent Technologi e s ) Nmap done : 1 I P addr e s s ( 1 host u p ) scanned in 5 . 9 5 s econds Because these scans all work the same way, I could keep just one of - s F , - s N, and - s X options, letting users emulate the others with - - s c anf l ag s . There are no plans to do this because the shortcut options 1 12 5.6. Custom Scan Types with --scanflags ilreeasier to remember and use. You can still try the emulated approach to show off your Nmap skills. Execute llD8p -sF --scanftags FINPSHURG target rather than the more mundane nmap -sX target. Warning In my experience, needlessly complex Nmap command-lines don't impress girls. They usually respond with a condescending sneer, presumably recognizing that the command is redundant. .7. TCP ACK Scan (- sA) This scan is different than the others discussed so far in that it never determines open (or even pen I f i ltered) ports. It is used to map out firewall rulesets, determining whether they are stateful or bot and which ports are filtered. ACK scan is enabled by specifying the - s A option. Its probe packet has only the ACK flag set (unless you se --scanfl ags). When scanning unfiltered systems, ope n and c l o sed ports will both return a RST packet. Nmap then labels them as un f i l t e red, meaning that they are reachable by the ACK packet, but whether they are open or c l o s ed is undetermined. Ports that don't respond, or send certain ICMP error ssages back, are labeled f i l tered. Table 5.5 provides the full details. Table 5.5. How Nmap interprets responses to an ACK scan probe Assigned State u n f i l t e red o response received (even after retransmissions) ICMP unreachable error (type 3, code I , 2 , 3, 9, I O , or 13) f i ltered f i l t ered ACK scan usage is similar to most other scan types in that you simply add a single option flag, -sA in this ease. Example 5.15 shows an ACK scan against Scanme. 5.7. TCP ACK Scan (-sA) 1 13 Example 5.15. A typical ACK Scan k r a d # nmap -sA - T 4 s canme . nmap . or g S t a r t i n g Nmap ( http : / / nmap . or g ) I n t e r e s t i n g port s on s canme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) : Not s hown : 9 9 4 f i l tered por t s PORT S TATE SERVICE 2 2 / tcp u n f i l tered s s h 2 5 / t cp u n f i l t ered smtp 5 3 / t cp u n f i l tered doma in 7 0 / t cp u n f i l t ered gopher 8 0 / tcp u n f i l t ered http 1 1 3 / t cp u n f i l tered auth Nmap done : 1 I P addr e s s ( 1 host up ) s canned i n 4 . 0 1 seconds One of the most interesting uses of ACK scanning is to differentiate between stateful and stateless firewalls. See Section 10.3.2, "ACK Scan" [260] for how to do this and why you would want to. Sometimes a combination of scan types can be used to glean extra information from a system. As an example, start by reviewing the FIN scan of Docsrv in Example 5 . 1 2, "FIN scan of Docsrv" [ 1 1 0]. Nmap finds the closed ports in that case, but 39 of them are listed as open I f i l t ered because Nmap cannot determine between those two states with a FIN scan. Now look at the ACK scan of the same host in Example 5.16, "An ACK scan of Docsrv" [ 1 1 5] . Two of those 39 previously unidentified ports are shown to be f i l t ered. The other 37 (based on the default port line above the table) are in the state u n f i l t er ed. That means open or c l o sed. If one scan type identifies a port as open or f i l t ered and another identifies it as open or c l o s ed, logic dictates that it must be ope n . By combining both scan types, we have learned that 37 ports on Docsrv are open, two are f i l t e r ed, and 961 are c l o s ed. While logical deduction worked well here to determine port states, that technique can't always be counted on. It assumes that different scan types always return a consistent state for the same port, which is i naccurate. Firewalls and TCP stack properties can cause different scans against the same machine to differ markedly. Against Docsrv, we have seen that a SYN scan considers the SSH port (t cp / 2 2 ) f i l t e r ed, while an ACK scan considers it u n f i l t ered. When exploring boundary conditions and strangely configured networks, interpreting Nmap results is an art that benefits from experience and intuition. 1 14 5.7. TCP ACK Scan (-sA) Example 5.16. An ACK scan of Docsrv t nmap -sA - T 4 docsrv . ca l dera . com Starting Nmap ( http : / / nmap . or g ) Interes t ing por t s on doc s r v . ca lder a . com ( 2 1 6 . 2 5 0 . 1 2 8 . 2 4 7 ) : Not shown : 9 9 8 u n f i l t ered port s PORT SERV I CE STATE 1 3 5 /tcp f i ltered msrpc 1 4 3 4 / tcp f i l tered m s - s q l -m Nmap done : 1 I P addr e s s ( 1 host up ) scanned in 7 . 2 0 s econds 5.8. TCP Window Scan (-sw) Window scan is exactly the same as ACK scan except that it exploits an implementation detai l of certain systems to differentiate open ports from closed ones, rather than always printing u n f i l t ered when a RST is returned. It does this by examining the TCP Window value of the RST packets returned. On some systems, open ports use a positive window size (even for RST packets) while closed ones have a zero window. Window scan sends the same bare ACK probe as ACK scan, interpreting the results as shown in Table 5.6. Table 5.6. How Nmap interprets responses to a Window scan ACK probe Probe Response Assigned State TCP RST response with non-zero window field open TCP RST response with zero window field c l o s ed No response received (even after retransmissions) f i l t ered ICMP unreachable error (type 3, code 1, 2, 3, 9, 10, or 13) f i l t ered This scan relies on an implementation detail of a minority of systems out on the Internet, so you can't always trust it. Systems that don't support it will usually return all ports c l o s ed. Of course, it is possible that the machine really has no open ports. If most scanned ports are c l osed but a few common port numbers (such as 22, 25, and 53) are ope n, the system is most likely susceptible. Occasionally, systems will even show the exact opposite behavior. If your scan shows 997 open ports and three closed or filtered ports, then those three may very well be the truly open ones. While this scan is not suited for every situation, it can be quite useful on occasion. Recall Example 5 . 1 2, "FIN scan of Docsrv" [ 1 1 0], which shows many open I f i l t ered ports not found in a basic SYN scan. The problem is that we can't distinguish between open and filtered ports with that FIN scan. The previous aection showed that we could distinguish them by combining FIN and ACK scan results. In this case, a Window scan makes it even easier by not requiring the FIN scan results, as shown in Example 5 . 17. 5.8. TCP Window Scan (-sW) 1 15 Example 5.17. Window scan of docsrv.caldera.com # nmap - sW - T 4 docsrv . ca l dera . com S t a r t ing Nmap ( http : / / nmap . org ) I nt e r e s t i n g por t s on docsrv . ca l der a . com ( 2 1 6 . 2 5 0 . 1 2 8 . 2 4 7 ) : Not shown : 9 6 1 c losed por t s PORT STATE SERVI CE 7 / tcp open echo 9 / tcp open di s card 1 1 / t cp systat open 1 3 / tcp dayt ime open 1 5 / tcp open net stat 1 9 / t cp open chargen ftp open 2 1 / tcp ssh 2 2 / t cp open 2 3 / t cp open t e l net open smtp 2 5 / t cp t ime 3 7 / t cp open f inger open 7 9 / t cp 8 0 / t cp http open 1 1 0 / tcp open pop3 1 1 1 / t cp open rpcbind f i l tered msrpc 1 3 5 / t cp ( 1 4 open port s omi t ted for brev i t y ] 1 4 3 4 / t cp f i l t ered m s - sql-m c a l l book 2 0 0 0 / t cp open l i s t en 2 7 6 6 / tcp open 3 0 0 0 / t cp open PPP 3 3 0 6 / t cp open my sql 6 1 1 2 / tcp open dtspc somet ime s -rpc3 3 2 7 7 0 / t cp open 3 2 7 7 1 / t cp open s ome t ime s -rpc5 3 2 7 7 2 / t cp open s ome t ime s -rpc7 Nmap done : 1 I P addr e s s ( 1 host up ) s canned i n 7 . 3 0 seconds These results are exactly what Ereet wanted ! The same 39 interesting ports are shown as with the FIN scan, but this time it distinguishes between the two filtered ports (MS-SQL and MSRPC) and the 37 that are actually open. These are the same results Ereet obtained by combining FIN and ACK scan results together in the previous section. Verifying results for consistency is another good reason for trying multiple scan types against a target network. 5.9. TCP Maimon Scan (- sM) The Maimon scan is named after its discoverer, Uriel Maimon. He described the technique in Phrack Magazine issue #49 (November 1996). Nmap, which included this technique, was released two issues later. This technique is exactly the same as NULL, FIN, and Xmas scan, except that the probe is FIN/ACK. According to RFC 793 (TCP), a RST packet should be generated in response to such a probe whether the port is open or closed. H owever, Uriel noticed that many BSD-derived systems simply drop the packet if the port is open. Nmap takes advan tage of this to determ ine open ports, as shown in Table 5. 7. 1 16 5.9. TCP Maimon Scan (-sM) Table 5.7. How Nmap interprets responses to a Maimon scan probe Probe Response Assigned State No response received (even after retransmissions) open I f i l t ered TCP RST packet c l o sed ICMP unreachable error (type 3, code 1, 2, 3, 9, 10, or 1 3) f i l t ered The Nmap flag for a Maimon scan is - sM. While this option was quite useful in 1 996, modern systems rarely exhibit this bug. They send a RST back for all ports, making every port appear closed. This result is shown in Example 5.18. Example 5.18. A failed Maimon scan f nmap -sM - T 4 para Starting Nmap ( h t tp : / / nmap . or g A l l 1 0 0 0 scanned por t s on para ( 1 9 2 . 1 6 8 . 1 0 . 1 9 1 ) are : c l osed MAC Addre ss : 0 0 : 6 0 : 1 0 : 3 8 : 3 2 : 9 0 ( Lucent Technologie s ) Nmap done : 1 IP addr e s s ( 1 host up ) s canned in 4 . 1 9 s econds 5.1 0. TCP Id le Scan (-s I) In 1998, security researcher Antirez (who also wrote the hping2 tool frequently used in this book) posted to the Bugtraq mailing list an ingenious new port scanning technique. Idle scan, as it has become known, allows for completely blind port scanning. Attackers can actually scan a target without sending a single packet to the target from their own IP address ! Instead, a clever side-channel attack allows for the scan to be bounced off a dumb "zombie host". Intrusion detection system (IDS) reports will finger the innocent zombie as the attacker. Besides being extraordinarily stealthy, this scan type permits discovery of IP-based trust relationships betwee n machines. While idle scanning is more complex than any of the techniques discussed so far, you don't need to be a TCP/IP expert to understand it. It can be put together from these basic facts: • • • One way to determine whether a TCP port is open is to send a SYN (session establishment) packet to the port. The target machine will respond with a SYN/ACK (session request acknowledgment) packet if the port is open, and RST (reset) if the port is closed. This is the basis of the previously discussed SYN scan. A machine that receives an unsolicited SYN/ACK packet will respond with a RST. An unsolicited RST will be ignored. Every IP packet on the Internet has a fragment identification number (IP ID). Since many operating systems simply increment this number for each packet they send, probing for the IPID can tell an attacker how many packets have been sent since the last probe. By combining these traits, it is possible to scan a target network while forging your identity so that it looks like an innocent zombie machine did the scanning. 5.10. TCP Idle Scan (-sl) 1 17 5.1 0.1 . Idle Scan Step by Step Fundamentally, an idle scan consists of three steps that are repeated for each port: I. Probe the zombie's IP ID and record it. 2. Forge a SYN packet from the zombie and send it to the desired port on the target. Depending on the port state, the target's reaction may or may not cause the zombie's IP ID to be incremented. 3. Probe the zombie's IP ID again. The target port state is then determined by comparing this new IP ID with the one recorded i n step 1 . After this process, the zombie's IP I D should have increased by either one or two. A n increase of one indicates that the zombie hasn't sent out any packets, except for its reply to the attacker's probe. This lack of sent packets means that the port is not open (the target must have sent the zombie either a RST packet, which was ignored, or nothing at all). An i ncrease of two indicates that the zombie sent out a packet between the two probes. This extra packet usually means that the port is open (the target presumably sent the zombie a SYN/ACK packet in response to the forged SYN, which induced a RST packet from the zombie). Increases larger than two usually signify a bad zombie host. It might not have predictable IP ID numbers, or might be engaged in communication unrelated to the idle scan. Even though what happens with a closed port is slightly different from what happens with a filtered port, the attacker measures the same result in both cases, namely, an IP ID increase of 1 . Therefore it is not possible for the idle scan to distinguish between closed and filtered ports. When Nmap records an IP ID increase of 1 it marks the port c l o s e d I f i l tered. For those wanting more detail, the following three diagrams show exactly what happens in the three cases of an open, closed, and filtered port. The actors in each are: I the attacker, � the zombie, and 1 18 the target. 5. 10. TCP Idle Scan (-sI) Figure 5.6. Idle scan of an open port Step I : Probe the zombie's IP ID. Step 2: Forge a SYN packet from the zombie. '� ��; � The attacker sends a SYN/ACK to the zombie. The zombie, not expecting the SYN/ACK, sends back a RST, disclosing its IP ID. Step 3: Probe the zombie's IP ID again. '�ACK �� � The target sends a SYN/ACK in The zombie's IP ID has in response to the SYN that appears creased by 2 since step 1, so the port is open ! to come from the zombie. The zombie, not expecting it, sends back a RST, incrementing its IP ID in the process. Figure 5.7. Idle scan of a closed port Step I : Probe the zombie's IP ID. '�ACK �i6'>1 � The attacker sends a SYN/ACK to the zombie. The zombie, not expecting the SYN/ACK, sends back a RST, disclosing its IP ID. This step is always the same. Step 2: Forge a SYN packet from the zombie. 1---r � Step 3: Probe the zombie's IP ID again. '�ACK � �� The target sends a RST {the port The zombie's IP ID has increased is closed) in response to the SYN by only 1 since step 1 , so the port is not open. that appears to come from the zombie. The zombie ignores the unsolicited RST, leaving its IP ID unchanged. Figure 5.8. Idle scan of a filtered port Step I : Probe the zombie's IP ID. '�ACK �i6'>1 � Just as in the other two cases, the attacker sends a SYN/ACK to the zombie. The zombie discloses its I P ID. Step 2: Forge a SYN packet from the zombie. � The target, obstinately filtering its port, ignores the SYN that appears to come from the zom bie. The zombie, unaware that anything has happened, does not increment its IP ID. Step 3: Probe the zombie's IP ID again. '�ACK �� � The zombie's IP ID has i ncreased by only 1 since step 1 , so the port is not open. From the attacker's point of view this filtered port is indistinguishable from a closed port. Idle scan is the ultimate stealth scan. Nmap offers decoy scanning (-D) to help users shield their identity, but that (unlike idle scan) still requires an attacker to send some packets to the target from his real IP address in order to get scan results back. One upshot of idle scan is that intrusion detection systems will generally 5. 10. TCP Idle Scan (-sI) 1 19 send alerts claiming that the zombie machine has launched a scan against them. So it can be used to frame some other party for a scan. Keep this possibility in mind when reading alerts from your IDS. A unique advantage of idle scan is that it can be used to defeat certain packet filtering firewalls and routers. IP source address filtering is a common (though weak) security mechanism for limiting machines that may connect to a sensitive host or network. For example, a company database server might only allow connections from the public web server that accesses it. Or a home user might only allow SSH (interactive login) connections from his work machines. A more disturbing scenario occurs when some company bigwig demands that network administrators open a firewall hole so he can access internal network resources from his home IP address. This can happen when executives are unwilling or unable to use secure VPN alternatives. Idle scanning can sometimes be used to map out these trust relationships. The key factor is that idle scan results list open ports from the zombie host's perspective. A normal scan against the aforementioned database server might show no ports open, but performing an idle scan while using the web server's IP as the zombie could expose the trust relationship by showing the database-related service ports as open. Mapping out these trust relationships can be very useful to attackers for prioritizing targets. The web server discussed above may seem mundane to an attacker until she notices its special database access. A disadvantage to idle scanning is that it takes far longer than most other scan types. Despite the optimized algorithms described in Section 5.10.4, "Idle Scan Implementation Algorithms" [ 1 22), A 1 5-second SYN scan could take 15 minutes or more as an idle scan. Another issue is that you must be able to spoof packets as if they are coming from the zombie and have them reach the target machine. Many ISPs (particularly dialup and residential broadband providers) now implement egress filtering to prevent this sort of packet spoofing. Higher end providers (such as colocation and T l services) are much less likely to do this. If this fi ltering is in effect, Nmap will print a quick error message for every zombie you try. If changing ISPs is not an option, you might try using another IP on the same ISP network. Sometimes the filtering only blocks spoofing of IP addresses that are outside the range used by customers. Another challenge with idle scan is that you must find a working zombie host, as described in the next section. 5.1 0.2. Fi n d i ng a Worki ng Idle Scan Zombie Host The first step in executing an IP ID idle scan is to find an appropriate zombie. It needs to assign IP ID packets incrementally on a global (rather than per-host it communicates with) basis. It should be idle (hence the scan name), as extraneous traffic wi l l bump up its IP ID sequence, confusing the scan logic. The lower the latency between the attacker and the zombie, and between the zombie and the target, the faster the scan will proceed. When an idle scan is attempted, Nmap tests the proposed zombie and reports any problems with it. If one doesn't work, try another. Enough Internet hosts are vulnerable that zombie candidates aren't hard to find. Since the hosts need to be idle, choosing a well-known host such as www.yahoo.com or google.com will almost never work. A common approach is to simply execute a Nmap ping scan of some network. You could use Nmap's random IP selection mode ( - i R), but that is likely to result in far away zombies with substantial latency. Choosing a network near your source address, or near the target, produces better results. You can try an idle scan using each available host from the ping scan results until you find one that works. As usual, it is best to ask permission before using someone's machines for unexpected purposes such as idle scanning. 1 20 5. 10. TCP Idle Scan (-sI) We didn't just choose a printer icon to represent a zombie in our illustrations to be funny-simple network devices often make great zombies because they are commonly both underused (idle) and built with simple network stacks which are vulnerable to IP ID traffic detection. Performing a port scan and OS identification (-0) on the zombie candidate network rather than just a ping scan helps in selecting a good zombie. As long as verbose mode (-v) is enabled, OS detection will usually determine the IP ID sequence generation method and print a line such as "IP ID Sequence Generation: Incremental". If the type is given as I n creme n t a l or Broken l i t t l e -endian i n creme n t a l , the machine i s a good zombie candidate. That i s still n o guarantee that it will work, a s Solaris and some other systems create a new IP ID sequence for each host they communicate with. The host could also be too busy. OS detection and the open port list can also help in identifying systems that are likely to be idle. While identifying a suitable zombie takes some initial work, you can keep re-using the good ones. 5.1 0.3. Executing an Id le Scan Once a suitable zombie has been found, performing a scan is easy. Simply specify the zombie hostname to the - s I option and Nmap does the rest. Example 5.19 shows an example of Ereet scanning the Recording Industry Association of America by bouncing an idle scan off an Adobe machine named Kiosk. Example 5.19. An idle scan against the RIAA t nmap -PN -p- - s I k iosk . adobe . com www . r i a a . com Starting Nmap ( h t tp : / / nmap . or g ) Idlescan u s i ng zombie k io s k . adobe . com ( 1 9 2 . 1 5 0 . 1 3 . 1 1 1 : 8 0 ) ; C l a s s : I n cremen t a l Interes t ing por t s o n 2 0 8 . 2 2 5 . 9 0 . 1 2 0 : ( The 6 5522 por t s s canned but not shown be l ow are in s t a t e : c lo s e d ) Port State Service 21 /tcp ftp open open 25 /tcp smtp 80/tcp open http 111 /tcp open sunrpc loc-srv 135/ tcp open 443/ tcp open https 102 7 /tcp !IS open 1030 /tcp open iadl unknown open 2306/tcp pcanywheredat a open 56 3 1 /tcp 7 9 3 7 /tcp open unk nown unknown 7938/tcp open unknown 36 8 9 0 / tcp open Nmap done : 1 I P addr e s s ( 1 h o s t up ) s canned i n 2 5 9 4 . 4 7 s e conds From the scan above, we learn that the RIAA is not very security conscious (note the open PC Anywhere, portmapper, and Legato nsrexec ports). Since they apparently have no firewall, it is unlikely that they have an IDS. But if they do, it will show kiosk.adobe.com as the scan culprit. The -PN option prevents Nmap from sending an initial ping packet to the RIAA machine. That would have disclosed Ereet's true address. The scan took a long time because -p- was specified to scan all 65K ports. Don't try to use kiosk for your scans, as it has already been removed. 5 . 10. TCP Idle Scan (-sl) 121 By default, Nmap forges probes to the target from the source port 80 of the zombie. You can choose a different port by appending a colon and port number to the zombie name (e.g. - s I k i o s k . adobe . com : 11 3). The chosen port must not be filtered from the attacker or the target. A SYN scan of the zombie should show the port in the open or c l o s ed state. 5.1 0.4. Idle Scan Implementation Algorithms While Section 5 . 1 0. 1 , "Idle Scan Step by Step" [ 1 1 8) describes idle scan a t the fundamental level, the Nmap implementation is far more complex. Key differences are parallelism for quick execution and redundancy to reduce false positives. Parallelizing idle scan is trickier than with other scan techniques due to indirect method of deducing port states. If Nmap sends probes to many ports on the target and then checks the new IP ID value of the zombie, the number of IP ID increments will expose how many target ports are open, but not which ones. This isn't actually a major problem, as the vast majority of ports in a large scan will be c l o sed I f i l t ered. Since only open ports cause the IP ID value to increment, Nmap will see no intervening increments and can mark the whole group of ports as c l o sed I f i 1 t e red. Nmap can scan groups of up to 100 ports in para\\e\. U Nmap probes a group then finds that the zombie IP ID has increased times, there must be open ports among that group. Nmap then finds the open ports with a binary search. It splits the group into two and separately sends probes to each. If a subgroup shows zero open ports, that group's ports are all marked c l o sed I f i l t e r ed. If a subgroup shows one or more open ports, it is divided again and the process continues until those ports are identified. While this technique adds complexity, it can reduce scan times by an order of magnitude over scanning just one port at a time. Reliability is another major idle scanning concern. If the zombie host sends packets to any unrelated machines during the scan, its IP ID increments. This causes Nmap to think it has found an open port. Fortunately, parallel scanning helps here too. If Nmap scans J OO ports in a group and the IP ID increase signals two open ports, Nmap splits the group into two fifty-port subgroups. When Nmap does an IP ID scan on both subgroups, the total zombie IP ID increase better be two again ! Otherwise, Nmap will detect the inconsistency and rescan the groups. It also modifies group size and scan timing based on the detected reliability rate of the zombie. If Nmap detects too many inconsistent results, it will quit and ask the user to provide a better zombie. Sometimes a packet trace is the best way to understand complex algorithms and techniques such as these. Once again , the Nmap --packet - t race makes these trivial to produce when desired. The remainder of this section provides an annotated packet trace of an actual seven port idle scan. The IP addresses have been changed to At t acker, Z ombi e , and Target and some irrelevant aspects of the trace lines (such as TCP window size) have been removed for clarity. Att ac k e r # nmap - s I Zombie -PN -p2 0 - 2 5 , 1 1 0 -r --packet-trace -v Target S t a r t i n g Nmap ( http : / / nmap . or g ) -PN \s necessary for stea\th, otherwise p\ng packets wou\d 'oe sent to the target from Attacker's rea\ address. Version scanning would also expose the true address, and so - sV is not specified. The -r option (turns off port randomization) is only used to make this example easier to follow. Nmap firsts tests Zombie's IP ID sequence generation by sending six SYN/ACK packets to it and analyzing the responses. This helps Nmap immediately weed out bad zombies. It is also necessary because some systems (usually Microsoft Windows machines, though not all Windows boxes do this) increment the IP ID by 256 for each packet sent rather than by one. This happens on little-endian machines when they don't convert the 1 22 5.10. TCP Idle Scan (-sl) IP ID to network byte order (big-endian). Nmap uses these initial probes to detect and work around this problem. SENT ( 0 . 0 0 6 0 s ) SENT ( 0 . 0 9 0 0 s ) SENT ( 0 . 1 8 0 0 s ) RCVD ( 0 . 1 5 5 0 s ) SENT ( 0 . 2 7 0 0 s ) RCVD ( 0 . 2 3 8 0 s ) SENT ( 0 . 3 6 0 0 s ) RCVD ( 0 . 3 2 8 0 s ) SENT ( 0 . 4 5 1 0 s ) RCVD ( 0 . 4 1 9 0 s ) RCVD ( 0 . 5 0 9 0 s ) RCVD ( 0 . 5 9 9 0 s ) Idlescan using TCP Attacker : 5 1 8 2 4 > Zombie : 8 0 SA id=3 5 9 9 6 TCP Attacker : 5 1 8 2 5 > Zombi e : 8 0 SA id=2 5 9 1 4 TCP Attacker : 5 1 8 2 6 > Zombi e : 8 0 SA i d=3 9 5 9 1 TCP Zombie : 8 0 > Attacker : 5 1 8 2 4 R id= 1 5 6 6 9 TCP Attacker : 5 1 8 2 7 > Zombie : 8 0 SA id=4 3 6 0 4 TCP Zombi e : 8 0 > Attacker : 5 1 8 2 5 R id• l 5 6 7 0 TCP Attacker : 5 1 8 2 8 > Zombie : 8 0 SA i d= 3 4 1 8 6 TCP Zombi e : 8 0 > At tacker : 5 1 8 2 6 R id• l 5 6 7 1 TCP Attacker : 5 1 8 2 9 > Zombie : 8 0 SA id=2 7 9 4 9 TCP Zombie : 8 0 > Attacker : 5 1 8 2 7 R id•l5672 TCP Zombi e : 8 0 > At tacker : 5 1 82 8 R id• 1 5 6 7 3 TCP Zombie : 8 0 > Attacker : 5 1 8 2 9 R id• l 5 6 7 4 zombie Zombie ( Zombie : 8 0 ) ; C l a s s : I ncrement a l This test demonstrates that the zombie is working fine. Every IP ID was a n increase o f one over the previous one. So the system appears to be idle and vulnerable to IP ID traffic detection. These promising results are still subject to the next test, in which Nmap spoofs four packets to Zombie as if they are coming from Target. Then it probes the zombie to ensure that the IP ID increased. If it hasn't, then it is likely that either the attacker's ISP is blocking the spoofed packets or the zombie uses a separate IP ID sequence counter for each host it communicates with. Both are common occurrences, so Nmap always performs this test. The last-known Zombie IP ID was 1 5674, as shown above. ( 0 . 5990s ) ( 0 . 6510s ) (0. 7110s) ( 0 . 7710s) ( 1 . 0BOOs ) ( 1 . 2290s) TCP TCP TCP TCP TCP TCP Target : 5 1 8 2 3 > Zombi e : 8 0 SA i d= 1 3 9 0 Target : 5 1 8 2 3 > Zombi e : 8 0 SA i d= 2 4 0 2 5 Target : 5 1 8 2 3 > Zombi e : 8 0 SA i d= l 5 0 4 6 Target : 5 1 8 2 3 > Zombi e : 8 0 SA id=4 8 6 5 8 Attacker : 5 1 9 8 7 > Zombi e : 8 0 SA i d= 2 7 6 5 9 Zombie : 8 0 > Attacker : 5 1 9 8 7 R id• 1 5 6 7 9 The four spoofed packets coupled with the probe from Attacker caused the Zombie to increase its IP ID from 15674 to 15679. Perfect! Now the real scanning begins. Remember that 1 5679 is the latest Zombie IP ID. 'tiating Idle scan a ga i n s t Target ( 1 . 2 2 9 0 s ) TCP Zombie : 8 0 > Target : 2 0 S i d= l 3 2 0 0 ( l . 2 2 9 0 s ) TCP Zombie : 8 0 > Target : 2 1 S i d=3 7 3 7 ( l . 2 2 9 0 s ) TCP Zombie : 8 0 > Target : 2 2 S id=6 5 2 9 0 ( l . 2 2 9 0 s ) TCP Zombie : 8 0 > Target : 2 3 S i d= l 0 5 1 6 ( l . 46 1 0 s ) TCP Attacker : 5 2 0 5 0 > Zombie : 8 0 SA i d=3 3 2 0 2 ( l . 6 0 9 0 s ) TCP Zombie : B O > Attacker : 52 0 5 0 R id• 1 5 6 8 0 p probes ports 20-23. Then it probes Zombie and finds that the new IP ID is 1 5680, only one higher the previous value of 1 5679. There were no IP ID increments in between those two known packets, ing ports 20-23 are probably c l o s ed I f i l t e r ed. It is also possible that a SYN/ACK from a Target has simply not arrived yet. In that case, Zombie has not responded with a RST and thus its IP ID has incremented. To ensure accuracy, Nmap will try these ports again later. 5 . 1 0. TCP Idle Scan (-sl) 1 23 SENT ( l . 8 5 1 0 s ) TCP Attacker : 5 1 9 8 6 > Zombi e : 8 0 SA id=4 9 2 7 8 RCVD ( l . 9 9 9 0 s ) TCP Zombie : B O > At t a cker : 5 1 9 8 6 R id= l 5 6 8 1 Nmap probes again because four tenths of a second has gone by since the last probe it sent. The Zombie (if not truly idle) could have communicated with other hosts during this period, which would cause inaccuracies later if not detected here. Fortunately, that has not happened: the next IP ID is 1 5681 as expected. SENT SENT SENT SENT RCVD (2 . 0000s) ( 2 . 0000s ) ( 2 . 0000s) ( 2 . 2300s) ( 2 . 3800s) TCP TCP TCP TCP TCP Zombie : B O > Target : 2 4 s i d= 2 3 9 2 8 Zombi e : B O > Target : 2 5 S id=5 0 4 2 5 Zomb i e : B O > Target : l l O s id= l 4 2 0 7 Attacker : 5 2 0 2 6 > Zombi e : B O SA id= 2 6 9 4 1 Zombie : 8 0 > Attacker : 5 2 0 2 6 R id= l 5 6 8 4 Nmap probes ports 24, 25, and 1 1 0 then queries the Zombie IP ID. It has jumped from 1 5681 to 15684. I t skipped 1 5682 and 1 5683, meaning that two of those three ports are likely open. Nmap cannot tell which two are open, and i t could also be a false positive. So Nmap drills down deeper, dividing the scan into subgroups. SENT RCVD SENT SENT SENT RCVD ( 2 . 6210s) ( 2 . 7690s) (2 . 7690s ) ( 2 . 7690s) ( 3 . 0000s) (3 . 1480s) TCP TCP TCP TCP TCP TCP At tacker : 5 1 8 6 7 > Zombi e : B O SA id=l 8 8 6 9 Zombi e : 8 0 > Attacker : 5 1 8 6 7 R id=l 5 6 8 5 Zombie : B O > Target : 2 4 s id=3 0 0 2 3 Zomb i e : 8 0 > Target : 2 5 s i d= 4 7 2 5 3 At tacker : 5 1 9 7 9 > Zomb i e : 8 0 SA i d= l 2 0 7 7 Zombi e : B O > Attacker : 5 1 9 7 9 R id= l 5 6 8 7 The first subgroup is ports 24 and 25. The IP ID jumps from 1 5685 to 1 5687, meaning that one of these two ports is most likely open. Nmap tries the divide and conquer approach again, probing each port separately. SENT RCVD SENT SENT RCVD ( 3 . 39 1 0 s ) ( 3 . 5390s ) ( 3 . 53 9 0 s ) (3 . 7710s) (3 . 9190s) TCP TCP TCP TCP TCP Attacker : 5 1 8 2 6 > Zombi e : 8 0 SA id= 3 2 5 1 5 Zombi e : B O > Attacker : 5 1 8 2 6 R id= l 5 6 8 8 Zombi e : B O > Target : 2 4 s id=4 7 8 6 8 Attacker : 5 2 0 1 2 > Z ombi e : B O SA id= l 4 0 4 2 Zombie : B O > Attacker : 5 2 0 1 2 R id= l 5 6 8 9 A port 2 4 probe shows no jump in the IP ID. So that port is not open. From the results so far, Nmap has tentatively determined: • Ports 20-23 are c l o sed I f i l t ered • Two of the ports 24, 25, and 1 10 are open • One of the ports 24 and 25 are open • Port 24 is c l o s ed I f i l tered Stare at this puzzle long enough and you'll find only one solution: ports 25 and 1 10 are open while the other five are c l o s ed I f i l t e r ed. Using this logic, Nmap could cease scanning and print results now. It used to do so, but that produced too many false positive open ports when the Zombie wasn't truly idle. So Nmap continues scanning to verify its results: SENT ( 4 . 1 6 0 0 s ) TCP At t a cker : 5 1 8 5 8 > Zombi e : B O SA id=6 2 2 5 RCVD ( 4 . 3 0 8 0 s ) TCP Zombie : B O > At tacker : 5 1 8 5 8 R id= l 5 6 9 0 SENT ( 4 . 3 0 8 0 s ) TCP Zombie : B O > Targe t : 2 5 S id=3 5 7 1 3 1 24 5. 10. TCP Idle Scan (-sI) NT ( 4 . 5 4 1 0 s ) TCP Attacker : 5 1 8 5 6 > Zombi e : B O SA id=2 8 1 1 8 ( 4 . 6 89 0 s ) TCP Zombi e : B O > Attacker : 5 1 8 5 6 R id= 1 5 6 92 iscovered open port 2 5 / t cp on Target NT ( 4 . 6 9 0 0 s ) TCP Zombie : B O > Target : l l O S id=9 9 4 3 NT ( 4 . 9 2 1 0 s ) TCP Attacker : 5 1 8 3 6 > Zombie : B O SA id=6 2 2 5 4 D ( 5 . 06 9 0 s ) TCP Zombie : B O > Attacker : 5 1 8 3 6 R id= l 5 6 9 4 VD Probes o f ports 2 5 and 1 10 show that they are open, a s we deduced previously. ( 5 . 0690s ) ( 5 . 0690s) T ( 5 . 06 9 0 s ) ( 5 . 0690s) ( 5 . 3200s) ( 5 . 4690s ) ENT ( 5 . 7 9 1 0 s ) RCVD ( 5 . 9 3 9 0 s ) TCP TCP TCP TCP TCP TCP TCP TCP Zombie : B O > Target : 2 0 s id= 8 1 6 8 Zombie : B O > Target : 2 1 s i d= 3 6 7 1 7 Zombie : B O > Target : 2 2 s i d= 4 0 6 3 Zombie : B O > Target : 2 3 s id=5 4 7 7 1 At tacker : 5 1 9 6 2 > Zombi e : B O SA i d= 3 8 7 6 3 Zombie : B O > Attacker : 5 1 9 6 2 R id= 1 5 6 95 Attacker : 5 1 8 8 7 > Zombi e : B O SA id=6 1 0 3 4 Zomb i e : B O > Attacker : 5 1 8 8 7 R id= 1 5 6 9 6 Just to be sure, Nmap tries ports 20-23 again. A Zombie IP I D query shows no sequence jump. O n the off chance that a SYN/ACK from Target to Zombie came in late, Nmap tries another IP ID query. This again shows no open ports. Nmap is now sufficiently confident with its results to print them. e Idlescan took 5 seconds to scan 7 port s . teresting por t s on Target : RT STATE SERV I CE O/tcp closed l f i l tered ftp-data l/tcp closed l f i l tered ftp 2/tcp c losed l f i l t ered s s h 3/tcp closed l f i l t ered te lnet 4/tcp closed l f i l tered priv-ma i l smtp 5/tcp open 10/tcp open pop3 E Nmap finished : 1 IP add r e s s ( 1 host up ) scanned in 5 . 9 4 9 s econds For complete detai ls on the Nmap idle scan implementation, read idle_scan . c c from the Nmap source code distribution. While port scanning is a clever abuse of predictable IP ID sequences, they can be exploited for many other purposes as well. Examples are peppered throughout this book, particularly in Chapter 1 0, Detecting and Subverting Firewalls and Intrusion Detection Systems [257). 5.1 1 . I P P rotocol Scan (-so) IP protocol scan allows you to determine which IP protocols (TCP, ICMP, IGMP, etc.) are supported by target machines. This isn't technically a port scan, since it cycles through IP protocol numbers rather than TCP or UDP port numbers. Yet it still uses the -p option to select scanned protocol numbers, reports its results within the normal port table format, and even uses the same underlying scan engine as the true port scanning methods. So it is close enough to a port scan that it belongs here. 5 . 1 1 . IP Protocol Scan (-sO) 1 25 Besides being useful in its own right, protocol scan demonstrates the power of open-source software. While the fundamental idea is pretty simple, I had not thought to add it nor received any requests for such functionality. Then in the summer of 2000, Gerhard Rieger conceived the idea, wrote an excellent patch implementing it, and sent it to the nmap-hackers mailing list. I incorporated that patch into the Nmap tree and released a new version the next day. Few pieces of commercial software have users enthusiastic enough to design and contribute their own improvements! · Protocol scan works in a similar fashion to UDP scan. Instead of iterating through the port number field of a UDP packet, it sends IP packet headers and iterates through the eight-bit IP protocol field. The headers are usually empty, containing no data and not even the proper header for the claimed protocol. An exception is made for certain popular protocols (including TCP, UDP, and ICMP). Proper protocol headers for those are included since some systems won't send them otherwise and because Nmap already has functions to create them. Instead of watching for ICMP port unreachable messages, protocol scan is on the lookout for ICMP protocol unreachable messages. Table 5.8 shows how responses to the IP probes are mapped to port states. Table 5.8. How Nmap interprets responses to an IP protocol probe Probe Response Assigned State Any response in any protocol from target host open (for protocol used by response, not necessarily probe protocol) ICMP protocol unreachable error (type 3, code 2) c l o sed Other ICMP unreachable errors (type 3, code I , 3, 9, f i l tered (though they prove ICMP is open if sent J O, or 1 3) from the target machine) No response received (even after retransmissions) open I f i l t ered Like open ports in the TCP or UDP protocols, every open protocol is a potential exploitation vector. In addition, protocol scan results help determine the purpose of a machine and what sort of packet filtering is in place. End hosts usually have little more than TCP, UDP, ICMP, and (sometimes) IGMP open, while routers often offer much more, including routing-related protocols such as GRE and EGP. Firewalls and VPN gateways may show encryption-related protocols such as IPsec and SWIPE. Like the ICMP port unreachable messages received during a UDP scan, ICMP protocol unreachable messages are often rate limited. For example, no more than one ICMP destination unreachable response is sent per second from a default Linux 2.4.20 box. Since there are only 256 possible protocol numbers, this is Jess of a problem than with a 65,536-port UDP scan. The suggestions in Section 5.4.2, "Speeding Up UDP Scans" [ 1 05] apply to speeding up IP protocol scans as well. Protocol scan is used the same way as most other scan techniques on the command line. Simply specify -so in addition to whatever general Nmap options please you. The normal port (-p) option is used to select protocol numbers. Or you can use -F to scan all protocols listed in the nmap-protoco l s database. By default, Nmap scans all 256 possible values. Example 5.20 shows Ereet scanning a router in Poland followed by a typical Linux box on my local network. 1 26 5. l l . IP Protocol Scan (-sO) Example 5.20. IP protocol scan of a router and a typical Linux 2.4 box I nmap -so 6 2 . 2 3 3 . 1 7 3 . 9 0 para Starting Nmap ( h t tp : / / nmap . or g nterest ing protocols o n ntwklan-6 2 - 2 3 3 - 1 7 3 - 9 0 . devs . f u turo . pl ( 6 2 . 2 3 3 . 1 7 3 . 9 0 ) : ot shown : 2 4 0 c l osed por t s SERV I CE OTOCOL STATE i cmp open open I f i l tered i p t cp open open I f i l t ered egp open I f i l tered i gp udp fi l tered open I f i l t ered gre swipe f i l t ered open I f i l t ered narp mob i l e f i l tered sun-nd f i l t ered open I f i l t e r ed i s o - i p open I f i l t ered e igrp open I f i l t ered o s p f i gp open I f i lt ered ipip pim f i l te red eresting protocols on para ( 1 9 2 . 1 6 8 . 1 0 . 1 9 1 ) : shown : 2 5 2 c l osed port s SERVICE TOCOL STATE i cmp open open / fi l t ered i gmp open t cp udp f i lt ered 0 0 : 6 0 : 1 D : 3 8 : 3 2 : 9 0 ( Lucent Technol og i e s ) .1 2. TCP FTP Bou nce Scan (-b) interesting feature of the FTP protocol (RFC 959) is support for so-called proxy FTP connections. This a user to connect to one FTP server, then ask that files be sent to a third-party server. Such a feature for abuse on many levels, so most servers have ceased supporting it. One of the abuses this feature is causing the FTP server to port scan other hosts. Simply ask the FTP server to send a file to each ting port of a target host in turn. The error message will describe whether the port is open or not. This good way to bypass firewal ls because organizational FTP servers are often placed where they have more to other internal hosts than any old Internet host would. Nmap supports FTP bounce scan with the ·on. It takes an argument of the form : ® : . name or IP address of a vulnerable FTP server. As with a normal URL, you may omit ername>: , in which case anonymous login credentials (user: anonymous rd:-wwwu s e r @ ) are used. The port number (and preceding colon) may be omitted as well, in which lhe default FTP port (21 ) on is used. 5 . 1 2. TCP FTP Bounce Scan (-b) 1 27 In Example 5.21 , I attempt to bounce off the main Microsoft FfP server to scan Google. Example 5.21. Attempting an FTP bounce scan # nmap -PN -b ftp . microsof t . com googl e . com Start ing Nmap http : / / nmap . or g Y o u r FTP bounce s erver doesn ' t a l l ow p r i v i l eged por t s , s k ipping them . Your FTP bounce server suck s , it won ' t let us feed bogus port s ! Frequent users of the FfP bounce scan better get used to that error message. This vulnerability was widespread in 1997 when Nmap was released, but has largely been fixed. Vulnerable servers are still around, so it is worth trying when all else fails. If bypassing a firewall is your goal, scan the target network for open port 21 (or even for any FfP services if you scan all ports with version detection), then try a bounce scan using each. Nmap will tell you whether the host is vul nerable or not. If you are just trying to cover your tracks, you don't need to (and, in fact, shouldn't) limit yourself to hosts on the target network. Before you go scanning random Internet addresses for vulnerable FfP servers, consider that sysadmins may not appreciate you abusing their servers in this way. Example 5 .22 shows a successful bounce scan against a few interesting ports on Scanme. The verbose option (-v) was given to provide extra detail. The given server type of "JD FfP Server" means that this is an HP JetDirect print server. Example 5.22. Successful FTP bounce scan krad- > nmap -p 2 2 , 2 5 , 1 3 5 -PN -v -b XXX . YY . 1 1 1 . 2 scanme . nmap . org Start ing Nmap ( http : / / nmap . or g Att empt ing conne c t i on to f tp : / / a nonymous : -wwwu s e r @ @ XXX . YY . l l l . 2 : 2 1 Connected : 2 2 0 JD FTP Server Ready Log i n creden t i a l s a ccepted by ftp s erver ! I n i t i a t ing TCP ftp bounce scan a ga i n s t s canme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) Add i n g open port 2 2 / t cp Adding open port 2 5 / t cp S canned 3 por t s in 1 2 s econds v i a the Bounce scan . I nt e r e s t ing por t s on s canme . nmap . or g ( 6 4 . 1 3 . 1 3 4 . 5 2 ) : PORT S TATE SERV I CE 2 2 / t cp open ssh 2 5 / t cp open smtp 1 3 5 / tcp f i l tered msrpc Nmap done : 1 I P addr e s s ( 1 host up ) scanned i n 2 1 . 7 9 seconds 5.1 3. Scan Code and Algorith ms In 2004, Nmap's primary port scanning engine was rewritten for greater performance and accuracy. The new engine, known as u l t r a_scan after its function name, handles SYN, connect, UDP, NULL, FIN, Xmas, ACK, window, Maimon, and IP protocol scans, as well as the various host discovery scans. That leaves onlf idle scan and FfP bounce scan using their own engines. 1 28 5 . 1 3 . Scan Code and Algorithms While the diagrams throughout this chapter show how each scan type works, the Nmap implementation is far more complex since it has to worry about port and host parallelization, latency estimation, packet loss detection, timing profiles, abnormal network conditions, packet filters, response rate limits, and much more. This section doesn't provide every low-level detail of the ul t r a_s can engine. If you are inquisitive enough to want that, you are better off getting it from the source. You can find ul t r a_s c a n and its high-level helper functions defined in s c an_eng i n e . cc from the Nmap tarball. Here I cover the most important algorithmic features. Understanding these helps in optimizing your scans for better performance, as described in Chapter 6, Optimizing Nmap Performance [ 1 35]. 5.1 3 .1 . Network Condition Mon itoring Some authors brag that their scanners are faster than Nmap because of stateless operation. They simply blast out a Hood of packets then listen for responses and hope for the best. While this may have value for quick surveys and other cases where speed is more important than comprehensiveness and accuracy, I don't fi nd it appropriate for security scanning. A stateless scanner cannot detect dropped packets in order to retransmit and throttle its send rate. If a busy router half way along the network path drops 80% of the scanner's packet flood, the scanner will still consider the run successful and print results that are woeful ly inaccurate. Nmap, on the other hand, saves extensive state in RAM while it runs. There is usually plenty of memory available, even on a PDA. Nmap marks each probe with sequence numbers, source or destination ports, ID fields, or other aspects (depending on probe type) which allow it to recognize responses (and thus drops). It then adjusts its speed appropriately to stay as fast as the network (and given command-line options) allow without crossing the line and suffering inaccuracy or unfairly hogging a shared network. Some administrators who have not installed an IDS might not notice an Nmap SYN scan of their whole network. But you better believe the administrator will investigate if you use a brute packet flooding scan ner that affects his Quake ping time ! While Nmap's congestion control algorithms are recommended for most scans, they can be overridden. The --min-rate option sends packets at the rate you specify (or higher) even if that exceeds Nmap's normal congestion control limits. Similarly, the - -max-ret r i e s option controls how many times Nmap may retransmit a packet. Options such as --mi n -rate 1 0 0 - -max-ret r i e s 0 will emulate the behavior of simple stateless scanners. You could double that speed by specifying a rate of 200 packets per second rather than 100 pps, but don't get too greedy-an extremely fast scan is of little value if the results are wrong or incomplete. Any use of - -mi n-rate is at your own risk. 5.1 3.2. Host and Port Paral lel ization Most of the diagrams i n this chapter illustrate using a technique to determine the state of a single port. Sending a probe and receiving the response requires a round trip time (RTT) between the source and target machines. If your RIT is 200 ms and you are scanning 65,536 ports on a machine, handling them serially would take at least 3.6 hours. Scan a network of 20,000 machines that way and the wait balloons to more than eight years. This is clearly unacceptal,)le so Nmap parallelizes its scans and is capable of scanning hundreds of ports on each of dozens of machines at the same time. This improves speeds by several orders of magnitude. The number of hosts and ports it scans at a time is dependent on arguments described in Chapter 6, Optimizing Nmap Performance [ 1 35], including - -mi n - h o s t gr oup, - -mi n -p a ra l l e l i sm, - T 4 , --max - r t t - t imeout, and many others. It also depends on network conditions detected by Nmap. 5 . 1 3 . Scan Code and Algorithms 1 29 When scanning multiple machines, Nmap tries to efficiently spread the load between them. If a machine appears overwhelmed (drops packets or its latency increases), Nmap slows down for that host while continuing against others at full speed. 5 .1 3.3. Round Trip Time Estimation Every time a probe response is received, Nmap calculates the microseconds elapsed since the probe was sent. We'll call this the instanceRTI, and Nmap uses it to keep a running tally of three crucial timing-related values: s r t t , rttvar, and t imeout. Nmap keeps separate values for each host and also merged values for a whole group of hosts scanned in parallel. They are calculated as follows: srtt The smoothed average round trip time. This is what Nmap uses a s its most accurate RTI guess. Rather than use a true arithmetic mean, the formula favors more recent results because network conditions change frequently. The formula is: new s r t t = oldsrtt + ( i n s t anceRTT - o l d s r t t ) I 8 rt tvar This is the observed variance or deviation in the round trip time. The idea is that if RTI values are quite consistent, Nmap can give up shortly after waiting the s r t t . If the variance is quite high, Nmap must wait much longer than the s r t t before giving up on a probe because relatively slow responses are common. The formula follows (ABS represents the absolute value operation): newr t tvar = o l d r t t var + ( ABS ( i n s t a n ceRTT - o l d s r t t ) - o l drt t var ) I 4 t imeout This is the amount of time Nmap is willing to wait before giving up on a probe. It is calculated as: t imeout = new s r t t + newr t tvar * 4 When a probe times out, Nmap may retransmit the probe or assign a port state such as f i 1 tered (depending on scan type). Nmap keeps some state information even after a timeout just in case a late response arrives while the overall scan is still in progress. These simple time estimation formulas seem to work quite well. They are loosely based on similar techniques used by TCP and discussed in RFC 2988, Computing TCP's Retransmission Timer. We have optimized those algorithms over the years to better suit port scanning. 5.1 3.4. Congestion Control Retransmission timers are fa r from the only technique Nmap gleaned from TCP. Since Nmap i s most commonly used with TCP, it is only fair to follow many of the same rules. Particularly since those rules are the result of substantial research into maximizing throughput without degrading into a tragedy of the commons where everyone selfishly hogs the network. With its default options, Nmap i s reasonably polite. Nmap uses three algorithms modeled after TCP to control how aggressive the scan is: a congestion window, exponential backoff, and slow start. The congestion window controls how many probes Nmap may have outstanding at once. If the window is full, Nmap won't send any more until a response is received or a probe times out. Exponential backoff causes Nmap to slow down dramatically when it detects dropped packets. The congestion 130 5 . 1 3 . Scan Code and Algorithms window is usually reduced to one whenever drops are detected. Despite slow being in the name, slow start is a rather quick algorithm for gradually increasing the scan speed to determine the performance limits of the network. All of these techniques are described in RFC 258 1 , TCP Congestion Control. That document was written by networking gurus Richard Stevens, Vern Paxson, and Mark Allman. It is only 10 pages long and anyone interested in implementing efficient TCP stacks (or other network protocols, or port scanners) should fi nd it fascinating. · When Nmap scans a group of targets, it maintains in memory a congestion window and threshold for each target, as well as a window and threshold for the group as a whole. The congestion window is the number of probes that may be sent at one time. The congestion threshold defines the boundary between slow start and congestion avoidance modes. During slow start, the congestion window grows rapidly in response to responses. Once the congestion window exceeds the congestion threshold, congestion avoidance mode begins, during which the congestion window increases more slowly. After a drop, both the congestion window and threshold are reduced to some fraction of their previous value. There is an important difference between TCP streams and Nmap port scans, however. In TCP streams, it's normal to expect ACKs in response to every packet sent (or at least a large fraction of them). In fact, proper growth of the congestion window depends on this assumption. Nmap often finds itself in a different situation: facing a target with a default-deny firewall, very few sent packets will ever be responded to. The same thing happens when ping scanning a block of network addresses that contains only a few live hosts. To compensate for this, Nmap keeps track of the ratio of packets sent to responses received. Any time the group congestion window changes, the amount of the change is multiplied by this ratio. In other words, when few packets receive responses, each response carries more weight. A graphical description of how the group congestion window and threshold vary during a typical port scan is shown in Figure 5.9. The congestion window is shown in black and the congestion threshold is in gray. Figure 5.9. Congestion window and threshold "' i 100 60 ---· - --- ------ · The congestion window starts low and the congestion threshold starts high. Slow start mode begins and the window size increases rapidly. The large "stairstep" jumps are the result of timing pings. At about 10 seconds, the congestion window has grown to 80 probes when a drop is detected. Both the congestion window and threshold are reduced. The congestion window continues to grow until about 80 seconds when another drop is detected. Then the cycle repeats, which is typical when network conditions are stable. 5 . 1 3 . Scan Code and Algorithms 131 Drops during a scan are nothing to be afraid of. The purpose of the congestion control algorithms is to dynamically probe the network to discover its capacity. Viewed in this way, drops are valuable feedback that help Nmap determine the correct size for the congestion wi ndow. 5.1 3.5. Ti m i ng probes Every technique discussed in this algorithm s section involves (at some level) network monitoring to detect and estimate network packet loss and latency. This really is critical to obtaining fast scan times. Unfortunately, good data is often difficult to come by when scanning heavily firewalled systems. These filters often drop the overwhelming majority of packets without any response. Nmap may have to send 20,000 probes or more to find one responsive port, making it difficult to monitor network conditions. To combat this problem, Nmap uses timing probes, also known as port scan pings. If Nmap has found at least one port responsive on a heavily filtered host, it will send a probe to that port every 1 .25 seconds that it goes without receiving responses from any other ports. This allows Nmap to conduct a sufficient level of monitoring to speed up or slow down its scans as network conditions allow. 5.1 3.6. I nferred Neig h bor Ti mes Sometimes even port scan pings won't help because no responsive ports at all have been found. The machine could be down (and scanned with -PN), or every single port could be filtered. Or perhaps the target does have a couple responsive ports, but Nmap has not been l ucky enough to find them yet. In these cases, Nmap uses timing values that it m aintains for the whole group of machines it is scanning at the same time. As long as at least one response has been received from any machine in the group, Nmap has something to work with. Of course Nmap cannot assume that hosts in a group always share similar timing characteristics. So Nmap tracks the timing variances between responsive hosts in a group. If they differ wildly, Nmap infers long timeouts for neighboring hosts to be on the safe side. 5.1 3.7. Adaptive Retransm ission The simplest of scanners (and the stateless ones) generally don't retransmit probes at all. They simply send a probe to each port and report based on the response or lack thereof. Slightly m ore complex scanners will retransmit a set number of times. Nmap tries to be smarter by keeping careful packet loss statistics for each scan against a target. If no packet loss is detected, Nmap may retransmit only once when it fails to receive a probe response. When m assive packet loss is evident, Nmap may retransmit ten or m ore times. This allows Nmap to scan hosts on fast, reliable networks quickly, while preserving accuracy (at the expense of some speed) when scanning problematic networks or machines. Even Nmap's patience isn't unlimited though. At a certain point (ten retransmissions), Nmap will print a warning and give up on further retransmissions. This prevents malicious hosts from slowing Nmap down too m uch with intentional packet drops, slow responses, and similar shenanigans. Such an attack is known as tarpitting and is com monly used against spammers. 5.1 3.8. Scan Delay Packet response rate limiting is perhaps the most pernicious problem faced by port scanners such as Nmap. For example, Linux 2.4 kernels limit ICMP error m essages returned during a UDP (- s u) or IP protocol (- so) scan to one per second. If Nmap counted these as normal drops, it would be continually slowing down (remember exponential backoft) but still end up having the vast majority of its probes dropped. Instead, 1 32 5 . 1 3 . Scan Code and Algorithms Nmap tries to detect this situation. When a large proportion of packets are being dropped, it implements a short delay (as little as 5 milliseconds) between each probe sent to a single target. If drops continue to be a major problem, Nmap will keep doubling the delay until the drops cease or Nmap hits the maximum allowed scan delay. The effects of scan delay while UDP scanning ports 1 -50 of a response rate-limited Linux host are shown in Figure 5.10. At the beginning, the scan rate is unlimited by scan delay, though of course other mechanisms such as congestion control impose their own limits. When drops are detected, the scan delay is doubled, meaning that the maximum scan rate is effectively halved. In the graph, for example, a maximum scan rate of five packets per second corresponds to a scan delay of 200 milliseconds. Figure 5.10. Scan rate as affected by scan delay cu ,.._ :g E: The maximum scan delay defaults to one second between probes. The scan delay is sometimes enabled when · a slow host can't keep up, even when that host has no explicit rate limiting rules. This can reduce total network traffic substantially by reducing wasted (dropped) probe packets. Unfortunately even small scan delay values can m ake a scan takes several times as long. Nmap is conservative by default, allowing second-long scan delays for TCP and UDP probes. If your priorities differ, you can configure maximum scan delays with --max-scan-de l a y as discussed in Chapter 5, Port Scanning Techniques and Algorithms (95 ) . 5 . 1 3. Scan Code and Algorithms 1 33 Chapter 6. Optimizing Nmap Performance 6.1 . Introd uction One of my highest Nmap development priorities has always been performance. A default scan (nmap ) of a host on my local network takes a fifth of a second. That is barely enough time to blink, but adds up when you are scanning hundreds or thousands of hosts. Moreover, certain scan options such as UDP scanning and version detection can i ncrease scan times substantially. So can certain firewall configurations, particularly response rate limiting. While Nmap utilizes parallelism and many advanced algorithms to accelerate these scans, the user has ultimate control over how Nmap runs. Expert users carefully craft Nmap commands to obtain only the information they care about while meeting their time constraints. While Nmap performance is a high priority, accuracy is even more important. Authors of competing scanners have given high-profile conference presentations about how their scanner only takes four seconds to scan an entire class B address space. These scanners are actually trivial to write, since they omit all the congestion control and packet loss detection algorithms, leaving just a tight loop spewing probe packets as fast as the system can generate or the wire can bear. Such scanners are often promoted as stateless-meaning they have also omitted the code to track and retransmit probes. You can achieve similar behavior with Nmap by adding Hags such as --mi n-rate 1 0 0 0 to request that Nmap send at least 1 ,000 packets per second, and --max-ret ries O to disable retransmission of timed-out probes. Yet I rarely recommend this. Ninety-nine percent of the packets may be dropped by the next router upstream, and the scanner will never know the difference. Unmetered packet blasting scanners such as Scanrand 1 are useful i n some situations, but Nmap takes a m uch more conservative and accurate route. Nmap assumes the worst (high latency and packet loss) of the target networks at first, then speeds up as it gathers statistics showing that it can safely do so. While this happens automatically, an administrator can quicken the learning process by passing hints about the network to Nmap. An example of such a hint would be - -max - r t t - t imeout 2 0 0 , which allows Nmap to assume that any responses to a target host probe will come within 200 milliseconds. This chapter first discusses high-level methodologies for i mproving scan times. Then it covers how timing templates and low-level controls are used to speed up Nmap without impacting accuracy. It finishes with a tutorial by Jack Mogren of the Mayo Clinic, detailing how he improved scan time against his 676,352-IP network from nearly a week to 46 hours. Considering the huge importance of scanner performance, this chapter may seems short. This is because the chapter focuses on high-level general scanning performance tips, while tips for optimizing specific scan techniques are spread throughout this book where those techniques are covered. 6.2. Scan Time Reduction Tec h n iq ues The ideal solution to long scan times is to reduce them. This section offers many high-level tips for doing so. Unlike many circumstances in life, tuning your Nmap command line can make a huge difference. 1 http://sectools.orgltools4.html#scanra11d 6. 1 . Introduction 1 35 Hot-rodding your Honda Accord with a coffee-can exhaust tip, a three-foot-high spoiler, and a big red "type R" sticker won't reduce your 0-60 time much. Yet Section 6.7, "Scanning 676,352 IP Addresses in 46 Hours" [ 1 43) describes how Jack Mogren shaved days off his Nmap runtime by simply adding a few stickers (I mean options) to his Nmap command line. 6.2.1 . Omit Non-critical Tests The electronic equivalent to buying a Hummer when you never leave the pavement or carry more than groceries is to launch an intense and comprehensive Nmap scan to obtain a relatively trivial amount of information. Wasting a few seconds per host rarely matters on a home network, but can make daily WAN scans infeasible for large enterprises. The following list details the most common over-scanning mistakes, starting with the most egregious newbie bloopers and followed by more subtle problems that even advanced users confront. Specify ping scan (-sP) when you only need to determine what hosts are online. Some people determine whether a host is online using the command nmap . While this works, it is overkill. Nmap will send two packets to determine that the host is up, then at least 1,000 to port scan the host. The problem is amplified when a whole network is scanned this way to find all online hosts, or one particular host. Rather than waste time port scanning, specify - s P to do a ping scan when all you wish to know is what hosts are up or what their MAC addresses are. Limit the number of ports scanned. By default, Nmap scans the most common 1 ,000 ports. On a fast network of responsive machines, this may take a fraction of a second per host. But Nmap must slow down dramatically when it encounters rate limiting or firewalls that drop probe packets without responding. UDP scans can be agonizingly slow for these reasons. Yet the vast majority of open ports fall into just a few hundred port numbers. A port scan will be about 10 times as fast if you only scan 100 ports instead of the default 1,000. You can scan j ust the most popular 1 00 ports with the -F (fast scan) option, specify an arbitrary number of top ports with - - t op-por t s , or provide a custom list of ports to -p. Skip advanced scan types (- s c, - s V, -0, - - t r a ceroute, and -A). Some people regularly specify the -A Nmap option, which gives them the works. It causes Nmap to do OS detection, version detection, script scanning (NSE), and traceroute as well as the default port scan. Version detection can be extraordinarily useful , but can also bog down a large scan. So can NSE. When pressed for time, you can always skip - s c and - sv on the large scale scan and then perform them on individual ports as necessary later. OS detection is not nearly as slow as version detection, but it can still easily take up 5- 10 seconds per online host. Even without this, you can often guess the OS based on the name, open ports, and MAC address on a LAN. And in many cases you may not care for the OS. So -0 is another candidate for only-as-necessary use. As a compromise, you can specify --os s c a n - l i m i t --max-os-tr ies 1 which tells Nmap not to retry OS detection attempts which fail to match, and also to skip OS detection against any on line hosts that don't have at least one open TCP port and one closed TCP port. OS detection isn't as accurate against such hosts anyway. 1 36 6.2. Scan Time Reduction Techniques Remember to turn off DNS resolution when it isn't necessary. By default, Nmap performs reverse-DNS resolution against every host that is found to be online. It is done against all hosts if you skip the ping step with -PN or specify -R. This was a major bottleneck when host DNS libraries were used to look up one IP at a time. While Nmap now has a fast parallel reverse-DNS system to speed queries, they still can take a substantial amount of time. Disable them with the -n option when you don't need the data. For simple scans (such as ping scans) against a large number of hosts, omitting DNS can sometimes reduce scan time by 20% or more. DNS time is not a major factor in more involved scans which probe thousands of ports or utilize intensive features such as version detection. If you want the Nmap host machine to handle name resolution (using the get hostbyaddr function), specify the - - s y s t ern-dn s option. Doing so can slow scans down dramatically. 6.2.2. Optim ize Ti m i ng Parameters Nmap offers dozens of options for providing hints and rules to control scan activity. These range from high level timing aggressiveness levels provided by the -T option (described in Section 6.6, "Timing Templates (·T)" [ 1 42)) to the finer-grained controls described in Section 6.5, "Low-Level Timing Controls" [ 1 4 1 ] . You can even combine the two. These options are particularly useful when scanning highly filtered networks where Nmap receives few responses to determine its own timing estimates. Scan time can often be safely cut in half. Most of these options will have little effect against a local LAN fi lled with responsive hosts, as Nmap can determine optimal values itself in that case. 6.2.3. Separate and Optim ize U DP Scans Scanning UDP ports i s important because many vulnerable services use that protocol, but the timing characteristics and performance requirements of UDP scans are much different than TCP scans. Of particular concern is ICMP error rate-limiting, which is extremely common and affects UDP scans far more often than TCP. for these reasons, I don't recommend combining TCP and UDP scans when performance is critical, even !bough Nmap supports doing so with options such as - s SU. You often want different timing flags for each tocol, requiring separate command lines. Section 5.4.2, "Speeding Up UDP Scans" [ 1 05) provides valuable tricks and real-life examples for improving UDP scan performance. . 2.4. Upgrade Nmap have been many cases where I have investigated reports of poor Nmap performance only to find that reporter used an ancient version that was many years out of date. The newest versions of Nmap have portant algorithmic improvements, bug fixes, performance-enhancing features such as local network ARP ning, and more. The first response to performance problems should be to compare your version of Nmap nmap -V) with the latest version available from http://nmap.org. Upgrade if necessary. If it is still not enough, try the other techniques in this chapter. 6.2. Scan Time Reduction Techniques 1 37 6.2.5. Execute Concu rrent Nmap I nstances Some people try to speed up Nmap by executing many copies in parallel against one target each. For example, the Nessus scanner used to do this by default. This is usually much less efficient and slower than letting Nmap run against the whole network. Nmap has its own parallelization system that is customized to its needs, and Nmap is able to speed up as it learns about network reliability when it scans a large group. Further, there is substantial overhead in asking the OS to fork 65,536 separate Nmap instances just to scan a class B. Having dozens of copies of Nmap running in parallel is also a memory drain since each instance loads its own copy of the data files such as nmap- s e r v i c e s and nmap-o s -db. While launching single-host Nmap scans in parallel is a bad idea, overall speed can usually be improved by dividing the scan into several large groups and executing those concurrently. Don't go overboard though. Five or ten Nmap processes are fine, but launching 100 Nmap processes at once is not recommended. Launching too many concurrent Nmap processes leads to resource contention. Another sort of concurrency is to run Nmap from different hosts at once. You can have cron (or At on Windows) schedule local hosts on each of your networks to start scanning machines local to them at the same time, then mail the results to a central data server. Scanning your Australian network from the U.S. will be slower than scanning it from a local machine on that network. The difference will be even greater if the U.S. machine must traverse extra firewalls to reach the distant network. 6.2.6. Scan From a Favorable Network Location Restrictive firewalls can turn a five-second scan into a multi-hour chore. The latency and packet loss associated with some Internet routes doesn't help either. If you can run Nmap from host(s) local to the target network, do so. Of course if the goal i s to view the network as an external attacker would, or to test the firewall, external scanning is required. On the other hand, scanning and securing the internal network provides defense in depth which i s critical against internal threats and those wily attackers who circumvent the firewall (see Chapter 10, Detecting and Subverting Firewalls and Intrusion Detection Systems [257)). When doing reverse DNS resolution, especially if you have a heavily burdened local nameserver, it can help to use a less busy nameserver or directly query the authoritative nameservers. This gain is usually slight and only worth doing for repeated or enormous scans. Of course, there are sometimes non-performance reasons for choosing nameservers. 6.2. 7. I ncrease Ava i lable Bandwidth and CPU Time You can occasionally improve Nmap scan times by increasing your available bandwidth or CPU power. This may be done either by installing a new data line or CPU, or by halting concurrently running applications which compete for these resources. For example, Nmap will run slower if you concurrently saturate your DSL line by downloading a pirate torrent of The Matrix Reloaded. It is far more common that Nmap is constrained by its own congestion control algorithms than being CPU-bound or limited by the available local bandwidth. These controls help prevent network flooding and increase accuracy. Increasing CPU power and local bandwidth won't help this sort of self-limiting by Nmap--timing options must be adjusted instead. You can test whether Nmap is CPU constrained by monitoring your CPU load with an application such as top on Unix or the Task Manager on Windows. If your CPU spends most of its time idle, then upgrading won't help much. To test Nmap's bandwidth usage, run it in 138 6.2. Scan Time Reduction Techniques verbose mode ( - v) . Nmap will then report the number of bytes sent and received and its execution time, as shown in Example 6. 1 . Example 6.1. Bandwidth usage over local 100 Mbps ethernet network t nrnap -v -n -p- sec . t i t an . net i Starting Nmap ( http : / / nmap . or g 1 1 0 lines c u t J teresting por t s on 1 9 2 . 1 6 8 . 0 . 8 : t shown : 6 5 5 3 4 c losed por t s RT STATE SERV I CE /tcp open s s h C Address : 0 0 : 1A : 6 B : C 1 : 3 3 : 3 7 ( US I ) p done : 1 IP addre s s ( 1 host up ) s canned i n 2 . 2 0 s econds Raw pack e t s sent : 6 5 5 3 6 ( 2 . 8 8 4MB ) I Rcvd : 6 5 5 3 6 ( 2 . 6 2 1 MB ) Multiply the byte values by eight and divide by the execution time to get the average bandwidth usage in bits per second. In Example 6. 1 , Nmap received 2,62 1 ,000 bytes (Nmap considers 1 ,000,000 bytes to be a MB) in 2.20 seconds. So receive traffic was about 9.5 Mbps (send rate was 1 0.5 Mbps). Therefore the 100 Mbps ethernet link isn't likely constraining Nmap, and upgrading to gigabit ethernet won't help much. Some consumer broadband devices and other equipment has a hard time dealing with the rate of packets sent by Nmap, even though the small packet size (usually Nmap sends empty headers) keeps bandwidth low. In Example 6. 1, "Bandwidth usage over local 100 Mbps ethernet network" [ 1 39], Nmap sent about 30,000 packets per second and received a similar number. Such high packet rates can cause problem with low-quality devices. In this case, we see that both send and receive packet counts were 65,536, which is the number of scanned ports (65,535) plus one for the initial ARP ping probe. Therefore Nmap did not encounter any packet drops requiring retransmission. This suggests again that the networking equipment was not a limiting factor-Nmap was probably CPU bound. 6.3. Coping Strateg ies for Long Scans While optimizing scan options to speed up a scan can take you a long way, there is a limit to how fast Nmap can run while preserving accuracy and treating competing network flows fairly. Large scans involving thousands of hosts, all 65K ports, UDP, or version detection are likely to take a while even after optimization. This section provides powerful strategies for coping with these long scans. 6.3.1 . Use a Multi-stage Approach A comprehensive security audit will need to include UDP and TCP scanning of all 65,536 ports for each protocol, usually with -PN just in case a machine is up but heavily filtered. Yet fewer than 100 of those port numbers are commonly used and most hosts are responsive with moderate host discovery options. So specify -F to perform a quick scan popular ports on known-online hosts first. That lets you analyze the online hosts and most of the open ports while you start the huge -PN scan of all TCP and UDP ports with version and 00 detection in the background. Short cut options for speeding up the quick scan are discussed in Section 6.2.1, "Omit Non-critical Tests" [ 1 36]. Once the slow scan is done, compare it to the earlier results to find any newly discovered hosts or ports. 6.3. Coping Strategies for Long Scans 1 39 6.3.2. Estimate and Plan for Scan Time I n many cases, the most frustrating aspect of long scans is having n o idea when they will complete. Nmap is now more helpful than it used to be in that it provides regular scan time estimates as long as verbose mode (-v) is enabled. Example 6.2. Estimating scan time # nmap - T 4 - s S -pO- - i R 5 0 0 -n - -mi n-host group 1 0 0 -v ( h t t p : //nmap . or g ) I n i t i a t i n g SYN S t ea l t h Scan aga i n s t 2 9 hos t s [ 6 5 5 3 6 por t s /hos t ) at 2 3 : 2 7 [. . .] SYN S t e a l t h Scan Timi n g : About 0 . 3 0 % done ; ETC : 0 9 : 4 5 ( 1 0 : 1 5 : 4 5 rema ining) S t a r t i n g Nmap Example 6.2 shows us that the SYN scan is likely to take ten hours and eighteen minutes (23 :27 to 9:45) to scan 29 hosts. So the total time Nmap will spend scanning the network can be roughly extrapolated by multiplying 21 minutes per host by the number of hosts online. If version detection or UDP are being done as well, you'll also have to watch the timing estimates for those. Another option is to wait until Nmap has fully completed scanning its first group of hosts. Then extrapolate the time taken for the size of that set over the size of the entire target network. This is simpler because you don't need to worry about individual scan components. Basing your estimates on the number of target IP addresses finished versus the target IP space size can be misleading, as online hosts are rarely evenly distributed among that IP space. They are usually found in clumps, often near the beginning of the IP space. So if the scan itself incl udes host discovery (i.e. no -PN option), a more accurate measure is to ping scan the entire network first and then base your estimates on the number of online hosts Nmap has completed scanning versus the number found online by the ping scan. · While occasional estimates are printed automatically in verbose mode, you can always request the current estimate by pressing (see Section 1 5. 1 5 , "Runtime Interaction" [ 41 O]). If the estimate is within your timeframe, you can schedule something else to do while it proceeds. That beats checking whether Nmap is done every 20 minutes. An estimate showing that Nmap won't finish on time is even more valuable. You can immediately work on optimizing the scan or lengthening the engagement. Your options are much more limited if you only determine the scan is too slow after the deadline passes and Nmap is still running. 6.4. Port Selection Data and Strateg ies Port scanning can be the most time consuming portion of an Nmap scan, even when the scan includes version detection or NSE scripts. Port scan time is roughly proportional to the number of ports scanned, so reducing the number of ports provides a significant performance boost. The down side is that reduced scans are less comprehensive, so you might miss open ports. The reality is that there are 65,536 ports in each protocol, and most of them are almost never open. I spent a summer conducting large-scale scans to determine the prevalence of each TCP and UDP port. The results include data from scanning tens of millions of Internet IP addresses as well as enterprise networks scanned from within. This section provides empirical results you can rely on to strike the right balance between speed and effectiveness in your scans. 140 6.4. Port Selection Data and Strategies While more than a hundred thousand (total) TCP and UDP ports exist, the vast majority of open ports fall within a much smaller set. According to our research, the top 10 TCP ports and top I ,075 UDP ports represent half of the open ports for their protocol. To catch 90% of the open ports, you need to scan 576 TCP ports and 1 1 ,307 UDP ports. By default, Nmap scans the top 1 ,000 ports for each scan protocol requested. This catches roughly 93% of the TCP ports and 49% of the UDP ports. With the -F (fast) option, only the top 100 ports are scanned, providing 78% TCP effectiveness and 39% for UDP. To specify a different number of ports, specify that value to the - - t op-po r t s option. Table 6.1 provides an approximation of the number of TCP or UDP ports you must scan to reach a given effectiveness rate for that protocol. Table 6.1. Required -- t op port s values for reaching various effectiveness levels - Effectiveness TCP ports required UDP ports required 10% I 5 20% 2 12 30% 4 27 40% 6 1 35 50% IO 1 ,075 60% 18 2,618 70% 44 5 , 1 57 80% 1 22 7,981 85% 236 9,623 90% 576 1 1 ,307 95% 1 ,558 1 3,035 99% 3,328 1 5,094 100% 65,536 65,536 While Nmap can handle port selection for you automatically (when you rely on defaults or use options such as -F or --top-por t s ), specifying ports explicitly with -p is often useful. In either case, familiarity with the most commonly seen open ports is important. The top ports according to our data are described in Section 4.1 .2, "What Are the Most Popular Ports?" [75) . 6.5. Low-Level Tim i ng Controls Nmap offers many fine-grained options for controlling scan speed. Most people use these options to speed Nmap up, but they can also be useful for slowing Nmap down. People do that to evade IDS systems, reduce network load, or even improve accuracy if network conditions are so bad that even Nmap's conservative default is too aggressive. Table 6.2 lists each low-level timing control option by function. For detailed usage information on every option, read Section 1 5 . 1 1 , "Timing and Performance" [394) . It is assumed that the reader is already familiar with the Nmap scanning algorithms described in Section 5 . 1 3 , "Scan Code and Algorithms" [ 1 28). 6.5. Low-Level Timing Controls 141 Table 6.2. Low-level timing controls by function Function Options Hostgroup (batch of hosts scanned concurrently) size --mi n - h o s t group, - -max - h o s t group Number of probes launched in parallel --min-par a l l e l i sm, - -max-par a l l e lism Probe timeout values --mi n - r t t - t imeou t , - -max-rtt-t imeout, - - i n i t i a l - r t t - t imeout Maximum number of probe retransmissions allowed - -max- r et r i e s Maximum time before giving u p o n a whole host - - h o s t - t imeout Control delay inserted between each probe against an - - s c an-del ay, --max- s can-de lay individual host Rate of probe packets sent per second --min-rate, - -max - r ate Defeat RST packet response rate by target hosts - -de feat - r s t -r a t e l imit 6.6. Ti m i ng Tem plates (-T) While the fine-grained timing controls discussed i n the previous section are powerful and effective, some people find them confusing. Moreover, choosing the appropriate values can sometimes take more time than the scan you are trying to optimize. So Nmap offers a simpler approach, with six timing templates. You can specify them with the - T option and their number ((}-5) or their name. The template names are paranoid (O), s ne a k y ( 1 ), p o l i t e ( 2 ), norma l ( 3 ), aggre s s ive ( 4 ), and i n s a ne ( 5 ). The first two are for IDS evasion. Polite mode slows down the scan to use Jess bandwidth and target machine resources. Normal mode is the default and so - T 3 does nothing. Aggressive mode speeds scans up by making the assumption that you are on a reasonably fast and reliable network. Finally insane mode assumes that you are on an extraordinarily fast network or are willing to sacrifice some accuracy for speed. These templates allow the user to specify how aggressive they wish to be, while leaving Nmap to pick the exact timing values. The templates also make some m inor speed adjustments for which fi ne-grained control options do not currently exist. For example, - T 4 prohibits the dynamic scan delay from exceeding 10 ms for TCP ports and - T S caps that value at 5 ms. Templates can be used in combination with fine-grained controls, and the granular options will override the general timing templates for those specific values. I recommend using - T 4 when scanning reasonably modern and reliable networks. Keep that option (at the beginning of the command line) even when you add fine-grained controls so that you benefit from those extra minor optimizations that it enables. Table 6.3 shows how the timing variables vary for each -T value. All time values are in milliseconds. Table 6.3. Timing templates and their effects TO Tl T2 T3 T4 Name Paranoid Sneaky Polite Normal Aggressive Insane m i n - r t t - t imeout J OO J OO 100 100 100 50 max - r t t - t imeout 300,000 1 5,000 10,000 10,000 1 ,250 300 142 6.6. Timing Templates (-T) TS TO Tl T2 T3 T4 TS initial -rtt -t imeout 300,000 1 5,000 1 ,000 1 ,000 500 250 max-ret ries 10 10 10 10 6 2 Initial (and minimum) scan delay 300,000 1 5,000 400 0 0 0 Maximum TCP scan delay 300,000 1 5,000 1 ,000 1 ,000 10 5 Maximum UDP scan delay 300,000 1 5,000 1 ,000 1 ,000 1 ,000 1 ,000 host-timeout 0 0 0 0 0 900,000 min-para l l e l i sm Dynamic, not affected by timing templates max-paral l e l i sm I Dynamic Dynamic min-hostgroup Dynamic, not affected by timing templates max-hostgr oup Dynamic, not affected by timing templates min-rate No minimum rate limit max-rate No maximum rate limit defeat-rst-ratel imit Not enabled by default (--scan-de lay) I I Dynamic Ifyou are on a decent broadband or ethernet connection, I would recommend always using -T 4 . Some people love TS though it is too aggressive for my taste. People sometimes specify - T 2 because they think it is less likely to crash hosts or because they consider themselves to be polite in general. They often don't realize just how slow -T p o l i t e really is. They scan may take ten times longer than a default scan. Machine crashes and bandwidth problems are rare with the default timing options (-T3) and so I normally recommend that for cautious scanners. Omitting version detection is far more effective than playing with timing values for reducing these problems. - While -TO and -Tl may be useful for avoiding IDS alerts, they will take an extraordinarily long time to scan thousands of machines or ports. For such a long scan, you may prefer to set the exact timing values you need rather than rely on the canned - T O and - T l values. 6.7. Scanning 676,352 I P Add resses i n 46 Hours This story was submitted by Jack L. Mogren of the Mayo Clinic. It functions as a tutorial, demonstrating the llleps he took to implement a regular Nmap scanning regime and reduce scan time of this huge network from a week to 46 hours. The Mayo Clinic has built a relatively large private network, with ARP tables indicating over 70,000 IP addresses in use. Our network management used to focus on creating and maintaining the physical architecture across three major campuses and several dozen satellites across the country. Our motto was "You need it? We11 build it". There was little regard for what was actually connected to the network. Network management conveniently ended at the data jack and suffered from the candy bar syndrome. It was crunchy and secure from the outside, but soft and chewy on the inside. We had well protected boundaries but few internal controls. 6. 7. Scanning 676,352 IP Addresses in 46 Hours 143 This attitude changed abruptly in January 2003 when the Slammer worm (W32.SQLExp) and its variants broke into our environment. Suddenly it became very important to know what was connected to our network. In the case of Slammer, we needed to know where all the devices running MS SQL Server 2000 or MSDE 2000 were l ocated and who the administrators were. Lacking this information, the effort to eradicate Slammer lasted several months. Thus was born the effort to "Know what's on the network". It sounds simplistic, but given size, complexi ty and network history, this was a major step forward and a new direction for our network management services. Nmap has proven to be a valuable tool in this effort. You can't beat the price, and I appreciate the advantages that the open-source community brings to its development. Especially OS fingerprinting and the many contributions provided by end users. I began experimenting with Nmap. My goal was to create a meaningful network inventory by using the Nmap -0 option to quickly perform remote host identification via TCP/IP fingerprinting. Let me start with a few words about our IP environment and my scanning platform. We currently own one class B and 44 class C ranges as well as using most of the private address space. That adds up to 676,352 possible IP addresses. I performed my scans from a Compaq DL380 running Red Hat Linux 8.0. My first attempt was this vanilla TCP SYN scan with OS detection (-0) and only ICMP echo requests for host discovery ( -PE): # nmap -0 -PE -v -ox mayo . xm l - i L ip_networ k s . t x t Unfortunately, that proceeded so slowly that it would have taken a week to scan our entire network. Given that all significant parts of our network were connected by at least a T I line ( 1 .54 Mbps), I added the i n sane canned timing policy ( - T 5 ). I also added fast scan mode (-F), which cut the number of ports scanned from about 1600 w 12002. I also added --ossca n - l imi t so Chae Nmap doesn 't waste time OS scanning hosts with no ports open. This resulted in the following command: # nmap -0 - T S -PE -F - - o s s can- l im i t -v -ox mayo . xml - i L ip_networks . txt Unfortunately, this looked l ike it would stil l take a few days. So I edited the nmap-servi ces file to trim down the number of ports to 270. The scan then finished in a little over 49 hours and found 66,558 devices. Tweaking the timing variables, removing the verbose option, and redirecting output to / dev / null reduced i that time to 46 hours. That left me with this f nal command: \ # nmap -0 - T S -PE -F - - o s s can- l im i t - -max-rt t -t imeout 1 0 0 - -max-para l l e l i sm 1 0 0 - -min-hos tgroup 1 0 0 -ox mayo . xml \ - i L ip_network s . t x t I plan to perform this scan on a weekly basis and provide the output in the XML format to an M S SQL database. Our other scan methods already feed into this database and we are able to create reports that help us meet our original goal of knowing what's on the network. I may decide to distribute the load by running subsets of the scanning on several systems. 2 With Nmap version 4.75 or higher, -F is even more effective in that it cuts the number of scanned ports to 144 6.7. Scanning 676,352 IP Addresses in 46 Hours JOO. Chapter 7. Service and A p p l icat i on Version Detect i on 7.1 . I ntrod uction While Nmap does many things, its most fundamental feature i s port scanning. Point Nmap at a remote machine, and it might tell you that ports 2 5 / t cp, 8 0 / t cp, and 5 3 / udp are open. Using its nmap- servi ces database of more than 2,200 well-known services, Nmap would report that those ports probably correspond to a mail server (SMTP), web server (HTTP), and name server (DNS) respectively. This lookup is usually accurate-the vast majority of daemons listening on TCP port 25 are, in fact, mail servers. However, you should not bet your security on this! People can and do run services on strange ports. Perhaps their main web server was already on port 80, so they picked a different port for a staging or test server. Maybe they think hiding a vulnerable service on some obscure port prevents "evil hackers" from finding it. Even more common lately is that people choose ports based not on the service they want to run, but on what gets through the firewal I . When ISPs blocked port 80 after major Microsoft ITS worms CodeRed and Nimda, hordes of users responded by moving their personal web servers to another port. When companies block Telnet access due to its horrific security risks, I have seen users simply run telnetd on the Secure Shell (SSH) port instead. Even if Nmap is right, and the hypothetical server above is running SMTP, HTTP, and DNS servers, that is · not a lot of information. When doing vulnerability assessments (or even simple network i nventories) of your companies or clients, you really want to know which mail and DNS servers and versions are running. Having an accurate version number helps dramatically in determining which exploits a server is vulnerable to. Do keep in mind that security fixes are often back-ported to earlier versions of software, so you cannot rely solely on the version number to prove a service is vulnerable. False negatives are rarer, but can happen when silly administrators spoof the version number of a vulnerable service to make it appear patched. Another good reason for determining the service types and version numbers is that many services share the same port number. For example, port 2 5 8 / t cp is used by both the Checkpoint Firewall- I GUI management interface and the yak Windows chat client. This makes a guess based on the nmap - s e r v i c e s table even less accurate. Anyone who has done much scanning knows that you also often find services listening on unregistered ports-these are a complete mystery without version detection. A final problem is that fi ltered UDP ports often look the same to a simple port scanner as open ports (see Section 5 .4, "UDP Scan (-sU)" [ 10 l ]). But if they respond to the service-specific probes sent by N map version detection, you know for sure that they are open (and often exactly what is running). Service scans sometimes reveal information about a target beyond the service type and version number. Miscellaneous information discovered about a service is collected in the "info" field. This is displayed in the VERS I ON column inside parentheses following the product name and version number. This field can include SSH protocol numbers, Apache modules, and much more. Some services also report their configured hostnames, which differ from machines' reverse DNS hostnames surprisingly often. The hostname field is reported on a Service I n f o line following the port table. It sounds like a minor information leak, but can have consequences. One year at the CanSecWest security conference, I was huddled up in my room with my laptop. Suddenly the tcpdump window in the corner of 7. 1 . Introduction 145 my screen went wild and I realized my machine was under attack. I scanned back and found an unusual high port sitting open. Upon connecting, the port spewed a bunch of binary characters, but one ASCII field in the output gave a configured domain name. The domain was for a small enough security company that I knew exactly who was responsible. I had the front desk ring his hotel room, and boy was he surprised when I asked him to stop probing my box. Two more fields that version detection can discover are operating system and device type. These are also reported on the S e r v i c e I n f o line. We use two techniques here. One is application exclusivity. I f we identify a service as Microsoft Exchange, we know the operating system is Windows since Exchange doesn't run on anything else. The other technique is to persuade more portable applications to divulge the platform information. Many servers (especially web servers) require very little coaxing. This type of OS detection is intended to complement Nmap's OS detection system ( -0) and can sometimes report differing results. Consider a Microsoft Exchange server hidden behind a port-forwarding Unix firewall . The Nmap version scanning subsystem obtains a l l o f this data by connecting to open ports and interrogating them for further information using probes that the specific services understand. This allows Nmap to give a detailed assessment of what is really running, rather than just what port numbers are open. Example 7.1 shows the actual output. Example 7.1. Simple usage of version detection # nmap -A - T 4 -F i n s ecure . or g Start i n g Nmap ( http : / /nmap . or g I n t e r e s t i ng port s on i n s e cure . er g ( 2 0 5 . 2 1 7 . 1 5 3 . 5 3 ) : ( Th e 1 2 0 6 port s s canned but not s hown bel ow are in s t at e : f i ltered ) PORT STATE SERV I C E VER S I ON OpenSSH 3 . l p l ( protocol 1 . 9 9 ) 2 2 / t cp open ssh 2 5 / t cp open smtp Qma i l smtpd 5 3 / t cp open domai n I SC BIND 9 . 2 . 1 Apache h t t p d 2 . 0 . 3 9 ( ( Un i x ) mod_per l / 1 . 9 9_0 7 -dev ) 8 0 / t cp open http 1 1 3 / tcp c losed auth Device t yp e : general purpos e Runn in g : L in u x 2 . 4 . X 1 2 . 5 . X OS deta i l s : L i n u x Kerne l 2 . 4 . 0 - 2 . 5 . 2 0 Nmap f i n i shed : 1 IP addr e s s ( 1 host up ) s canned in 3 4 . 9 6 2 seconds Nmap version detection offers the following advanced features (fully described later): • High speed, parallel operation via non-blocking sockets and a probe/match definition grammar designed for efficient yet powerful implementation. • Determines the application name and version number where available-not j ust the service protocol. • • Supports both the TCP and UDP protocols, as well as both textual ASCII and packed binary services. Multi-platform support, including Linux, Windows, Mac OS X, FreeBSD/NetBSD/OpenBSD, and all the other platforms on which Nmap is known to work. 146 7.1 . Introduction Solaris, • If SSL is detected, Nmap connects using OpenSSL (if available) and tries to determine what service is listening behind that encryption layer. This allows it to discover services like HTTPS, POP3S, IMAPS, etc. as well as providing version details. • If a SunRPC service is discovered, Nmap launches its brute-force RPC grinder to find the program number, name, and version number. • 1Pv6 is supported, including TCP, UDP, and SSL over TCP. • Community contributions: if Nmap gets data back from a service that it does not recognize, a service fingerprint is printed along with a submission URL. This system is patterned after the extremely successful Nmap OS Detection fingerprint submission process. New probes and corrections can also be submitted. • Comprehensive database: Nmap recognizes more than one thousand service signatures, covering more than 180 unique service protocols from ACAP, AFP, and AIM to XML-RPC, Zebedee, and Zebra. 7.2. Usage and Examples Before delving into the technical details of how version detection is implemented, here are some examples demonstrating its usage and capabilities. To enable version detection, j ust add - s V to whatever Nmap flags you normally use. Or use the A option, which turns on version detection and other Advanced and Aggressive features later. It is really that simple, as shown in Example 7.2. - Example 7.2. Version detection against www.microsoft.com nmap -A -T4 -F www.microsoft . com Starting Nmap ( http : //nmap . org ) lnteresting ports on 8 0 . 6 7 . 6 8 . 3 0 : The 1 2 0 8 ports scanned but not shown below are i n state : closed) f()�!cp ����E ::�VICE �:!���I SSH (protocol 1 . 5 ) AkamaiGHost (Akamai ' s HTTP Acceleration service) http /tcp open 3/tcp open ssl/http AkamaiGHost (Akamai ' s HTTP Acceleration service) vice type : general purpose nning : Linux 2 . l . X l 2 . 2 . X details : Linux 2 . 1 . 1 9 - 2 . 2 . 2 5 Nmap finished : 1 IP address ( 1 host up) scanned in 1 9 . 2 2 3 seconds I � This preceding scan demonstrates a couple things. First of all, it is gratifying to see www.Microsoft.Com served off one of Akamai's Linux boxes. More relevant to this chapter is that the listed service for port 443 is s s l / ht tp. That means that service detection first discovered that the port was SSL, then it loaded up OpenSSL and performed service detection again through SSL connections to discover a web server running AkamiGHost behind the encryption. Recall that -T 4 causes Nmap to go faster (more aggressive timing) and -F tells Nmap to scan only ports registered in nmap - s e r v i c e s . Example 7.3 is a longer and m ore diverse example. 7.2. Usage and Examples 147 Example 7.3. Complex version detection # nmap -A - T 4 localhost Starting Nmap ( http : / / nmap . or g I ntere s t i ng por t s on f e l i x ( 1 2 7 . 0 . 0 . 1 ) : ( The 1 6 4 0 port s s canned but not s hown be l ow are in s ta t e : c losed ) PORT STATE SERV I CE VERS I ON 2 1 / t cp open ftp WU-FTPD wu- 2 . 6 . 1 - 2 0 OpenSSH 3 . l p l ( protocol 1 . 9 9 ) 2 2 / t cp open s s h 5 3 / t cp open doma i n I SC B IND 9 . 2 . 1 7 9 / t cp open f i nger L inux f i ngerd 1 1 1 / t cp open rpcbind 2 ( rpc # 1 0 0 0 0 0 ) 4 4 3 / t cp open s s l / http Apache httpd 2 . 0 . 3 9 ( ( U n i x ) mod_pe r l / 1 . 9 9_ 0 4 -dev) 5 1 5 / t cp open printer 6 3 1 / tcp open ipp CUPS 1 . 1 9 5 3 / tcp open rndc? 5 0 0 0 / t cp open s s l / ft p WU-FTPD wu-2 . 6 . 1 - 2 0 OpenSSH 3 . l p l ( protocol 1 . 9 9 ) 5 0 0 1 / tcp open s s l / s s h 5 0 0 2 / tcp open s s l / doma i n I SC B I ND 9 . 2 . 1 5 0 0 3 / t cp open s s l / f i nger L i nux f i ngerd 6 0 0 0 / tcp open X l l ( ac c e s s den i ed ) 8 0 0 0 / tcp open http-proxy Junkbu ster webproxy 8 0 8 0 / t c p open http Apache ht tpd 2 . 0 . 3 9 ( ( Un i x ) mod_per l / 1 . 9 9_ 0 4 -dev) Apache ht tpd 2 . 0 . 3 9 ( ( Un i x ) mod_per l / 1 . 9 9_0 4 -dev ) 8 0 8 1 / t cp open http Device t ype : general purpose Runn i ng : L inux 2 . 4 . X l 2 . 5 . X OS detai l s : L i nux Kernel 2 . 4 . 0 - 2 . 5 . 2 0 Nmap f i n i shed : 1 I P addr e s s ( 1 host up ) s canned in 4 2 . 4 9 4 s econds You can see here the way RPC services are treated, with the brute-force RPC scanner being used to determine that port 1 1 1 is rpcbind version 2. You may also notice that port 5 1 5 gives the service as pr i nter, but that version column is empty. Nmap determined the service name by probing, but was not able to determine anything else. On the other hand, port 953 gives the service as "rn d c ? ". The question mark tells us that Nmap was not even able to determine the service name through probing. As a fallback, rndc is mentioned because that has port 953 registered in nmap - s e r v i ce s . Unfortunately, none of Nmap's probes elicited any sort of response from rndc. If they had, Nmap would have printed a service fingerprint and a submission URL so that it could be recognized in the next version. As it is, Nmap requires a special probe. One might even be available by the time you read this. Section 7. 7, "Community Contributions" [ 1 64] provides details on writing your own probes. It is also worth noting that some services provide much more information than j ust the version number. Examples above include whether X 1 1 permits connections, the SSH protocol number, and the Apache module versions list. Some of the Apache modules even had to be cut from the output to fit on this page. A few early reviewers questioned the sanity of running services such as SSH and finger over SSL. This was actually just fun with stunnel 1 , in part to ensure that parallel SSL scans actually work. 1 http://www.stunnel.org/ 148 7.2. Usage and Examples 7.3. Technique Descri bed Nmap version scanning is actually rather straightforward. It was designed to be as simple as possible while still being scalable, fast, and accurate. The truly nitty-gritty details are best discovered by downloading and reviewing the source code, but a synopsis of the techniques used follows. Nmap first does a port scan as per your instructions, and then passes all the open or open I f i l t ered TCP and/or UDP ports to the service scanning module. Those ports are then interrogated in parallel, although a single port is described here for simplicity. I. Nmap checks to see if the port is one of the ports to be excluded, as specified by the E x e l ude directive in nmap- service-probe s . If it is, Nmap will not scan this port for reasons mentioned in Section 7.6, "nmap-service-probes File Format" [ 1 58]. 2. I f the port is TCP, Nmap starts by connecting to it. If the connection succeeds and the port had been in the open I fi l tered state, it is changed to ope n. This is rare (for TCP) since people trying to be so stealthy that they use a TCP scan type which produces open I f i l t ered ports (such as FIN scan) generally know better than to blow all of their stealth by performing version detection. 3. Once the TCP connection is made, Nmap listens for roughly five seconds. Many common services, including most FTP, SSH, SMTP, Tel net, POP3, and IMAP servers, identify themselves in an initial welcome banner. Nmap refers to this as the "NULL probe", because Nmap j ust listens for responses without sending any probe data. If any data is received, Nmap compares it to hundreds of signature regular expressions in its nmap-servi ce-probe s file (described in Section 7.6, "nmap-service-probes File Format" [ 1 58]). If the service is fully identified, we are done with that port! The regular expression includes substrings that can be used to pick version numbers out of the response. In some cases, Nmap gets a "soft match" on the service type, but no version info. In that case, Nmap continues but only sends probes that are known to recognize the soft-matched service type. 4. At this point, Nmap UDP probes start, and TCP connections end up here if the NULL probe above fails or soft-matches. Since the reality is that most ports are used by the service they are registered to in nmap- services, every probe has a list of port numbers that are considered to be most effective. For example, the probe called GetRequest that recognizes web servers (among other services) lists 80-85, 8000-8010, and 8080-8085 as probable ports. Nmap sequentially executes the probe(s) that match the port number being scanned. Each probe includes a probe string (which can be arbitrary ASCII text or \xHH escaped binary), which is sent to the port. Responses that come back are compared to a list of regular expressions of the same type as discussed in the NULL probe description above. As with the NULL probe, these tests can either result in a full match (ends processing for the remote service), a soft match (limits future probes to those which match a certain service), or no match at all. The exact list of regular expressions that Nmap uses to test for a match depends on the probe fallback configuration. For instance, the data returned from the X I I Probe is very unlikely to match any regular expressions crafted for the GetRequest probe. On the other hand, it is likely that results returned from a Probe such as RTSPRequest might match a regular expression crafted for GetRequest since the two protocols being tested for are closely related. So the RTSPRequest probe has a fallback to GetRequest matches. For a more comprehensive explanation, see Section 7.3.1, "Cheats and Fallbacks" [ 1 5 1 ]. 7.3. Technique Described 149 · If any response during version detection is ever received from a UDP port which was in the open I f i 1 tered state, that state is changed to open. This makes version detection an excellent complement to UDP scan, which is forced to label all scanned UDP ports as open I f i 1 tered when some common firewall rules are in effect. While combining UDP scanning with version detection can take many times as long as a plain UDP scan, it is an effective and useful technique. This method is described i n Section 5.4. 1 , "Disambiguating Open from Filtered UDP Ports" [ 1 02]. 5. In most cases, the NULL probe or the probable port probe(s) (there is usually only one) described above matches the service. Since the NULL probe shares its connection with the probable port probe, this allows service detection to be done with only one brief connection in most cases. With UDP only one packet is usually required. But should the NULL probe and probable port probe(s) fail, Nmap goes through all of the existing probes sequentially. In the case of TCP, Nmap must make a new connection for each probe to avoid having previous probes corrupt the results. This worst-case scenario can take a bit of time, especially since Nmap must wait about five seconds for the results from each probe because of slow network connections and otherwise slowly responding services. Fortunately, Nmap utilizes several automatic techniques to speed up scans: • Nmap makes most probes generic enough to match many services. For example, the GenericLines probe sends two blank lines {"\ r \ n \ r \ n") to the service. This matches daemons of many diverse service types, including FfP, ident, POP3, UUCP, Postgres, and whois. The GetRequest probe matches even more service types. Other examples include "he lp \ r \ n" and generic RPC and MS SMB probes. • If a service matches a s o f tma t c h directive, Nmap only needs to try probes that can potentially match that service. • All probes were not created equal ! Some match many more services than others. Because of this, Nmap uses the rarity metric to avoid trying probes that are extremely unlikely to match. Experienced Nmap users can force all probes to be tried regardless or limit probe attempts even further than the default by using the --ver s i o n - i n t e n s i ty, - -ver s i on-a l l , and --vers ion-l i ght options discussed i n Section 7.3.2, "Probe Selection and Rarity" [ 1 52]. 6. One of the probes tests whether the target port is running SSL. If so (and if OpenSSL is available), Nmap connects back via SSL and restarts the service scan to determine what is listening behind the encryption. A special directive allows different probable ports for normal and SSL tunneled connections. For example, Nmap should start against port 443 (HTTPS) with an SSL probe. But after SSL is detected and enabled, Nmap should try the GetRequest probe against port 443 because that port usually has a web server listening behind SSL encryption. 7. Another generic probe identifies RPC-based services. When these are found, the Nmap RPC grinder (discussed later) is initiated to brute force the RPC program number/name and supported version numbers. Similarly, an SMB post-processor for fingerprinting Windows services may be added eventually. 8. If at least one of the probes elicits some sort of response, yet Nmap is unable to recognize the service, the response content is printed to the user in the form of a.fingerprint. If users know what services are actually listening, they are encouraged to submit the fingerprint to Nmap developers for integration into Nmap, as described in Section 7.7. 1 , "Submit Service Fingerprints" [ 1 64]. 1 50 7.3. Technique Described 7.3.1 . Cheats and Fal lbacks Even though Nmap waits a generous amount of time for services to reply, sometimes an application is slow to respond to the NULL probe. This can occur for a number of reasons, including slow reverse DNS lookups performed by some services. Because of this, Nmap can sometimes match the results from a subsequent probe to a match line designed for the NULL probe. For example, suppose we scan port 25 (SMTP) on a server to determine what is listening. As soon as we connect, that service may conduct a bunch of DNS blacklist lookups to determine whether we should be treated as spammers and denied service. Before it finishes that, Nmap gives up waiting for a NULL probe response and sends the next probe with port 25 registered, which is " HELP\ r \ n " . When the service finally completes its anti-spam checks, it prints a greeting banner, reads the Help probe, and responds as shown in Example 7.4. Example 7.4. NULL probe cheat example output 20 hcsw . org E SMTP Sendmail 8 . 12 . 3/8 . 12 . 3/Debian-7 . 1 ; Tue, [cut ] 14-2 . 0 . 0 This is sendmail version 8 . 12 . 3 14-2 . 0 . 0 Topics : 4-2 . 0 . 0 HELO EHLO MAIL RCPT DATA 4-2 . 0 . 0 RSE T NOOP QUIT HELP VRFY AUTH E XPN VE RB ETRN DSN STARTT LS For more info use "HELP " . To report bugs in the implementation send email to 4-2 . 0 . 0 sendmail-bugs@sendmail . org . 4-2 . 0 . 0 For local information send email to Postmaster at your site . 14 2 . 0 . 0 End of HE LP info Nmap reads this data from the socket and finds that no regular expressions from the Help probe match the data returned. This is because Nmap normally expects to receive the ESMTP banner during the NULL probe and match it there. Because this is a relatively common scenario, Nmap "cheats" by trying to match responses to any of the NULL Probe match lines if none of the probe-specific Jines match. In this case, a null match line exists which reports that the program is Sendmail, the version is 8 . 1 2.3/8 . 1 2.3/Debian-7. l , and the hostname is hcsw.org. The NULL probe cheat is actually just a specific example of a more general Nmap feature: fallbacks. The fallback directive is described in detail in Section 7.6, "nmap-service-probes File Format" [ 1 58]. Essentially, any probe that is likely to encounter results that can be matched by regular expressions in other probes has a fallback directive that specifies these other probes. example, in some configurations of the popular Apache web server, Apache won't respond to the GetRequest ("GET I HTTP / 1 . 0 \ r \ n \ r \ n") probe because no virtual host name has been specified. Nmap is still able to correctly identify these servers because those servers usually respond to the HTTPOptions probe That probe has a fallback to the GetRequest regular expressions, which are sufficiently general to recognize Apache's responses to the HTTPOptions probes. For . 7.3. Technique Described 151 7.3.2. Probe Selection and Rarity In determining what probes to use, Nmap considers their r a r i t y . This is an indication of how \ike\-y \\'e probe is to return useful data. If a probe has a high rarity, it is considered less common and is less likely to be tried. Nmap users can specify which probes are tried by changing the intensity level of the version scan, as described below. The precise algorithm Nmap uses when determining which probes to use follows: I . For TCP, the NULL probe is always tried first. 2. All probes that have the port being scanned listed as a probable port (see Section 7.6, "nmap-service-probes File Format" [ 1 58)) are tried in the order they appear in nmap - s e rvi ce-pr obe s . 3. A l l other probes that have a rarity value less than or equal to the current intensity value o f the sca n are tried, also in the order they appear in nmap-servi ce-pr obe s . Once a probe is found to match, the algorithm terminates and results are reported. Because all of Nmap's probes (other than the NULL probe) have a rarity value associated with them, it is relatively easy to control how many of them are tried when performing a version scan. Simply choose an intensity level appropriate for a scan. The higher an intensity level, the more probes will be tried. So if a very comprehensive scan is desired, a high intensity level is appropriate-even though it may take longer than a scan conducted at a lower intensity level. Nmap's default intensity level is 7 but Nmap provides the following switches for different scanning needs: - - ve r s i on - i n t e n s i t y Sets the intensity level of a version scan to the specified value. If 0 is specified, only the NULL probe (for TCP) and probes that list the port as a probable port are tried. Example: nmap -sV --version-intensity 3 scanme.nmap.org - - ve r s i on - i n t e n s i t y Sets the intensity level to 2. Example: nmap -sV --version-light scanme.nmap.org --ver s i o n -a l l Sets the intensity level to 9. Since all probes have a rarity level between probes. Example: nmap -sV --version-all scanme.nmap.org I and 9, this tries all of the 7.4. Tech n iq ue Demonstrated If the English description above i s not clear enough, you can see for yoursel f how it works by adding the --ve r s i o n - t r a c e (and usually -d (debugging)) options to your Nmap command line. This shows all the connection and data read/write activity of the service scan. An annotated real-world example follows. # . nmap -sSV T 4 -F -d --version-trace insecure . erg Starting Nmap ( http : //nmap . org ) Host insecure . erg ( 2 0 5 . 2 1 7 . 1 5 3 . 5 3 ) appears to be up . . . good . - 1 52 7.4. Technique Demonstrated Initiating SYN Stealth Scan against insecure . erg (205 . 2 1 7 . 153 . 53 ) at 19 : 53 Initiating service scan against 4 services on 1 host at 19 : 53 The SYN scan has found 4 open ports-now we are beginning a service scan against each of them in parallel. We start with a TCP connection for the NULL probe: Starting probes (2 . 0750s ) Starting probes NSOCK ( 2 . 0770s ) Starting probes NSOCK ( 2 . 0830s) Starting probes NSOCK ( 2 . 0860s) NSOCK ( 2 . 0870s) NSOCK ( 2 . 0870s) NSOCK ( 2 . 0870s) NSOCK (2 . 0870s) NSOCK ( 2 . 0870s ) NSOCK ( 2 . 0870s) NSOCK ( 2 . 0870s) NSOCK ( 2 . 0870s) NSOCK against new service : 205 . 217 . 153 . 53 : 22 ( tcp) TCP connection requested to 205 . 2 1 7 . 153 . 53 : 22 ( IOD # 1 ) EID against new service : 205 . 2 1 7 . 153 . 53 : 25 ( tcp) TCP connection requested to 2 05 . 217 . 1 53 . 53 : 25 ( IOD #2 ) EID against new service : 2 05 . 2 1 7 . 153 . 53 : 53 ( tcp) TCP connection requested to 2 05 . 2 1 7 . 153 . 53 : 53 ( IOD # 3 ) EID against new service : 205 . 21 7 . 153 . 53 : 80 ( tcp) TCP connection requested to 2 05 . 217 . 153 . 53 : 80 ( IOD # 4 ) EID Callback : CONNECT SUCCESS for EID 32 [205 . 21 7 . 153 . 53 : 80) Read request from IOD # 4 [ 2 05 . 21 7 . 153 . 53 : 80 ) (timeout : 5000ms ) EID 42 Callback : CONNECT SUCCESS for EID 2 4 [205 . 21 7 . 153 . 53 : 53 ) Read request from IOD # 3 [ 2 05 . 2 1 7 . 153 . 53 : 53 ) ( timeout : 5000ms ) EID 50 Callback : CONNECT SUCCESS for EID 16 [205 . 21 7 . 153 . 53 : 25 ) Read request from IOD #2 [ 2 05 . 2 1 7 . 153 . 53 : 25 ) ( timeout : 5000ms ) E I D 5 8 Callback : CONNECT SUCCESS for E I D 8 [ 2 05 . 21 7 . 153 . 53 : 22 ) Read request from IOD #1 [ 2 05 . 2 1 7 . 153 . 53 : 22 ) ( timeout : 5000ms ) EID 66 8 16 24 32 At this point, NULL probe connections have successfully been made to all four services. It starts at 2 seconds because that is how long the ping and SYN scans took. ( 2 . 0880s) Callback : READ SUCCESS for EID 66 [205 . 2 1 7 . 153 . 53 : 2 2 ) ( 23 bytes ) : SSH-l . 99-0penSSH_3 . lpl . Service scan match : 205 . 21 7 . 153 . 53 : 22 is ssh . Version : I OpenSSH l 3 . lpl l protocol 1 . 99 1 NSOCK SSH was nice enough to fully identify itself immediately upon connection as OpenSSH 3. l p l . One down, three to go. (2 . 0880s) Callback : READ SUCCESS for EID 58 [ 205 . 21 7 . 153 . 53 : 25 ) ( 2 7 bytes ) : 2 2 0 core . lnxnet . net ESMTP . . Service scan soft match : 2 05 . 21 7 . 1 53 . 53 : 25 is smtp NSOCK The mail server on port 25 also gave us a useful banner. We do not know what type of mail server it is, but starting with 2 2 0 and including the word E SMTP tells us it is a mail (SMTP) server. So Nmap softmatches smtp, meaning that only probes able to match SMTP servers are tried from now on. Note that non-printable characters are represented by dots-so the " . . " after ESMTP is real ly the "\r \ n " line termination sequence. NSOCK NSOCK NSOCK ( 2 . 0880s) ( 7 . 0880s) ( 7 . 0880s) Read request from IOD #2 [205 . 21 7 . 153 . 53 : 25 ) ( timeout : 4996ms ) E I D 7 4 Callback : READ TIMEOUT for EID 7 4 [205 . 21 7 . 153 . 53 : 25 ) Write request for 6 bytes to IOD #2 E I D 8 3 [ 205 . 21 7 . 153 . 53 : 25 ) : HELP . . 7.4. Technique Demonstrated 153 NSOCK ( 7 . 0880s ) Read request from IOD #2 ( 205 . 217 . 153 . 53 : 25 ] (timeout : 5000ms ) E ID 9 0 Nmap listens a little longer on the SMTP connection, j ust in case the server has more to say. The read request times out after five seconds. Nmap then finds the next probe which is registered to port 25 and has SMTP signatures. That probe simply consists of HELP \ r \ n , which Nmap writes into the connection. NSOCK ( 7 . 0880s ) NSOCK ( 7 . 0880s) NSOCK ( 7 . 0880s ) Callback : READ TIMEOUT for E ID 50 (205 . 21 7 . 153 . 53 : 53] Write request for 32 bytes to IOD #3 E ID 99 (205 . 217 . 153 . 53 : 53 ] : . . . . . . . . . . . . . . . version . bind . . . . Read request from IOD #3 ( 205 . 2 17 . 153 . 53 : 53 ] (timeout : 5000ms ) E ID 106 . The DNS server o n port 5 3 does not return anything at all. The first probe registered to port 5 3 in nmap - servi ce-probes is DNSVersionBindReq, which queries a DNS server for its version number. This is sent onto the wire. NSOCK ( 7 . 0880s ) NSOCK ( 7 . 0880s ) NSOCK ( 7 . 0880s ) Callback : READ TIMEOUT for E ID 42 (205 . 21 7 . 153 . 53 : 80] Write request for 18 bytes to IOD #4 E ID 115 [205 . 217 . 153 . 53 : 80 ] : GE T I HTTP/1 . 0 . . . . Read request from IOD #4 (205 . 2 1 7 . 153 . 53 : 80 ] ( timeout : 5000ms ) E ID 122 The port 80 NULL probe also failed to return any data. A n HTIP GET request i s sent, since that probe is registered to port 80. NSOCK ( 7 . 0920s ) Callback : READ SUCCESS for E ID 122 (205 . 21 7 . 1 53 . 53 : 80 ] [ EOF] ( 1 5858 bytes ) Service scan match : insecure . erg (205 . 2 17 . 153 . 53 ) : 80 is http . Version : ! Apache httpd l 2 . 0 . 39 1 (Unix) mod__perl/1 . 99_07-dev . . Apache returned a huge ( 15KB) response, so it is not printed. That response provided detailed configuration information, which Nmap picks out of the response. There are no other probes registered for port 80. So if this had failed, Nmap would have tried the first TCP probe in nmap-servi ce-probe s . That probe simply sends blank lines ("\ r \ n \r \ n"). A new connection would have been made in case the GET probe confused the service. NSOCK ( 7 . 0920s ) Callback : READ SUCCESS for E ID 106 (205 . 217 . 153 . 53 : 53 ] ( 50 bytes ) : . O . . . . . . . . . version . bind . . . . . . . 9 . 2 . 1 Service scan match : insecure . erg (205 . 21 7 . 153 . 53 ) : 53 i s domain . Version : I ISC BIND l 9 . 2 . l l I Port 53 responded to our DNS version request. Most of the response (as with the probe) is binary, but you can clearly see the version 9.2. 1 there. If this probe had failed, the next probe registered to port 53 is a DNS server status request ( 14 bytes: \ O \ x O C \ 0 \ 0 \ x l 0 \ 0 \ 0 \ 0 \ 0 \ 0 \ 0 \ 0 \ 0 \ 0 ). Having this backup probe helps because many more servers respond to a status request than a version number request. NSOCK ( 7 . 0920s ) Callback : READ SUCCESS for E ID 90 (205 . 2 1 7 . 153 . 53 : 25 ] ( 55 bytes ) : 2 1 4 qmail home page : http . . . 1 54 7.4. Technique Demonstrated Service scan match : insecure . org (205 . 21 7 . 153 . 53 ) : 25 is smtp . Version : l qmai l smtpd l I I Port 25 gives a very helpful response to the Help probe. Other SMTP servers such as Postfix, Courier, and Exim can often be identified by this probe as well. If the response did not match, Nmap would have given up on this service because it had already softmatched smtp and there are no more SMTP probes in nmap-servi ce-probe s . The service scan took 5 seconds to scan 4 services on 1 host . This service scan run went pretty well. No service required more than one connection. It took five seconds because Qmail and Apache hit the five-second NULL probe timeout before Nmap sent the first real probes. Here is the reward for these efforts: Interesting ports on insecure . org (205 . 21 7 . 153 . 53 ) : (The 1212 ports scanned but not shown below are in state : closed) PORT STATE S ERV IC E VERSION OpenSSH 3 . lpl (protocol 1 . 99 ) 22/tcp open ssh 25/tcp open smtp qmail smtpd 53/tcp open domain ISC BIND 9 . 2 . 1 80/tcp open http Apache httpd 2 . 0 . 39 ( (Unix) mod_perl/ 1 . 99_07-dev) Hlnap finished : 1 IP address ( 1 host up) scanned in 7 . 104 seconds 7.5. Post-processors Nmap is usually finished working on a port once it has deduced the service and version information as demonstrated above. However, there are certain services for which Nmap performs additional work. The post-processors presently available are Nmap Scripting Engine integration, RPC grinding, and SSL tunneling. Windows SMB interrogation is under consideration. 7.5.1 . Nmap Scripting Eng i ne I ntegration The regular-expression based approach of version detection i s powerful, but i t cannot recognize everything. Some services cannot be recognized by simply sending a standard probe and matching a pattern to the response. Some services require custom probe strings or a complex multi-step handshaking process. Others require more advanced processing than a regular expression to recognize a response. For example, the Skype v2 service was designed to be difficult to detect due to the risk that i ncumbent carriers (such as phone companies providing DSL lines) would consider them a competitor and degrade or block the service from their subscribers. The only way we could find to detect this service i nvolved analyzing responses to two different probes. Similarly, we could recognize more SNMP services if we tried a few hundred different community names by brute force. Neither of these tasks are well suited to traditional Nmap version detection, but both are easily accomplished with the Nmap Scripting Language. For these reasons, version detection now calls NSE by default to handle some tricky services, as described in Section 9.10, "Version Detection Using NSE" [25 1 ) . 7.5. Post-processors 155 7.5.2. R PC Grind i ng SunRPC (Sun Remote Procedure Call) is a common Unix protocol used to implement many services including NFS. Nmap ships with an nmap-rpc database of almost 600 RPC programs. Many RPC services use high-numbered ports and/or the UDP transport protocol, making them available through many poorly configured firewalls. RPC programs (and the infrastructure libraries themselves) also have a long history of serious remotely exploitable security holes. So network administrators and security auditors often wish to learn more about any RPC programs on their networks. If the portmapper (rpcbind) service (UDP or TCP port 1 1 1 ) is available, RPC services can be enumerated with the Unix rpcinfo command. Example 7.5 demonstrates this against a default Solaris 9 server. Example 7.5. Enumerating RPC services with rpcinfo > rpcinfo -p ultra program vers proto 100000 4 tcp 100000 4 udp 100232 10 udp 1 00083 1 tcp 100221 1 tcp 100068 5 udp 100229 1 tcp 1 00230 1 tcp 1 00242 1 tcp 100001 4 udp 1 00002 3 udp 1 00002 3 tcp 100008 1 udp 100012 1 udp 100011 1 udp 100024 1 udp 100024 1 tcp 100133 1 udp 100133 1 tcp port 111 111 32777 32775 32777 32778 32779 32781 32783 32780 32782 32 785 32784 32786 32788 32790 32787 32790 32787 rpcbind rpcbind sadmind ttdbserverd kcms_server cmsd metad metamhd rpc . metamedd rstatd rusersd rusersd walld sprayd rquotad status status nsm_addrand nsm_addrand [ Dozens of l i n e s c u t for brevi t y ] This example shows that hosts frequently offer many RPC services, which increases the probability that one is exploitable. You should also notice that most of the services are on strange high-numbered ports (which may change for any number of reasons) and split between UDP and TCP transport protocols. Because the RPC information is so sensitive, many administrators try to obscure this information by blocking the portmapper port ( 1 1 1 ). Unfortunately, this does not close the hole. Nmap can determine all of the same information by directly communicating with open RPC ports through the following three-step process. I . The TCP and/or UDP port scan finds all of the open ports. 2. Version detection determines which of the open ports use the SunRPC protocol. 3. The RPC brute force engine determines the program identity of each RPC port by trying a null command against each of the 600 programs numbers in nmap-rpc. Most of the time Nmap guesses wrong and 1 56 7.5 . Post-processors receives an error message stating that the requested program number is not listening on the port. Nmap continues trying each number in its list until success is returned for one of them. Nmap gives up in the unlikely event that it exhausts all of its known program numbers or if the port sends malformed responses that suggest it is not really RPC. The RPC program identification probes are done in parallel, and retransmissions are handled for UDP ports. This feature is automatically activated whenever version detection finds any RPC ports. Or it can be performed without version detection by specifying the - s R option. Example 7.6 demonstrates direct RPC scanning done as part of version detection. Example 7.6. Nmap direct RPC scan nmap -F -A -sSU ultra Starting Nmap ( http : //nmap . org Interesting ports on ultra . nmap . org ( 1 92 . 168 . 0 . 50 ) : (The 2171 ports scanned but not shown below are in state : closed) PORT STAT E S E RVICE VE RSION [A 32776/tcp open kcms_server 1 (rpc #10022 1 ) 32777 /tcp open kcms_server 1 (rpc #10022 1 ) 32777/udp open sadmind 1 0 (rpc #100232) 32778/tcp open me tad 1 ( rpc #100229 ) 32778/udp open cmsd 2-5 ( rpc #10006 8 ) 32779/tcp open me tad 1 (rpc #100229 ) 2-4 (rpc #100001) 32779/udp open rstatd 32780/tcp open metamhd 1 (rpc #100230) 2-4 (rpc #10000 1 ) 32780/udp open rstatd 1 (rpc #100024 J 32786/tcp open status 32786/udp open sprayd 1 (rpc #100012) 32787/tcp open status 1 (rpc #100024) 1 (rpc #100011 ) 32787/udp open rquotad Device type : general purpose Running : Sun Solaris 9 OS details : Sun Solaris 9 Nmap finished : 1 IP address ( 1 host up) scanned in 252 . 701 seconds # wh ole bunch o f por t s c u t 32 7 7 6 /udp open sadmind for brevi ty] 10 ( rpc # 1 0 0 2 3 2 ) 7.5.3. SSL Post-processor Notes As discussed in the technique section, Nmap has the ability to detect the SSL encryption protocol and then launch an encrypted session through which it executes normal version detection. As with the RPC grinder discussed previously, the SSL post-processor is automatically executed whenever an appropriate (SSL) port is detected. This is demonstrated by Example 7.7. 7.5. Post-processors 157 Example 7.7. Version scanning through SSL nmap -PN -sSV -T4 -F www . amazon . com Starting Nmap ( http : //nmap . org ) Interesting ports on 207-171-184-16 . amazon . com (207 . 1 71 . 184 . 16 ) : (The 1214 ports scanned but not shown below are in state : filtered) PORT STATE SERV ICE VERSION 80/tcp open http Apache Stronghold httpd 2 . 4 . 2 (based on Apache 1 . 3 . 6) 443/tcp open ssl/http Apache Stronghold httpd 2 . 4 . 2 (based on Apache 1 . 3 . 6) Nmap finished : 1 IP address (1 host up) scanned in 3 5 . 03 8 seconds Note that the version information is the same for each of the two open ports, but the service is http on port 80 and s s l / h t t p on port 443. The common case of HTTPS on port 443 is not hard-coded-Nmap should be able to detect SSL on any port and determine the underlying protocol for any service that Nmap can detect in clear-text. If Nmap had not detected the server listening behind SSL, the service listed would be s s l / u n k n own. If Nmap had not been built with SSL support, the service listed would have simply been s s 1 . The version column would be blank in both of these cases. The SSL support for Nmap depends on the free OpenSSL library2 . It is not included in the Linux RPM binaries, to avoid breaking systems which lack these libraries. The Nmap source code distribution attempts to detect OpenSSL on a system and link to it when available. See Chapter 2, Obtaining, Compiling, Installing, and Removing Nmap [25] for details on customizing the build process to include or exclude OpenSSL. 7.6. runap-service-probe s Fi le Format As with remote OS detection ( -0), Nmap uses a flat fi le to store the version detection probes and match strings. While the version of nmap - s e r v i c e s distributed with Nmap is sufficient for most users, understanding the file format allows advanced Nmap hackers to add their own services to the detection engine. Like many Unix fi les, nmap-servi ce-probes is line-oriented. Lines starting with a hash (#) are treated as comments and ignored by the parser. Blank lines are ignored as well. Other lines must contain one of the directives described below. Some readers prefer to peek at the examples in Section 7.6.9, "Putting It All Together" [ 1 63] before tackling the following dissection. 7.6.1 . Exclude D irective Syntax: E x c l ude Examples: Exclude 53 , T : 9100, U-30000-40000 This directive excludes the specified ports from the version scan. It can only be used once and should be near the top of the file, above any Probe directives. The Exclude directive uses the same format as the Nmap -p switch, so ranges and comma separated lists of ports are supported. In the nmap - s e rvice-probes included with Nmap the only ports excluded are TCP port 9100 through 9107. These are common ports for 2 http://www.openssl.org 1 58 7.6. nmap-service-probes File Format pinters to listen on and they often print any data sent to them. So a version detection scan can cause them *> print many pages full of probes that Nmap sends, such as SunRPC requests, help statements, and X 1 1 probes. This behavior is often undesirable, especially when a scan is meant to be stealthy. However, Nmap's default behavior of avoiding scanning this port can make it easier for a sneaky user to hide a service: simply run it on an excluded port such as 9100 and it is less likely to be identified by name. The port scan will still show it as open. Users can override the Exclude directive with the - -a l l p o r t s option. This causes version detection to interrogate all open ports. 7.6.2. Probe Di rective Syntax: Probe Examples: obe obe obe TCP GetRequest q l GE T I HTTP/1 . 0\r\n\r\n l UDP DNSStatusRequest q l \0\0\xl0\0\0\0\0\0\0\0\0\0 1 TCP NULL q I I The Probe directive tells Nmap what string to send to recognize various services. All of the directives discussed later operate on the most recent Pr obe statement. The arguments are as follows: This must be either TCP or UDP. Nmap only uses probes that match the protocol of the service it is trying to scan. This is a plain English name for the probe. It is used in service fingerprints to describe which probes elicited responses. Tells Nmap what to send. It must start with a q, then a delimiter character which begins and ends the string. Between the delimiter characters is the string that is actually sent. It is formatted similarly to a C or Perl string in that it allows the following standard escape characters: \ \ \ 0 , \ a , \ b, \ f , \ n , \ r , \ t , v, One Pr obe l i n e in nmap- servi ce-pr obe s has a n empty probe string, a s shown in the third example above. This is the TCP NULL probe which just listens for the initial banners that many services send. If your delimiter character ( I in these examples) is needed for your probe string, you need to choose a different delimiter. \ \xHH. 7.6.3. match Directive Syntax: match [ ] Examples: match match match ftp m/A22 0 . *Welcorne to PureFTPd ( \d\S+) / p/PureFTPd/ v/$1/ ssh m/ASSH- ( [ . \d) + ) -OpenSSH_ ( \S+ ) / p/OpenSSH/ v/$2/ i/protocol $1/ mysql m/A . \0\0\0\n ( 4\ . [ - . \w) + ) \0 . . . \0/s p/MySQL/ i/$1/ 7.6. nmap-service-probes File Format 1 59 match match match match chargen m l @ABCDEFG H I JKLMNOPQRSTUVWXYZ I uucp m l A logi n : Pas sword : Logi n incorrect \ . $ 1 p / SunOS uucpd/ o / SunOS/ printer m l A ( [ \w-_ . ] + ) : lpd : I l legal service requ e s t \ n $ 1 p / lpd/ h / $ 1 / a f s m l A [ \ d \ D ] { 2 8 ) \ s * ( OpenAF S ) ( [ \d \ . ] { 3 ) [ A \ s \ 0 ] * ) \ 0 1 p / $ 1 / v / $ 2 / The match directive tells Nmap how to recognize services based on responses to the string sent by the previous P r obe directive. A single P r obe line may be followed by dozens or hundreds of ma t c h statements. If the given pattern matches, an optional version specifier builds the application name, version number, and additional info for Nmap to report. The arguments to this directive follow: This is simply the service name that the pattern matches. Examples would be s s h, smtp, http, or s nrnp. As a special case, you can prefix the service name with s s l / , as in s s l / vrnware -auth. In that case, the service would be stored as vrnwa r e - a u t h tunneled by SSL. This is useful for services which can be fully recognized without the overhead of making an SSL connection. This pattern is used to determine whether the response received matches the service given in the previous parameter. The format is l ike Perl , with the syntax being rn/ [ regex ] I [ opt s ] . The "m" tells Nmap that a match string is beginning. The forward slash ( / ) is a del imiter, which can be substituted by almost any printable character as long as the second slash is also replaced to match. The regex is a Perl-style regular expression3 . This is made possible by the excellent Perl Compatible Regular Expressions (PCRE) library (http://www.pcre. org). The only options currently supported are 'i', which makes a match case-insensitive and 's' which includes newlines in the '.' specifier. As you might expect, these two options have the same semantics as in Perl. Subexpressions to be captured (such as version numbers) are surrounded by parentheses as shown in most of the examples above. The section actually contains six optional fields. Each field begins with an identifying letter (such as h for "hostname"). Next comes a delimiter character which the signature writer chooses. The preferred delimiter is slash ( ' I ' ) unless that is used in the field itself. Next comes the field value, followed by the delimiter character. The following table describes the six fields: Table 7.1 . ve r s i oninfo field formats and values Field format Value description p / vendorprodu c t n arne / Includes the vendor and often service name and is of the form "Sun Solaris rexecd'', "ISC BIND named", or "Apache httpd". 3 v / ve r s i o n / The application version "number", which may include non-numeric characters and even multiple words. i / info/ Miscellaneous further information which was immediately available and might be useful. Examples include whether an X server is open to unauthenticated connections, or the protocol number of SSH servers. h / ho s t n arne / The hostname (if any) offered up by a service. This is common for protocols such as SMTP and POP3 and is useful because these http:llwww.perl.com/doclmanual/html/podlperlre.html 160 7.6. nmap-service-probes File Format Field format Value description hostnames may be for internal networks or otherwise differ from the straightforward reverse DNS responses. o/oper atingsys tem/ The operating system the service is running on. This may legitimately be different than the OS reported by Nmap IP stack based OS detection. For example, the target IP might be a Linux box which uses network address translation to forward requests to an Microsoft ITS server in the DMZ. In this case, stack OS detection should report the OS as Linux, while service detection reports port 80 as being Windows. d / devicetype / The type of device the service is running on. Some services disclose this information, and it can be inferred in many more cases. For example, the HP-ChaiServer web server only runs on printers. Any of the six fields can be omitted. In fact, all of the fields can be omitted if no further information on the service is available. Any of the version fields can include numbered strings such as $ I or $2, which are replaced (in a Perl-like fashion) with the corresponding parenthesized substring in the . In rare cases, a helperfunction can be applied to the replacement text before insertion. The $ P ( ) helper function will filter out unprintable characters. This is useful for converting Unicode UTF- 16 encoded strings such as W \ 0 0 \ 0 R\ O K \ O G \ O R \ 0 0 \ 0 U \ O P \ 0 into the ASCII approximation WORKGROUP. It can be used in any ver s i o n i n f o field by passing it the number of the match you want to make printable, like this: i I $ P ( 3 ) I . Another helper function i s $ SUBST ( ) . This i s used for making substitutions i n matches before they are printed. It takes three arguments. The first is the substitution number in the pattern, just as you would use in a normal replacement variable such as $ 1 or $ 3 . The second and third arguments specify a substring you wish to find and replace, respectively. All instances of the match string found in the substring are replaced, not just the first one. For example, the VanDyke VShell sshd gives its version number in a format such as 2_2_3_5 7 8 . We use the versioninfo field v I $ SUBST ( 1 , 11_11 , 11 • 11 ) I to convert it to the more conventional form 2 . 2 . 3 . 5 7 8 . 7.6.4. softmat ch Di rective Syntax: s o ftmatch Examples: ftp m/A220 [ - . \w J +ftp . *\r\n$/i smtp m l A220 [ - . \w ) +SMTP . *\r\n l pop3 m l A\+OK [ -\ ( \ ) \ ( \ ) ! , /+ : <>@ . \w ) +\r\n$ 1 The softmatch directive is similar in format to the match directive discussed above. The main difference is that scanning continues after a softmatch, but it is limited to probes that are known to match the given service. This allows for a normal ("hard") match to be found later, which may provide useful version information. See Section 7.3, "Technique Described" [ 1 49] for more details on how this works. Arguments are not defined here because they are the same as for mat ch a bove, except that there is never a argument. Also as with mat ch, many s o ftma t c h statements can exist within a single Pr obe section. 7.6. nmap-service-probes File Format 161 7.6.5. port s and s s lport s Directives Syntax: p o r t s Examples: ports ports 2 1 , 43, 110, 113, 199, 505, 540, 1248, 5432, 30444 1 1 1 , 4 0 4 5 , 3 2 750-32 8 1 0 , 3 8 9 7 8 This line tells Nmap what ports the services identified by this probe are commonly found on. It should only be used once within each Probe section. The syntax is a slightly simplified version of that taken by the Nmap -p option. See the examples above. More details on how this works are in Section 7.3, "Technique Described" [ 1 49) . Syntax: s s lp o r t s Example: sslports 4 43 This is the same as 'port s ' directive described above, except that these ports are often used to wrap a service in SSL. For example, the HTTP probe declares "s s lpor t s 4 4 3" and SMTP-detecting probes have an "s s lport s 4 6 5" line because those are the standard ports for HTTPS and SMTPS respectively. The format is the same as with por t s . This optional directive cannot appear more than once per Pr obe. 7.6.6. totalwaitms Di rective Syntax: t o t a l wai tms Example: totalwaitrns 5000 This rarely necessary directive specifies the amount of time Nmap should wait before giving up on the most recently defined Probe against a particular service. The Nmap default is usually fine. 7.6.7. rarity Di rective Syntax: r a r i t y Example: rarity 6 The rarity directive roughly corresponds to how frequently this probe can be expected to return useful results. The higher the number, the more rare the probe is considered and the less likely it is to be tried against a service. More details can be found in Section 7.3.2, "Probe Selection and Rarity" [ 1 52) . 162 7.6. nmap-service-probes File Format 7.6.8. f allback Syniax: fallback Example: lback GetRequest, GenericLines tbi s optional directive specifies which probes should be used as fall backs for if there are no matches in the current Probe section. For more information on fallbacks see Section 7.3 . 1 , "Cheats and Fallbacks" [ 1 5 1 ] . For TCP probes without a fall back directive, Nmap first tries match lines in the probe itself and then does an implicit fall back to the NULL probe. If the fall back directive is present, Nmap first tries match lines from the probe itself, then those from the probes specified in the fallback directive (from left to right). Finally, Nmap will try the NULL probe. For UDP the behavior is identical except that the NULL probe is never tried. 7.6.9. Putting It All Together Here are some examples from nmap-servi ce-probe s which put this a l l together (to save space many lines have been skipped). After reading this far into the section, the following should be understood. e Exclude directive takes a comma separated list of ports . e format is exactly the same as the -p switch . lude T : 9100-9107 is is the NULL probe that just compares any banners given to us tttt######################NEXT PROBE############################## TCP NULL q 1 1 it for at least 5 seconds for data . Otherwise an Nmap default is used . alwaitms 5000 ndows 2003 h ftp m/A220 [ -] Microsoft FTP Service\r\n/ p/Microsoft ftpd/ ch ftp m/A220 ProFTPD ( \d\S+ ) Server/ p/ProFTPD/ v/$1/ tmatch ftp m/A220 [ - . \w ] +ftp . * \r\n$/i h ident ml Aflock\ ( \ ) on closed filehandle . *midentd l p/midentd/ i/broken/ h imap ml A\* OK Welcome to Binc IMAP v ( \d [- . \w] + ) I p/Binc IMAPd/ v$1/ tmatch imap m/ A\* OK [- . \w ] +imap ( - . \w ] +\r\n$/i ch lucent-fwadm m l A0001 ; 2$ 1 p/Lucent Secure Management Server/ h meetingmaker m/A\xcl, $/ p/Meeting Maker calendaring/ opster 1 . 2 . 0 . 1 on Linux 1 . 1 ch napster ml A1$ 1 p/Lopster Napster P2P client/ q l help\r\n\r\n l 7.6. nmap-service-probes File Format 163 match chargen m l @ABCDEFGHIJKLMNOPQRSTUVWXYZ I match echo m 1 Ahelp\r\n\r\n$ 1 7.7. Com m u n ity Contri butions No matter how technically advanced a service detection framework is, it would be nearly useless without comprehensive database of services against which to match. This is where the open source nature of Nm really shines. The Insecure.Org lab is pretty substantial by geek standards, but it can never hope to run m than a tiny percentage of machine types and services that are out there. Fortunately experience with detection fingerprints has shown that Nmap users together run all of the common stuff, plus a staggerin array of bizarre equipment as well . The Nmap OS fingerprint database contains more than a thousand entries, including all sorts of switches, WAPs, VoIP phones, game consoles, Unix boxes, Windows hosts, printeis, routers, PDAs, firewalls, etc. Version detection also supports user submissions. Nmap users have contributed thousands of services. There are three primary ways that the Nmap community helps to make this an exceptional database: submitting service fingerprints, database corrections, and new probes. 7.7.1 . Submit Service Fingerpri nts If a service responds to one or more of Nmap's probes and yet Nmap is unable to identify that service, Nmap prints a service fingerprint like this one: SF-Port21-TCP : V=3 . 40PVT16%D=9/6%Time=3FSA961C%r (NULL, 3F, "220\x20stage\x20F SF : TP\x20server\x20\ (Version\x202\ . 1WU\ ( 1 \ ) \+SC0-2\ . 6 \ . l\+-sec\) \x20ready\ SF : . \r\n" ) %r (GenericLines, 81, "220\x20stage\x20FTP\x20server\x20\ (Version\x SF : 202\ . 1WU\ ( 1 \ ) \+SC0-2\ . 6 \ . 1\+-sec\) \x20ready\ . \r\n500\x20 ' ' : \x20command\ SF : x20not\x20understood\ . \r\n500\x20 ' ' : \x20command\x20not\x20understood\ . \ SF : r\n" ) ; If you receive such a fingerprint, and are sure you know what daemon version is running on the target host, . please submit the fingerprint at the URL Nmap gives you. The whole submission process is anonymous (unless you choose to provide identifying info) and should not take more than a couple mi nutes. If you are feeling particularly helpful, scan the system again using -d (Nmap sometimes gives longer fingerprints that way) and paste both fingerprints into the fingerprint box on the submission form. Sometimes people read the fi le format section and submit their own working match lines. This is OK, but please submit the service fi ngerprint(s) as well because existing scripts make integrating and testing them relatively easy. For those who care, the information in the fingerpri nt above is port number (21 ), protocol (TCP), Nmap version (3.40PVT 16), date (September 6), Unix time in hex, and a sequence of probe responses in the form r( { } , { } , " { } "). 7. 7.2. Su bmit Database Corrections T.h is is another easy way to help improve the database. When integrating a service fingerprint submitted for "chargen on Windows XP" or "FooBar FTP server 3.9.21 3", it is difficult to determine how general the match is. Will it also match chargen on Solaris or FooBar FTP 2.7? Since there is no good way to tell, a very specific name is used in the hope that people will report when the match needs to be generalized. The only reason the Nmap DB is so comprehensive is that thousands of users have spent a few minutes each to submit 1 64 7.7. Community Contributions new information. If you scan a host and the service fingerprint gives an incorrect OS, version number, application name, or even service type, please let us know as described below: Upgrade lo the latest Nmap (Optional) Many Linux distributions and other operating systems ship with ancient versions of Nmap. The Nmap version detection database is improved with almost every release, so check your version number by running nmap -V and then compare that to the latest available from http://nmap. org/download.html. The problem you are seeing may have already been corrected. Installing the newest version takes only a few minutes on most platforms, and is valuable regardless of whether the version detection flaw you are reporting still exists. But even if you don't have time to upgrade right now, submissions from older releases are still valuable. Be absolutely certain you know what is running Invalid "corrections" can corrupt the version detection DB. If you aren't certain exactly what is running on the remote machine, please find out before submitting. Generate a fingerpri nt Run the command nmap -0 -PN -sSV -T4 -d --version-trace -p , where is the port running the misidentified service on the host. If the service is UDP rather than TCP, substitute - s UV for - s SV. Send us your correction Now simply submit your correction to us at http:!linsecure. org!cgi-bin!submit. cgi?corr-service. Thanks for contributing to the Nmap community and helping to make version detection even better ! 7.7.3. Submit New Probes Suppose Nmap fails to detect a service. If it received a response to any probes at all, it should provide a fingerprint that can be submitted as described above. But what if there is no response and thus a fi ngerprint is not available? Create and submit your own probe ! These are very welcome. The following steps describe the process. Steps for creating a new version detection probe l. Download the latest version of Nmap from http://nmap.org and try again. You would feel a bit silly spending time developing a new probe j ust to find out that it has already been added. Make sure no fingerprint is available, as it is better to recognize services using existing probes if possible than to create too many new ones. If the service does not respond to any of the existing probes, there is no other choice. 2. Decide on a good probe string for recognizing the service. An ideal probe should elicit a response from as many instances of the service as possible, and ideally the responses should be unique enough to differentiate between them. This step is easiest if you understand the protocol very well, so consider reading the relevant RFCs and product documentation. One simple approach is to simply start a client for the given service and watch what initial handshaking is done by sniffing the network with Wireshark or tcpdump, or connecting to a listening Netcat. 3. Once you have decided on the proper string, add the appropriate new Probe l ine to Nmap (see Section 7.3, ''Technique Described" [ 1 49] and Section 7.6, "nmap-service-probes File Format" [ 1 58]). Do not put in any match lines at first, although a por t s directive to make this new test go first against the registered 7.7. Community Contributions 165 ports is OK. Then scan the service with Nmap a few times. You should get a fingerprint back sh · the service's response to your new probe. Send the new probe line and the fingerprints (against diffi machines if possible, but even a few against the same daemon helps to note differences) to Fyodor < f yodor @ i n s ecure . org>. It will likely then be integrated into future versions of Nmap. Any d you can provide on the nature of your probe string is helpful as well. For custom services that only a on your network, it is better to simply add them to your own nrnap-servi ce-pr obe s rather than global Nmap. 7.8. SOLUTION : Find Al l Servers Runn ing a Insecu re or Nonstandard Appl ication Version 7.8.1 . Problem A common task i s scanning a range ofIP addresses to find all servers o f a particular version or even satisfying a particular property. This is something that Nmap's version detection excels in. One of the most popular database application is the open-source MySQL server. MySQL can be configured to disallow all remote logins from untrusted IPs. This is a good security practice when remote logins aren't required. A case in point: in 2005 a MySQL remote code execution vulnerability was discovered and published4 . Fortunately, an attacker must be able to log in first-no doubt saving the Internet from yet a nother devastating worm. In light of problems like this and the fact that SQL logins and passwords are frequently guessable or discoverable through SQL injection attacks, intuition, and inside knowledge of the network, remote logins should be denied when possible. The problem for a network administrator is to discover MySQL servers that needlessly allow logins from untrusted IPs and take appropriate defensive measures. Note This solution was contributed by Nmap developer Doug Hoyte. 7.8.2. Sol ution Nmap's version detection comes i n handy i n this situation because i t adds the word unauthorized to the service detection info line when the server forbids our host any access. If we want to scan the network of 10.0.0.0/24 a simple yet effective strategy is to run the following command from an untrusted source: nmap -sV -p 3306 -oG 10.0.0-mysqls-032506.gnmap 10.0.0.0/24 Next we can use the Unix grep utility to find IPs that accept connections from our IP and don't disallow logins by default (grep's -v switch specifies inverse results and only prints out li nes that don 't match the given pattern): grep 'Ports: 3306/open/tcp//mysql' 10.0.0-mysqls-032506.gnmap I grep 4 -v unauthorized http://www.sec11riryfoc11s.com/bidll2781 166 7.8. SOLUTION: Find All Servers Running an Insecure or Nonstandard Application Version The resulting output shows the MySQL servers that allow remote logins: Host : Host : Host : Host : Host : 10 . 0 . 0 . 33 ( foe . com) Ports : 3306/open/tcp//mysql//MySQL 4 . 1 . 1 1 / 10. 0 . 0 . 72 (bar . com) Ports : 3306/open/tcp//mysql//MySQL 4 . 0 . 2 4-standard/ 10 . 0 . 0 . 99 ( ) Ports : 3306/open/tcp//mysql//MySQL 4 . l . l l-Debian_4sarge2/ 10 . 0 . 0 . 154 ( ) Ports : 3306 /open/tcp//mysql//MySQL 4 . 0 . 25-standard/ 10 . 0 . 0 . 155 ( ) Ports : 3306/open/tcp//mysql//MySQL 4 . 0 . 25-standard/ 7.8.3. Discussion The trick to this i s understanding some MySQL protocol basics and knowing how to read the nmap-service-pr obe s file. Grepping the file for Pr obe and my s q l match lines leads to the following (line wrapped) output: /usr/local/share/nmap/nmap-service-probes I egrep ' A (Probe l match mysql ) ' TCP NULL q I I mysql m/A . \0\0\0\xffj\x04 . *Host . * is not allowed to connect to this MySQL server$/ p/MySQL/ i/unauthorized/ match mysql m l A . \0\0\0\xffj \x04Host hat keine Berechtigung, eine Verbindung zu diesem MySQL Server herzustellen\ . I p/MySQL/ i/unauthorized; German/ m/A . \0\0\0 . . . Al sistema ' [- . \w] + ' non e consentita la connessione a questo server MySQL$/ p/MySQL/ i/unauthorized; Italian/ m l A . \0\0\0\xffi?\x04?Host . * is blocked because of many connection errors\ . I p/MySQL/ i/blocked - too many connection errors/ m/A . \0\0\0 . (3\ . [ - . \w] + ) \0 . * \x08\x02\0\0\0\0\0\0\0\0\0\0\0\0\0\0$/s p/MySQL/ v/$l/ m/A . \0\0\0\n (3\ . [ - . \w] + ) \0 . . . \0/s p/MySQL/ v/$1/ m/A . \0\0\0\n ( 4 \ . [ - . \w] + ) \0 . . . /s p/MySQL/ v/$1/ m ! A . \0\0\0\n (5\ . [ - . \w] + ) \0 . . . \O i s p/MySQL/ v/$1/ tch mysql m l A . \0\0\0\xffj \x04 ' [ \d . J + ' . * MySQL l s p/MySQL/ GenericLines q i \r\n\r\n l GetRequest q l GE T / HTTP/ 1 . 0\r\n\r\n l HTTPOptions q ! OPTIONS / HTTP/ 1 . 0\r\n\r\n l $ cat Probe match We see that the my sql match lines are designed to be triggered by the NULL probe so no custom probes ll'C needed to determine which servers allow remote logins (for that see Section 7.9, "SOLUTION: Hack Version Detection to Suit Custom Needs, such as Open Proxy Detection" [ 1 68)). By looking at these my s q l match lines that we discover MySQL services that don't allow remote logins will result in an i nfo field taining the word unauthor i z ed . addition to service types and version numbers, there are many cases where version detection is able to Jaalier useful information on scan targets. The probes file is full of such gems that can turn a time-consuming k of protocol research, script coding, locating test servers, and debugging into a simple Nmap command. few interesting tidbits of information that version detection can sometimes reveal are: .ta Whether a CVS pserver is properly configured 7.8. SOLUTION: Find All Servers Running an Insecure or Nonstandard Application Version 167 • The usernames used by popular peer-to-peer fi le sharing clients • Whether an X server is accepting connections • Language and other localization parameters of many services • The wordsize of the target's CPU • The configured botnames of popular IRC bots such as eggdrop • Whether posting is allowed on Internet news (NNTP) servers The version detection database is constantly growing and being refined thanks to the amazing Nmap user community and their service fingerprint submissions. This solution is a good example of how investigating the capabilities of Nmap's service detection can provide elegant, sometimes non-obvious solutions to many diverse problems. 7.9. SOLUTION : Hack Version Detection to Su it Custom Needs, such as Open Proxy Detection 7.9.1 . Problem A n important part o f securing any network i s identifying dangerous hosts. Nmap's service detection system is a flexible, reliable way to do this. It can help identify vulnerable versions of software, find misconfigured servers, and more. But sometimes actually trying to misuse services in ways the stock version scan doesn't dare to is the best way to determine if they are actually vulnerable. Open proxies are servers that will blindly relay requests from untrusted hosts to servers of their choosing. Running these inside a network can be extremely dangerous for many reasons, as attackers can: • Launch attacks that appear to come from your network • Steal bandwidth or other network services from you • Pretend to be an internal client to further escalate their privileges inside your organization This provides good motivation for hacking version detection to specifically try to exploit open proxies. We could probably map out which ports are proxies by using Nmap's normal proxy match li nes, but the best, and only real way to prove an application is vulnerable is to actually exploit it yourself. Note This solution was contributed by Nmap developer Doug Hoyte. 168 7.9. SOLUTION: Hack Version Detection to Suit Custom Needs, such as Open Proxy Detection 7.9.2. Solution The first thing we d o i s copy the nmap - s e r v i ce-pr obe s file so we can work o n a temporary copy: mkdir - /proxydet e c t cp /us r / l oca l / shar e / nmap/ nmap-service-probes - /proxydet e c t Next we want to temporarily force Nmap to use our temporary file: export NMAPD I R= $HOME /proxyde t e c t Now we need to add a probe and match I ine to the file, so open up your favorite editor and place the following text into your copy of nmap - s e r v i ce-probe s . A good place to put it is after all the match lines in the NULL probe, but i m mediately before the next Probe line (GenericLines). Probe TCP ProxyProbe q l GE T h t tp : / / i n s ecure . org/ HTTP/ 1 . 1 \ r \ nH o s t : insecure � . org\r \n\r\n l rar ity 1 ports 1 - 6 5 5 3 5 totalwa i tms 2 0 0 0 0 match proxy m l A HTTP / l . [ 0 1 ] 2 0 0 OK\ r ? \n . * T I TLE> I nsecure . O l s p / Open HTTP P roxy ! ! / Now Nmap will actually try to request an HTTP download from i n secure . o r g by treating any scanned ports as proxies. We will start to see the following in scans of networks containing open proxies: PORT STATE SERVICE VERS I ON 8 0 /tcp open proxy Open HTTP Proxy ! ! 7.9.3. Discussion The placement of our probe, the low r a r i t y value, and extensive por t s range help ensure that our custom probe is tried very soon into the service scan so that other probes like GetRequest don't simply identify this as a proxy before we've had a chance to use our active probe. We also used a t o t a l wai tms directive to make Nmap wait longer for this probe to time out. This can be necessary because not only are we dealing with the latency and unreliability of the connection between us and the proxy, but also the latency and unreliability of the connection between the proxy and the server containing the page we requested ( i n secure . o r g). Keep in mind that many other protocols can be proxied in addition to HTTP. Version detection will identify proxies for many of them including FTP, .POP3, IMAP, and SMTP. SOCKS proxies have special match lines that determine information on the authentication options the proxy has configured. As we did in this solution, often we can use version detection to tell whether such proxies are open or not by using custom probes files. However, m ore complicated tests are probably best done with NSE scripts. 7.9. SOLUTION: Hack Version Detection to Suit Custom Needs, such as Open Proxy Detection 169 Chapter 8. Remote OS Detection 8.1 . Introd uction When exploring a network for security auditing or inventory/administration, you usually want to know more lban the bare IP addresses of identified machines. Your reaction to discovering a printer may be very different than to finding a router, wireless access point, telephone PBX, game console, Windows desktop, or Unix server. Finer grained detection (such as distinguishing Mac OS X 1 0.4 from 1 0.3) is useful for determining vulnerability to specific flaws and for tailoring effective exploits for those vulnerabilities. In part due to its value to attackers, many systems are tight-lipped about their exact nature and operating system configuration. Fortunately, Nmap includes a huge database of heuristics for identifying thousands of different systems based on how they respond to a selection of TCP/IP probes. Another system (part of version detection) interrogates open TCP or UDP ports to determine device type and OS details. Results of these two systems are reported independently so that you can identify combinations such as a Checkpoint firewall awarding port 80 to a Windows IIS server. While Nmap has supported OS detection since 1998, this chapter describes the 2nd generation system released in 2006. 8.1 .1 . Reasons for OS Detection While some benefits of discovering the underlying OS and device types o n a network are obvious, others are more obscure. This section lists the top reasons I hear for discovering this extra information. Determining vu lnerability of target hosts It is sometimes very difficult to determine remotely whether an available service is susceptible or patched for a certain vulnerability. Even obtaining the application version number doesn't always help, since OS distributors often back-port security fixes without changing the version number. The surest way to verify that a vulnerability is real is to exploit it, but that risks crashing the service and can lead to wasted hours or even days of frustrating exploitation efforts if the service turns out to be patched. OS detection can help reduce these false positives. For example, the Rwho daemon on unpatched Sun Solaris 7 through 9 may be remotely exploitable (Sun alert #57659). Remotely determining vulnerability is difficult, but you can rule it out by finding that a target system is running Solaris 1 0. Taking this from the perspective of a systems administrator rather than a pen-tester, imagine you run a large Sun shop when alert #57659 comes out. Scan your whole network with OS detection to find machines which need patching before the bad guys do. Tailoring exploits Even after you discover a vulnerability in a target system, OS detection can be helpful in exploiting it. Buffer overflows, format-string exploits, and many other vulnerabilities often require custom-tailored shellcode with offsets and assembly payloads generated to match the target OS and hardware architecture. In some 8. l . Introduction 171 cases, you only get one try because the service crashes if you get the shellcode wrong. Use OS detection first or you may end up sending Linux shellcode to a FreeBSD server. Network i nventory and support While it isn't as exciting as busting root through a specially crafted format string exploit, there are many administrative reasons to keep track of what is running on your network. Before you renew that IRIX support contract for another year, scan to see if anyone still uses such machines. An inventory can also be useful for IT budgeting and ensuring that all company equipment is accounted for. Detecting unauthorized and dan gerous devices With the ubiquity of mobile devices and cheap commodity networking equipment, companies are increasingly fi nding that employees are extending their networks in undesirable ways. They may install a $20 wireless access point (WAP) in their cubicle without realizing (or caring) that they just opened up the protected corporate network to potential attackers in the parking lot or nearby buildings. WAPs can be so dangerous that Nmap has a special category for detecting them, as demonstrated in Section 8.8, "SOLUTION: Detect Rogue Wireless Access Points on an Enterprise Network" [202]. Users may also cause sysadmins grief by connecting insecure and/or worm-infected laptops to the corporate network. Regular scanning can detect unauthorized devices for investigation and containment. Social engi neering Another possible use is social engineering. Lets say that you are scanning a target company and Nmap reports a "Datavoice TxPORT PRISM 3000 T l CSU/DSU 6.22/2.06". You could call up the target pretending to be Datavoice support and discuss some issues with their PRISM 3000. Tel l them you are about to announce a big security hole, but are first providing the patch to valued customers. Some naive administrators migbl assume that only an authorized engineer from Datavoice would know so much about their CSU/DSU. course the patch you send them is a Trojan horse that gives you remote access to sniff and traipse through their network. Be sure to read the rest of this chapter for detection accuracy and verification advice befi trying this. If you guess the target system wrong and they call the police, that will be an embarrassing st to tell your cellmates. 8.2. Usage and Examples The inner workings of OS detection are quite complex, but it is one of the easiest features to use. Sim add -0 to your scan options. You may want to also increase the verbosity with -v for even more OS-rela details. This is shown in Example 8.1 . 172 8.2. Usage and Examples pie 8.1. OS detection with verbosity (-0 -v) scanme . nmap . org rting Nmap ( http : //nmap . org eresting ports on scanme . nmap . org (64 . 13 . 134 . 52 ) : shown : 994 filtered ports T STATE S ERVIC E open ssh closed smtp open domain closed gopher open http tcp closed auth ce type : general purpose ing: Linux 2 . 6 . X etails : Linux 2 . 6 . 20-1 (Fedora Core 5 ) e guess : 11 . 433 days (since Thu Sep 18 13 : 13 : 01 2008) Sequence Prediction : Difficulty=204 (Good luck ! ) ID Sequence Generation : All zeros 1 IP address ( 1 host up) scanned in 6 . 2 1 seconds Raw packets sent : 2021 ( 9 0 . 526KB) I Rcvd : 23 ( 13268) p -0 -v - v options caused Nmap to generate the following six extra line items: ice type All fingerprints are classified with one or more high-level device types, such as router, pr i nt e r , fi r a ll , or (as in this case) g e n e r a l purpo s e . These are further described in the section called "Device and OS classification (Class lines)" [ 1 96) . Several device types may be shown, in which case they will be separated with the pipe symbol as in "Dev i ce Type : r outer I f i rewa l l". ew ning This field is also related to the OS classification scheme described in the section called "Device and OS classification (Class lines)" [ 1 96) . It shows the OS Family (Li nux i n this case) and OS generation (2 . 6 . X) if available. If there are multiple OS families, they are separated by commas. When Nmap can't narrow down OS generations to one specific choice, options are separated by the pipe symbol (' I ') Examples include OpenBSD 3 . X , N e t B S D 3 . X I 4 . X and L i nux 2 . 4 . X I 2 . 5 . X I 2 . 6 . X . If Nmap finds too many O S families to print concisely, it w i l l omit this line. When there are n o perfect matches, Nmap changes the field to Runn i ng ( JU S T GUE S S I NG ) and adds an accuracy percentage ( 100% is a perfect match) in parentheses after each candidate family name. If no fingerprints are close matches, the line is omitted. This line gives the detailed description for each fingerprint that matches. While the Devi ce t ype and Running lines are from predefined enumerated lists that are easy to parse by a computer, the OS details line contains free-form data which is useful to a human reading the report. This can include more exact version numbers, device models, and architectures specific to a given fingerprint. In this example, theonly matchi ng fingerprint was L i nux 2 . 6 . 2 0 - 1 ( Fedora Core 5 ) . When there are multiple exact matches, they are comma-separated. If there aren't any perfect matches, but some close guesses, 8.2. Usage and Examples 173 the field is renamed Aggr e s s ive OS gue s s e s and fingerprints are shown followed by a percentage in parentheses which specifies how close each match was. Uptime guess As part of OS detection, Nmap receives several SYN/ACK TCP packets in a row and checks the headers for a timestamp option. Many operating systems use a simple counter for this which starts at zero at boot time then i ncrements at a constant rate such as twice per second. By looking at several responses, Nmap can determine the current values and rate of increase. Simple linear extrapolation determines boot time. The timestamp algorithm is used for OS detection too (see the section called "TCP timestamp option algorithm (TS)" [ 1 82]) since the increment rate on different systems varies from 2 Hz to l ,OOO Hz. The uptime guess is labeled a "guess" because various factors can make it completely inaccurate. Some operating systems do not start the timestamp counter at zero, but initialize it with a random value, making extrapolation to zero meaningless. Even on systems using a simple counter starting at zero, the counter eventually overflows and wraps around. With a 1 ,000 Hz counter increment rate, the counter resets to zero roughly every 50 days. So a host that has been up for 102 days will appear to have been up only two days. Even with these caveats, the uptime guess is accurate much of the time for most operating systems, so it is printed when available, but only in verbose mode. The uptime guess is omitted if the target gives zeros or no timestamp options in its SYN/ACK packets, or if it does not reply at all. The line is also omitted i f Nmap cannot discern the timestamp increment rate or it seems suspicious (like a 30-year uptime). Network Distance A side effect of one of the OS detection tests allows Nmap to compute how many routers are between it and a target host. The distance is zero when you are scanning localhost, and one for a machine on the same network segment. Each additional router on the path adds one to the hop count. The Network D i s t an c e line is not printed in this example, since Nmap omits the line when it cannot be computed (no reply to the relevant probe). TCP Sequence Prediction Systems with poor TCP initial sequence number generation are vulnerable to blind TCP spoofing attacks. In other words, you can make a full connection to those systems and send (but not receive) data while spoofing a different IP address. The target's logs will show the spoofed IP, and you can take advantage of any trust relationship between them. This attack was all the rage in the mid-nineties when people commonly used rlogin to allow logins to their account without any password from trusted IP addresses. Kevin Mitnick is alleged to have used this attack to break into Tsutomu Shimomura's computers in December 1994. The good news is that hardly anyone uses rlogin anymore, and many operating systems have been fixed to use unpredictable initial sequence numbers as proposed by RFC 1948. For these reasons, this line is only printed in verbose mode. Sadly, many vendors still ship vul nerable operating systems and devices 1 • Even the fixed ones often vary in implementation, which leaves them valuable for OS detection purposes. The class describes the ISN generation algorithm used by the target, and difficulty is a rough estimate of how hard the system makes blind IP spoofing (0 is the easiest). The parenthesized comment is based on the difficulty index and ranges from T r i v i a l j oke to E a s y, Medi um, F o rmidable, Worthy cha l l e nge, and finally Good luck ! Further details about sequence tests are provided in the section called "TCP ISN greatest common divisor (GCD)" [ 1 80] . 1 A fascinating visual look at this is available from http:/llcamtttf.coredttmp.cx/newtcpl 174 8.2. Usage and Examples While the rlogin family is mostly a relic of the past, clever attackers can still find effective uses for blind TCP spoofing. For example, it al lows for spoofed HTTP requests. You don't see the results, but just the URL (POST or GET request) can have dramatic side effects. The spoofing allows attackers to hide their identity, frame someone else, or exploit IP address restrictions. IP ID sequence generation Many systems unwittingly give away sensitive information about their traffic levels based on how they generate the lowly 16-bit ID field in IP packets. This can be abused to spoof a port scan against other systems and for other mischievous purposes discussed in Section 5. 10, "TCP Idle Scan (-sI)" [ 1 1 7 ] . This field describes the ID generation algorithm that Nmap was able to discern. More information on how it classifies them is available in the section called "TCP IP ID sequence generation algorithm (Tl)" [ 1 8 1 ] . Note that many systems use a different IP ID space for each host they communicate with. In that case, they may appear vul nerable (such as showing the I n cr eme n t a l class) while still being secure against attacks such as the idle scan. For this reason, and because the issue is rarely critical, the IP ID sequence generation line is only printed in verbose mode. If Nmap does not receive sufficient responses during OS detection, it will omit the whole line. The best way to test whether a host is vulnerable to being an idle scan zombie is to test it with s I . - While TCP fingerprinting is a powerful method for OS detection, interrogating open ports for clues i s another effective approach. Some applications, such as Microsoft IIS, only run on a single platform (thus giving it away), while many other apps divulge their platform in overly verbose banner messages. Adding the s v option enables Nmap version detection, which is trained to look for these clues (among others). In Example 8.2, Nmap catches the platform details from an FfP server. - Example 8.2. Using version scan to detect the OS ' nmap -sv -0 -v 129 . 128 . X . XX ftarting Nmap ( http : / / nmap . org ) (nteresting ports on [hostname ) ( 129 . 12 8 . X . XX ) : t shown : 994 closed ports STATE S E RV ICE P<)RT VE RSION 1/tcp open ftp HP-UX 1 0 . x ftpd 4 . 1 /tcp open ssh OpenSSH 3 . 7 . lpl (protocol 1 . 99 ) 11/tcp open rpc 5/tcp filtered microsoft-ds oracle-tns Oracle TNS Listener 26/tcp open rpc 775/tcp open exact OS matches for host P Sequence Prediction : Class=truly random Difficulty=9999999 ( Good luck ! ) P ID Sequence Generation : Incremental rvice Info : OS : HP-UX In this example, the line "No exact OS mat ches for host" means that TCP/IP fingerprinting failed to find an exact match. Fortunately, the Servi ce I n f o field a few lines down discloses that the OS is HP-UX. If several operating systems were detected (which can happen with NAT gateway boxes that redirect ports to several different machines), the field would be OS s and the values would be comma separated. The Service I n f o line can also contain hostnames and device types found during the version scan. The focus of this chapter is on TCP/IP fi ngerprinting though, since version detection was covered in Chapter 7, Service and Application Version Detection [ 1 45]. 8.2. Usage and Examples 175 With two effective OS detection methods available, which one should you use? The best answer is usually both. In some cases, such as a proxy firewall forwarding to an application on another host, the answers may legitimately differ. TCP/IP fingerprinting will identify the proxy while version scanning will generally detect the server running the proxied application. Even when no proxying or port forwarding is involved, using both techniques is beneficial. If they come out the same, that makes the results more credible. If they come out wildly different, investigate further to determine what is going on before relying on either. Since OS and version detection go together so well, the -A option enables them both. OS detection is far more effective if at least one open and one closed TCP port are found. Set the - - o s s ca n - l imit option and Nmap will not even try OS detection against hosts which do not meet this criteria. This can save substantial time, particularly on -PN scans against many hosts. You still need to enable OS detection with -0 (or -A) for this to have any effect. Another OS detection option is - - o s s c an -gue s s . When Nmap is unable to detect a perfect OS match, it sometimes offers up near-matches as possibilities. The match has to be very close for Nmap to do this by default. If you specify this option (or the equivalent - - f u z z y option), Nmap will guess more aggressively. Nmap still tells you when an imperfect match is printed and display its confidence level (percentage) for each guess. When Nmap performs OS detection against a target and fails to find a perfect match, it usually repeats the attempt. By default, Nmap tries five times if conditions are favorable for OS fingerprint submission, and twice when conditions aren't so good. The - -max- o s - t r i e s option lets you change this maximum number of OS detection tries. Lowering it (usually to l ) speeds Nmap up, though you miss out on retries which could potentially identify the OS. Alternatively, a high value may be set to allow even more retries when conditions are favorable. This is rarely done, except to generate better fingerprints for submission and integration into the Nmap OS database. · Like j ust about every other part of Nmap, results ultimately come from the target machine itself. While rare, systems are occasionally configured to confuse or mislead Nmap. Several programs have even been developed specifically to trick Nmap OS detection (see Section 1 1 .5.4, "OS Spoofing" [302)). Your best bet is to use numerous reconnaissance methods to explore a network, and don't trust any one of them. TCP/IP fingerprinting requires collecting detailed information about the target's IP stack. The most commonly useful results, such as TTL information, are printed to Nmap output whenever they are obtained. Slightly less pertinent information, such as IP ID sequence generation and TCP sequence prediction difficulty, is only printed in verbose mode. But if you want all of the IP stack details that Nmap collected, you can find it in a compact form called a subjectfingerprint. Nmap sometimes prints this (for user submission purposes) when it doesn't recognize a host. You can also force Nmap to print it (in normal, interactive, and XML formats) by enabling debugging with (-d). Then read Section 8.5, "Understanding an Nmap Fingerprint" [ 1 9 1 ] t o interpret it. 8.3. TCP/I P Fingerpri nti ng Methods Su pported by Nmap Nmap OS fingerprinting works by sending up to 1 5 TCP, UDP, and ICMP probes to known open and closed ports of the target machine. These probes are specially designed to exploit various ambiguities in the standard protocol RFCs. Then Nmap listens for responses. Dozens of attributes in those responses are analyzed and 176 8.3. TCP/IP Fingerprinting Methods Supported by Nmap combined to generate a fingerprint. Every probe packet is tracked and resent at least once if there is no response. All of the packets are IPv4 with a random IP ID value. Probes to an open TCP port are skipped if no such port has been found. For closed TCP or UDP ports, Nmap will first check if such a port has been found. If not, Nmap will just pick a port at random and hope for the best. The following sections are highly technical and reveal the hidden workings of Nmap OS detection. Nmap can be used effectively without understanding this, though the material can help you better understand remote networks and also detect and explain certain anomalies. Plus, some of the techniques are pretty cool. Readers in a hurry may skip to Section 8.7, "Dealing with Misidentified and Unidentified Hosts" ( 1 99) . But for those of you who are ready for a journey through TCP explicit congestfon notification, reserved UDP header bits, initial sequence numbers, bogus flags, and Christmas tree packets: read on! Even the best of us occasionally forget byte offsets for packet header fields and flags. For quick reference, the 1Pv4, TCP, UDP, and ICMP header layouts can be found in Section 7, "TCP/IP Reference" [xxvi] . The layout for ICMP echo request and destination unreachable packets are shown in Figure 8.1 and Figure 8.2. Figure 8.1. ICMP echo request or reply header layout Offset o 3 2 ��-�_._._,__.__.-'l ���__.__._.___.__. ���(�o ��-�-1-'-�C �-(� 0 ,_._ T� h� C_ yp d e O) e O or 8 ) ecks u m -r 1.. d�efi_ ti� b. c. r..,__.._._..,_..._.s�e-q�u.e� n. r ._...... e� n� e_N e� ._._..,._._.. �u .m _... 1 3 Data : Echo reply (type 0 ) m ust return any data sent i n echo request Figure 8.2. ICMP destination unreachable header layout Offset o 2 3 l'-''--'-'- ---'--'--'----'--+--'--'--.L.-L-'---'�___._...__._ ._ _,__.__.____,__._-L-_,__.__..__..__.___..___.---'I u n u se d ( m u st b e O ) �'"""'"'_,...........,. ..,. .... . _,_ .., �� ���_..,����-p�_..._,...._....-io-� .. � 1 2 3 _i Data : Original ( received) I P h eader, p l u s at least the first 8 data bytes 8.3.1 . Probes Sent This section describes each IP probe sent by Nmap as part of TCP/IP fingerprinting. It refers to Nmap response tests and TCP options which are explained in the following section. Sequence generation (SEQ, OP S, WIN, and Tl) A series of six TCP probes i s sent to generate these four test response lines. The probes are sent exactly 1 10 milliseconds apart so the total time taken is 550 ms. Exact timing is important as some of the sequence algorithms we detect (initial sequence numbers, IP IDs, and TCP timestamps) are time dependent. This 8.3. TCP/IP Fingerprinting Methods Supported by Nmap 177 timing value was chosen to take above 500 ms so that we can reliably detect the common 2 Hz TCP timestamp sequences. Each probe is a TCP SYN packet to a detected open port on the remote machine. The sequence and acknowledgment numbers are random (but saved so Nmap can differentiate responses). Detection accuracy requires probe consistency, so there is no data payload even if the user requested one with - -dat a- length. d These packets vary in the TCP options they use and the TCP window field value. The following list provi es EOL scaling. window reflect not do values field window listed The the options and values for all six packets. is the end-of-option s-list option, which many sniffing tools don't show by default. Packet # 1 : window scale ( 10), NOP, MSS ( 1460), timestamp (TSval: OxFFFFFFFF; TSecr: 0), SACK permitted. The window field is I . • Packet #2: MSS ( 1400), window scale (0), SACK permitted, timestamp (TSval: OxFFFFFFFF; TSecr: 0), EOL. The window field is 63. • Packet #3: Timestamp (TSval : OxFFFFFFFF; TSecr: 0), NOP, NOP, window scale (5), NOP, MSS (640). The window field is 4. • • Packet #4 : SACK permitted, Timestamp (TSval : OxFFFFFFFF; TSecr: 0), window scale ( IO), EOL. The window field is 4. • • FF F; TSecr: 0), window scale Packet #5: MSS (536), SACK permitted, Timestamp (TSval: OxFFFFF ( 10), EOL. The window field is 1 6. Packet #6: MSS (265), SACK permitted, Timestamp (TSval: OxFFFFFFFF; TSecr: 0). The window field is 5 1 2. The results of these tests include four result category l ines. The first, S EQ, contains results based on sequence analysis of the probe packets. These test results are GCD, SP, I SR, T I , I I , T S , and S S . The next line, OPS contains the TCP options received for each of the probes (the test names are 01 through 0 6 ). Similarly, the W I N line contains window sizes for the probe responses (named W l through W 6 ). The final line related to these probes, T l , contains various test values for packet # I . Those results are for the R, OF, T, TG, W, S, A, F , O, RD, and Q tests. These tests are only reported for the first probe since they are almost always the same for each probe. ICMP echo (IE) The IE test involves sending two ICMP echo request packets to the target. The first one has the IP DF bit set, a type-of-service (TOS) byte value of zero, a code of nine (even though it should be zero), the sequence number 295, a random IP ID and ICMP request identifier, and a random character repeated 1 20 times for the data payload. The second ping query is similar, except a TOS of four ( I P_TOS_RE L I AB I L I TY) is used, the code is zero, 1 50 bytes of data is sent, and the IP ID, request ID, and sequence numbers are incremented by one from the previous query values. The results of both of these probes are combined into a IE line containing the R, DF I , T, TG, TOSI, CD, S I , and D L I tests. The R value is only true (Y) if both probes elicit responses. The T, and CD values are for 1 78 8.3. TCP/IP Fingerprinting Methods Supported by Nmap the response to the first probe only, since they are highly unlikely to differ. The DF I , TOS I , S I , and D L I are custom tests for this special dual-probe ICMP case. These ICMP probes follow immediately after the TCP sequence probes to ensure valid results of the shared IP ID sequence number test (see the section called "Shared IP ID sequence Boolean (SS)" [ 1 82]). TCP explicit congestion notification (ECN) This probe tests for explicit congestion notification (ECN) support in the target TCP stack. ECN is a method for improving Internet performance by allowing routers to signal congestion problems before they start having to drop packets. It is documented in RFC 3168. Nmap tests this by sending a SYN packet which also has the ECN CWR and ECE congestion control flags set. For an unrelated (to ECN) test, the urgent field value of OxF7F5 is used even though the urgent flag is not set. The acknowledgment number is zero, sequence number is random, window size field is three, and the reserved bit which immediately precedes the CWR bit is set. TCP options are WScale ( 10), NOP, MSS ( 1460), SACK permitted, NOP, NOP. The probe is sent to an open port. lf a response is received, the R, DF, T, TG, W, o, cc, and Q tests are performed and recorded. TCP (T2-T7) The six T2 through T 7 tests each send one TCP probe packet. With o n e exception, the TCP options data in each case is (in hex) 0 3 0 3 OAO 1 0 2 0 4 0 1 0 9 0 8 OAFFFFFFFF 0 0 0 0 0 0 0 0 0 4 0 2 . Those 20 bytes correspond to window scale ( 10), NOP, MSS (265), Timestamp (TSval: OxFFFFFFFF; TSecr: 0), then SACK permitted. The exception is that T 7 uses a Window scale value of 1 5 rather than 1 0. The variable characteristics of each probe are described below: • T2 sends a TCP • T3 sends a TCP • T4 sends a TCP ACK packet with IP DF and a window field of 1 024 to an open port. • TS • T6 sends • null (no flags set) packet with the IP DF bit set and a window field of 1 28 to an open port. packet with the SYN, FIN, URG, and PSH flags set and a window field of 256 to an open port. The IP OF bit is not set. sends a TCP SYN packet without IP OF and a window field of 31 337 to a closed port. a TCP ACK packet with IP DF and a window field of 32768 to a closed port. sends a TCP packet with the FIN, PSH, and URG flags set and a window field of 65535 to a closed port. The IP OF bit is not set. 1i In each of these cases, a line is added to the fi ngerprint with results for the R, DF, T, TG, w, S, A, F, o, RD, llld Q tests. This probe is a UDP packet sent to a closed port. The character 'C' (Ox43) is repeated 300 times for the data Acid. The IP ID value is set to Ox 1 042 for operati ng systems which allow us to set this. If the port is truly dosed and there is no firewall in place, Nmap expects to receive an ICMP port unreachable message in 8.3. TCP/IP Fingerprinting Methods Supported by Nmap 179 return. That response is then subjected to the R, DF, T, TG, TOS, I PL, UN, RI PL, R I D, R I PCK, RUCK, RUL, and RUD tests. 8.3.2. Response Tests The previous section describes probes sent by Nmap, and this one completes the puzzle by describing the barrage of tests performed on responses. The short names (such as DF, R, and R I PCK) are those used in the nmap-os -db fingerprint database to save space. All numerical test values are given in hexadecimal notation, without leading zeros, unless noted otherwise. The tests are documented in roughly the order they appear in fi ngerprints. TCP ISN greatest com mon d ivisor (Geo) The SEQ test sends six TCP SYN packets to an open port of the target machine and collects SYN/ACK packets back. Each of these SYN/ACK packets contains a 32-bit initial sequence number (ISN). This test attempts to determine the smallest number by which the target host increments these values. For example, many hosts (especially old ones) always increment the ISN in multiples of 64,000. The first step in calculating this is creating an array of differences between probe responses. The first element is the difference between the 1 st and 2nd probe response ISNs. The second element is the difference between the 2nd and 3rd responses. There are five elements if Nmap receives responses to all six probes. Since the next couple of sections reference this array, we wil l call it di f f 1 . If an ISN is lower than the previous one, Nmap looks at both the number of values i t would have to subtract from the first value to obtain the second, and the number of values it would have to count up (including wrapping the 32-bit counter back to zero). The smaller of those two values is stored in di f f l . So the difference between Ox20000 followed by Ox 15000 is OxBOOO. The difference between OxFFFFFFOO and OxCOOO is OxCOFF. This test value then records the greatest common divisor of all those elements. This GCD is also used for calculating the SP result. . TCP ISN cou nter rate (I SR) This value reports the average rate of increase for the returned TCP initial sequence number. Recall that a difference is taken between each two consecutive probe responses and stored in the previously discussed d i f f l array. Those differences are each divided by the amount of time elapsed (in seconds-will generally be about 0.1 ) between sending the two probes which generated them. The result is an array, which we'll call s e q_r a t e s containing the rates of ISN counter increases per second. The array has one element for each d i f f l value. An average is taken of the array values. If that average is less than one (e.g. a constant ISN is used), I SR is zero. Otherwise I SR is eight times the binary logarithm (log base-2) of that average value, rounded to the nearest i nteger. TC P ISN seq uence pred ictability index (SP) While the I S R test measures the average rate o f initial sequence number increments, this value measures the ISN variability. It roughly estimates how difficult it would be to predict the next ISN from the known sequence of six probe responses. The calculation uses the difference array (seq_r a t e s ) and GCD values discussed in the previous section. This test is only performed if at least four responses were seen. If the previously computed GCD value is greater than nine, The elements of the previously computed seq_r a t e s array are divided by that value. 180 8.3. TCP/IP Fingerprinting Methods Supported by Nmap We don't do the division for smaller GCD values because those are usually caused by chance. A standard deviation of the array of the resultant values is then taken. If the result is one or less, SP is zero. Otherwise the binary logarithm of the result is computed, then it is multiplied by eight, rounded to the nearest integer, and stored as SP. Please keep in mind that this test is only done for OS detection purposes and is not a full-blown audit of the target ISN generator. There are many algorithm weaknesses that lead to easy predictability even with a high SP value. TC P IP ID sequence generation algorithm (TI) This test examines the IP header I D field for every response to the TCP S E Q probes. The test is only included if at least three probes were returned. It then classifies the target IP ID generator based on the algorithm below. Note that difference values assume that the counter can wrap. So the difference between an IP ID of 65,100 followed by a value of 700 is 1 1 36. The difference between 2,000 followed by 1 , 1 00 is 64,636. Here are the calculation details: I. I f all of the ID numbers are zero, TI is set to z . 2 . I f the IP I D sequence ever increases by a t least 20,000, T I i s set to R D (random). 3. If all of the IP IDs are identical, TI is set to that value in hex. 4. If any of the differences between two consecutive IDs exceed 1000, and is not evenly divisible by 256, TI is set to RI (random positive increments). If the difference is evenly divisible by 256, it must be at least 256,000 to cause this RI result. 5. If all of the differences are divisible by 256 and no greater than 5 1 20, TI is set to BI (broken increment). This happens on systems like Microsoft Windows where the IP ID is sent in host byte order rather than network byte order. It works fine and isn't any sort of RFC violation, though it does give away host architecture details which can be useful to attackers. 6. If all of the di fferences are less than ten, T I is set to I (incremental). We allow difference up to ten here (rather than requiring sequential ordering) because traffic from other hosts can cause sequence gaps. 7. If none of the previous steps identify the generation algorithm, the test is omitted from the fingerprint. MP IP I D sequence generation algorithm (I I) is test i s similar to T I above, except that it evaluates I P IDs from the ICMP responses t o our two ping bes. It is only included if both responses are received. IP ID differences are absolute (assume wrapping) are calculated as described in T I . The result is easier to calculate than T I . There is no RD result because aren't enough samples to support it. I I is calculated as follows: If both ID numbers are zero, I I is set to z If both IP IDs are identical, T I is set to that value in hex. 8.3. TCP/IP Fingerprinting Methods Supported by Nmap 181 3. If the absolute difference IDs exceed 1000, and is not evenly divisible by 256, I I is set to RI (random positive increments). If the difference is evenly divisible by 256, it must be at least 256,000 to cause this R I result. 4. If the IP ID difference is divisible by 256 and no greater than 51 20, I I is set to BI (broken increment). This happens on systems like Microsoft where the IP ID is sent in host byte order rather than network byte order. It works fi ne and isn't any sort of RFC violation, though it does give away host architecture details which can be useful to attackers. 5. If the difference is less than ten, I I is set to I (incremental). We allow difference up to ten here (rather than requiring sequential ordering) because traffic from other hosts can cause sequence gaps. 6. If none of the previous steps identify the generation algorithm, the test is omitted from the fingerprint. Shared IP ID seq uence Boolean (ss) This Boolean value records whether the target shares its IP ID sequence between the TCP and ICMP protocols. If our six TCP IP ID values are 1 17, 1 1 8 , 1 1 9, 1 20, 1 2 1 , and 1 22, then our ICMP results are 1 23 and 1 24, it is clear that not only are both sequences incremental, but they are both part of the same sequence. If, on the other hand, TCP IP ID values are 1 17- 1 22 but the ICMP values are 32,917 and 32,918, a different sequence is being used. This test is only included if I I is R I , B I , or I and T I is the same. If SS is included, the result is S if the sequence is shared and O (other) if it is not. That determination is made by the following algorithm: Let avg be the final TCP sequence response IP ID minus the first TCP sequence response IP ID, divided by the difference in probe numbers. If probe # 1 returns an IP ID of 10,000 and probe #6 returns 20,000, avg would be (20,000 - 10,000) I (6 - 1 ) , which equals 2,000. If the first ICMP echo response IP ID is less than the fi nal TCP sequence response IP ID plus three times a vg, the S S result is s . Otherwise it is 0. TCP timestamp option algorithm (TS) T S is another test which attempts to determine target OS characteristics based on how it generates a series of numbers. This one looks at the TCP timestamp option (if any) in responses to the SEQ probes. It examines the TSval (first four bytes of the option) rather than the echoed TSecr (last four bytes) value. It takes the difference between each consecutive TSval and divides that by the amount of time elapsed between Nmap sending the two probes which generated those responses. The resultant value gives a rate of timestamp increments per second. Nmap computes the average increments per second over all consecutive probes and then calculates the T S as follows: I . If any of the responses have no timestamp option, TS is set to U (unsupported). 2. If any of the timestamp values are zero, Ts is set to 0 . 3. I f the average increments per second falls within the ranges 0 - 5 6 6 , 7 0 - 1 5 0 , or 1 5 0 - 3 5 0 , T s i s set to 1 , 7, or 8, respectively. These three ranges get special treatment because they correspond to the 2 Hz, JOO Hz, and 200 Hz frequencies used by many hosts. . 1 82 8.3. TCP/IP Fingerprinting Methods Supported by Nmap 4. In all other cases, Nmap records the binary logarithm of the average increments per second, rounded to the nearest integer. Since most hosts use 1 ,000 Hz frequencies, A is a common result. TCP options (o, 0 1 - 0 6) This test records the TCP header options in a packet. It preserves the original ordering and also provides some information about option values. Because RFC 793 doesn't require any particular ordering, implementations often come up with unique orderings. Some platforms don't implement all options (they are, of course, optional). When you combine all of those permutations with the number of different option values that implementations use, this test provides a veritable trove of information. The value for this test is a string of characters representing the options being used. Several options take arguments that come immediately after the character. Supported options and arguments are all shown in Table 8. l . Table 8.1. O test values Option Name Character Argument (if any) End of Options List (EOL) L No operation (NOP) N Maximum Segment Size (MSS) M The value is appended. Many systems echo the value used in the corresponding probe. Window Scale (WS) w The actual value is appended. Timestamp (TS) T The T is followed by two binary characters representing the TSval and TSecr values respectively. The characters are 0 if the field is zero and 1 otherwise. Selective ACK permitted (SACK) s As an example, the string M5B4NW3NNT 1 1 means the packet includes the MSS option (value Ox5B4) followed by a NOP. Next comes a window scale option with a value of three, then two more NOPs. The final option is a timestamp, and neither of its two fields were zero. If there are no TCP options in a response, the test will exist but the value string will be empty. If no probe was returned, the test is omitted. While this test is generally named o, the six probes sent for sequence generation purposes are a special case. Those are inserted into the special OPS test line and take the names 01 through 06 to distinguish which probe packet they relate to. The "O" stands for "options". Despite the different names, each test 01 through 06 is processed exactly the same way as the other O tests. TCP initial window size (w, Wl-W6) This test simply records the 16-bit TCP window size of the received packet. It is quite effective, since there are more than 80 values that at least one OS is known to send. A down side is that some operating systems have more than a dozen possible values by themselves. This leads to false negative results until we collect all of the possible window sizes used by an operating system. While this test is generally named w, the six probes sent for sequence generation purposes are a special case. Those are inserted into a special W I N test line and take the names Wl through W6 . The window size is recorded for all of the sequence number probes because they differ in TCP MSS option values, which causes some 8.3. TCP/IP Fingerprinting Methods Supported by Nmap 183 operating systems to advertise a different window size. Despite the different names, each test is processed exactly the same way. Responsiveness (R) This test simply records whether the target responded to a given probe. Possible values are Y and N. If there is no reply, remaining fields for the test are omitted. A risk with this test involves probes that are dropped by a firewall. This leads to R=N in the subject fingerprint. Yet the reference fingerprint in nmap- o s -db may have R=Y if the target OS usually replies. Thus the firewall could prevent proper OS detection. To reduce this problem, reference fingerprints generally omit the R=Y test from the I E and U l probes, which are the ones most likely to be dropped. In addition, if Nmap is missing a closed TCP port for a target, it will not set R=N for the T S , T 6 , or T 7 tests even if the port it tries is non-responsive. A fter all, the lack of a closed port may be because they are all filtered. IP don't fragment bit (DF) The IP header contains a single bit which forbids routers from fragmenting a packet. If the packet is too large for routers to handle, they will just have to drop it (and ideally return a "destination unreachable, fragmentation needed" response). This test records Y if the bit is set, and N if it isn't. Don't frag ment (ICMP) (DF I) This i s simply a modified version of the D F test that i s used for the special I E probes. I t compares results of the don't fragment bit for the two ICMP echo request probes sent. It has four possible values, which are enumerated in Table 8.2. Table 8.2. DFI test values · Value Description N Neither of the ping responses have the DF bit set. s Both responses echo the DF value of the probe. y Both of the response DF bits are set. 0 The one remaining other combination-both responses have the DF bit toggled. IP i n itial time-to-live (T) IP packets contain a field named time-to-live (TTL) which is decremented every time they traverse a router. If the field reaches zero, the packet must be discarded. This prevents packets from looping endlessly. Because operating systems differ on which TTL they start with, it can be used for OS detection. Nmap determines how many hops away it is from the target by examining the ICMP port unreachable response to the Ul probe. That response includes the original IP packet, including the already-decremented TTL field, received by the target. By subtracting that value from our as-sent TTL, we learn how many hops away the machine is. Nmap then adds that hop distance to the probe response TTL to determine what the initial TTL was when that ICMP probe response packet was sent. That initial TTL value is stored in the fingerpri nt as the T result. 184 8.3. TCP/IP Fingerprinting Methods Supported by Nmap Even though an eight-bit field like TIL can never hold values greater than OxFF, this test occasionally results in values ofOx 100 or higher. This occurs when a system (could be the source, a target, or a system in between) corrupts or otherwise fails to correctly decrement the TIL. It can also occur due to asymmetric routes. Nmap can also learn from the system interface and routing tables when the hop distance is zero (localhost scan) or one (on the same network segment). This value is used when Nmap prints the hop distance for the user, but it is not used for T result computation. IP initial time-to-l ive guess (TG) It is not uncommon for Nmap to receive no response to the U l probe, which prevents Nmap from learning how many hops away a target is. Firewalls and NAT devices Jove to block unsolicited UDP packets. But since common TIL values are spread well apart and targets are rarely more than 20 hops away, Nmap can make a pretty good guess anyway. Most systems send packets with an initial TIL of 32, 60, 64, I 28, or 255. So the TIL value received in the response is rounded up to the next value out of 32, 64, 1 28, or 255. 60 is not in that list because it cannot be reliably distinguished from 64. It is rarely seen anyway. The resulting guess is stored in the TG field. This TIL guess field is not printed i n a subject fingerprint if the actual TIL (T) value was discovered. Expl icit congestion notification (cc) This Lest is only used for the ECN probe. That probe is a SYN packet which includes the CWR and ECE congestion control flags. When the response SYN/ACK is received, those flags are examined to set the CC (congestion control) test value as described in Table 8.3. Table 8.3. cc test values Value Description y Only the ECE bit is set (not CWR). This host supports ECN. N Neither of these two bits is set. The target does not support ECN. s Both bits are set. The target does not support ECN, but it echoes back what it thinks is a reserved bit. 0 The one remaining combination of these two bits (other). TCP miscel laneous quirks (Q) This tests for two quirks that a few implementations have in their TCP stack. The first is that the reserved field in the TCP header (right after the header length) is nonzero. This is particularly likely to happen i n response to the E C N test a s that o n e sets a reserved b i t in the probe. If this is seen in a packet, an "R" is recorded in the Q string. The other quirk Nmap tests for is a nonzero urgent pointer field value when the URG flag is not set. This is also particularly likely to be seen in response to the ECN probe, which sets a non-zero urgent field. A "U" is appended to the Q string when this is seen. The Q stri ng must always be generated in alphabetical order. If no quirks are present, the Q test is empty but still shown. 8.3. TCP/IP Fingerprinting Methods Supported by Nmap 185 TCP sequence number (s) This test examines the 32-bit sequence number field in the TCP header. Rather than record the field value as some other tests do, this one examines how it compares to the TCP acknowledgment number from the probe that elicited the response. It then records the appropriate value as shown in Table 8.4. Table 8.4. S test values Value Description z Sequence number is zero. A Sequence number is the same as the acknowledgment number in the probe. A+ Sequence number is the same as the acknowledgment number in the probe plus one. 0 Sequence number is something else (other). ICMP sequence number(SI) This test looks a t the sequence number i n ICMP echo response packets. I t i s only used for the two I E echo request probes. The four values it can take are shown in Table 8.5. Table 8.5. SI test values Value Description z Both sequence numbers are set to 0. s Both sequence numbers echo the ones from the probes. When they both use the same non-zero number, it is recorded here. 0 Any other combination. TCP acknowledgment number (A) This test is the same as S except that it tests how the acknowledgment number in the response compares to the sequence number in the respective probe. The four possible values are given in Table 8.6. Table 8.6. A test values Value Description z Acknowledgment number is zero. s Acknowledgment number is the same as the sequence number in the probe. S+ Acknowledgment number is the same as the sequence number in the probe plus one. 0 Acknowledgment number is something else (other). 186 8.3. TCP/IP Fingerprinting Methods Supported by Nmap TCP flags (F) This field records the TCP flags in the response. Each letter represents one flag, and they occur in the same order as in a TCP packet (from high-bit on the left, to the low ones). So the value SA represents the SYN and ACK bits set, while the value AS is illegal (wrong order). The possible flags are shown in Table 8.7. Table 8.7. F test values Character Flag name Flag byte value E ECN Echo (ECE) 64 u Urgent Data (URG) 32 A Acknowledgment (ACK) 16 p Push (PSH) 8 R Reset (RST) 4 s Synchronize (SYN) 2 F Final (FIN) 1 TCP RST data checksum (RD) Some operating systems return ASCII data such as error messages in reset packets. This is explicitly allowed by section 4.2.2.12 of RFC 1 1 22. When Nmap encounters such data, it performs a CRC 16 checksum and reports the results. When there is no data, RD is set to zero. Some of the few operating systems that may return data in their reset packets are HP-UX and versions of Mac OS prior to Mac OS X. IP type of service (TOS) This test simply records the type of service byte from the IP header of ICMP port unreachable packets. This byte is described in RFC 791 . The value is not recorded for other responses (such as TCP or echo response packets) because variations there are usually caused by network devices or host services rather than reflecting the target OS itself. IP type of service for ICMP responses (TOS I) This test compares the I P type of service (TOS) bytes from the responses t o both I E test ICMP echo request probes. The possible values are shown in Table 8.8. Table 8.8. TOS I test values Value Description z Both TOS values are zero. s Both TOS values are each the same as in the corresponding probe. When they both use the same non-zero number, it is recorded here. 0 Any other combination. 8.3. TCP/IP Fingerprinting Methods Supported by Nmap 1 87 I P total length (IPL) This test records the total length (in octets) of an IP packet. It is only used for the port unreachable res elicited by the U l test. That length varies by implementation because they are allowed to choose how m data from the original probe to include, as long as they meet the minimum RFC 792 requirement. requirement is to incl ude the original IP header and at least eight bytes of data. U nused port unreachable field nonzero (UN) An ICMP port unreachable message header is eight bytes long, but only the first four are used. RFC 7 states that the last four bytes must be zero. A few implementations (mostly ethernet switches and so specialized embedded devices) set it anyway. The value of those last four bytes is recorded in this field. Retu rned probe I P total length value (RIPL) ICMP port unreachable messages (as are sent i n response to the u 1 probe) are required to include the header which generated them. This header should be returned just as they received it, but some implementatio send back a corrupted version due to changes they made during IP processing. This test simply records t returned IP total length value. If the correct value of Ox 148 (328) is returned, the value G (for good) is sto instead of the actual value. Returned probe I P I D value (RID) The u 1 probe has a static IP ID value of Ox 1042. If that value is returned in the port unreachable message, the value G is stored for this test. Otherwise the exact value returned is stored. Some systems, such as Solaris, manipulate IP ID values for raw IP packets that Nmap sends. In such cases, this test is ski pped . We have found that some systems, particularly HP and Xerox printers, flip the bytes and return Ox4210 instead. Integrity of retu rned probe I P checksum val ue (RIPCK) The IP checksum is one value that we don 't expect to remain the same when returned in a port unreachable message. After all, each network hop during transit changes the checksum as the TTL is decremented. However, the checksum we receive should match the enclosing IP packet. If it does, the value G (good ) is stored for this test. If the returned value is zero, then z is stored. Otherwise the result is I (i nvalid). Integ rity of retu rned probe U D P length and checksum (RUL and RUCK) The UDP header length and checksum values should be returned exactly as they were sent. If so, G is recorded for these tests. Otherwise the value actually returned is recorded. The proper length is Ox l 34 (308). Integ rity of retu rned U D P data (RUD) If the UDP payload returned consists of 300 'C' (Ox43) characters as expected, a G is recorded for this test. Otherwise I (invalid) is recorded. 188 8.3. TCP/IP Fingerprinting Methods Supported by Nmap MP response code (co) code value of an ICMP echo reply (type zero) packet is supposed to be zero. But some implementations ngly end other values, particularly if the echo request has a nonzero code (as one of the I E tests does). response code values for the two probes are combined into a CD value as described in Table 8.9. Description Both code values are zero. Both code values are the same as in the corresponding probe. When they both use the same non-zero number, it is shown here. Any other combination. P data length for ICMP responses (DLI) n data is included with an ICMP echo request packet, i t i s supposed to be returned i ntact i n the sponding echo response. But some implementations truncate the data anyway. This tests looks at both P responses to the I E probes, and assigns a value as described in Table 8. 10. hie 8.10. DLI test values Description Neither response includes any data. Both responses return all data sent in the corresponding request. If at least one of the responses truncates the data, the largest amount of data returned (in either packet) is stored here. When they both truncate the data length to the same non-zero number, it is shown here. This value only counts actual data, not the IP or ICMP headers . . 4. Fingerpri nting Methods Avoided by map map supports many more OS detection techniques than any other program, and we are always interested hearing about new ideas. Please send them to the Nmap development list (nmap-dev) for discussion. ever there are some methods that just aren't a good fit. This section details some of the most i nteresting . While they aren't supported by Nmap, some are useful in combination with Nmap to verify findings learn further details . . 4.1 . Passive Fi ngerpri nti ng sive fingerprinting uses most of the same techniques as the active fingerprinting performed by Nmap. difference is that a passive system simply sniffs the network, opportunistically classifying hosts as it rves their traffic. This is more difficult than active fingerprinting, since you have to accept whatever 8.4. Fingerprinting Methods Avoided by Nmap 189 communication happens rather than designing your own custom probes. It is a valuable technique, but doesn't belong i n a fundamentally active tool such as Nmap. Fortunately, Michal Zalewski has written the excellent pOf2 passive OS fi ngerprinting tool. He also devised a couple of the current Nmap OS fingerprinting tests. 3 Another option is SinFP by GomoR, which supports both active and passive fingerprinting. TCP/IP fi ngerprinting works well for distinguishing different operating systems, but detecting different versions of the same operating system can be troublesome. The company must change their stack in some way we can differentiate. Fortunately, many OS vendors regularly update their systems to comply with the latest standards. But what about those who don't? Most of them at least get around to fixing exploitable stack bugs eventually. And those fixes are easy to detect remotely. First send the exploit payload, be it a land attack, teardrop, ping of death, SYN flood, or WinNuke. Send one attack at a time, then immediately try to contact the system again. If it is suddenly non-responsive, you have narrowed down the OS to versions which didn't ship with the fix. 0 Warning If you use denial of service (DoS) exploits as part of your OS detection suite, remember to perform those tests last. 8.4.3. Retransm ission Ti mes TCP i mplementations have significant leeway in exactly how long they wait before retransmitting packets. The proof-of-concept tools Ring and Cron-OS are available to exploit this. They send a SYN packet to an open port, then ignore the SYN/ACK they receive rather than acknowledging it with an ACK (to complete the connection) or a RST (to kill it). The target host will resend the SYN/ACK several more times, and these tools track every subsecond of the wait. While some information can indeed be gleaned from this technique, there are several reasons that I haven't incorporated the patch into Nmap: • It usually requires modifying the source host firewal l rules to prevent your system from replying with a RST packet to the SYN/ACK it receives. That is hard to do in a portable way. And even if it was easy, many users don't appreciate applications mucking with their firewall rules. • It can be very slow. The retransmissions can go on for several minutes. That is a long time to wait for a test that doesn't give all that much information in the first place. • It can be inaccurate because packet drops and latency (which you have to expect in real-world environments) can lead to bogus results. I have enumerated these reasons here because they also apply to some other proposed OS detection methods. I would love to add new tests, but they must be quick and require few packets. Messing with host firewall is unacceptable. I try to avoid making ful l TCP connections for stack fingerprinting, though that is done for OS detection as part of the version scanning system. 2 3 http:lllcamtufcoredump.cxlpOfsht1nl http:/lwww.gomor.org/bin/view/Sinfp 190 8.4. Fingerprinting Methods Avoided by Nmap 8.4.4. IP Frag mentation I P fragmentation i s a complex system and implementations are riddled with bugs and inconsistencies. Possible tests could examine how overlapping fragments are assembled or time the defragmentation timeouts. These ' tests are avoided for Nmap because many firewalls and other i nline devices defragment traffic at gateways. Thus Nmap may end up fingerprinting the firewall rather than the true destination host. In addition, fragments are difficult to send on some operating systems. Linux 2.6 kernels have a tendency to queue the fragments you are trying to send and assemble them itself before transmission. 8.4.5. Open Port Patterns The target host OS can often be guessed simply by looking a t the ports which are open. Microsoft Windows machines often have TCP ports 1 35 and 1 39 open. Windows 2000 and newer also listen on port 445. Meanwhile, a machine running services on port 22 (ssh) and 631 (Internet Printing Protocol) is likely running Unix. While this heuristic is often useful, it just isn't reliable enough for Nmap. Combinations of ports can be obscured by firewall rules, and most mainstream protocols are available on multiple platforms. OpenSSH servers can be run on Windows4, and the "Windows SMB" ports can be serviced by Samba5 running on a Unix machine. Port forwarding clouds the issue even further. A machine which appears to be running Microsoft IIS might be a Unix firewall simply forwarding port 80 to a Windows machine. For these reasons, Nmap does not consider open port numbers during TCP/IP stack fingerprinting. However, Nmap can use version detection i nformation (see Chapter 7, Service and Application Version Detection [ 1 45)) to separately discover operating system and device type information. By keeping the OS detection results discovered by OS detection and version detection separate, Nmap can gracefully handle a Checkpoint firewall which uses TCP port forwarding to a Windows web server. The stack fingerprinting results should be "Checkpoint Firewall- I " while version detection should suggest that the OS is Windows. Keep i n mind that only a small fraction of version detection signatures i nclude OS and device type information-we can only populate these fields when the application divulges the i nformation or when it only runs on one OS or device type. 8.5. Understand i ng an Nmap Fi ngerpri nt When Nmap stores a fingerprint in memory, Nmap uses a tree o f attributes and values i n data structures that users need not even be aware of. But there is also a special ASCII-encoded version which Nmap can print for users when a machine is unident.i fied. Thousands of these serialized fingerprints are also read back every time Nmap runs (with OS detection enabled) from the nmap- o s -db database. The fingerprint format is a compromise between human comprehension and brevity. The format is so terse that it looks like line noise ID many inexperienced users, but those who read this document should be able to decipher fingerprints with eue. There are actually two types of fingerprints, though they have the same general structure. The fingerprints ofknown operating systems that Nmap reads in are called referencefingerprints, while the fingerprint Nmap · Jays after scanning a system is a subject fingerprint. The reference fi ngerprints are a bit more complex · e they can be tailored to match a whole class of operating systems by adding leeway to (or omitting) 8.5. Understanding an Nmap Fingerprint 191 tests that aren't so reliable while allowing only a single possible value for other tests. The reference fingerprin also have OS details and classifications. Since the subject tests are simpler, we describe them first. 8.5.1 . Decod ing the Su bject Fingerpri nt Format If Nmap performs OS fingerprinting on a host and doesn't get a perfect OS matches despite promisi ng conditions (such as finding both open and closed ports accessible on the target), Nmap prints a subject fi ngerprint that shows all of the test results that Nmap deems relevant, then asks the user to submit the data to Nmap.Org. Tests aren't shown when Nmap has no useful results, such as when the relevant probe responses weren't received. A special line named SCAN gives extra details about the scan (such as Nmap version number) that provide useful context for integrating fingerprint submissions into nmap-os -db. A typical subject fingerprint is shown in Example 8.3. Example 8.3. A typical subject fingerprint OS : SCAN (V=4 . 62%D=S/21%0T=80%CT=l%CU=36 069%PV=Y%DS=l%G=Y%M=001839%TM=483466E OS : 0%P=i686-pc-linux-gnu ) S EQ ( SP=C9%GCD=l% ISR=CE %TI=Z%II=I %TS=8 ) 0PS (Ol=MSB4S OS : Tl lNW0%02=MSB4STllNW0%03=MSB4NNTl lNW0%04=MSB4STl lNW0%05=MSB4STl lNW0%06=M OS : SB4STl l ) WIN (Wl=l6A0%W2=16A0%W3=16A0%W4=16A0%W5=16A0%W6=16AO ) ECN (R=Y%DF=Y OS : %T=40%W=l6D0%0=MSB4NNSNW0%CC=N%Q=) Tl (R=Y%DF=Y%T=40%S=0%A=S+%F=AS%RD=0%Q= OS : ) T2 (R=N) T3 (R=Y%DF=Y%T=40%W=l6A0%S=O%A=S+%F=AS%0=MSB4ST1 1NW0%RD=0%Q=)T4 (R OS : =Y%DF=Y%T=40%W=0%S=A%A=Z%F=R%0=%RD=0%Q=) T5 (R=Y%DF=Y%T=40%W=0%S=Z%A=S+%F= OS :AR%0=%RD=0%Q= ) T6 (R=Y%DF=Y%T=40%W=0%S=A%A=Z%F=R%0=%RD=0%Q=) T7 (R=Y%DF=Y%T= OS : 40%W=0%S=Z%A=S+%F=AR%0=%RD=0%Q=) Ul (R=Y%DF=N%T=40%TOS=C0%IPL=l64%UN=0%RIP OS : L=G%RID=G%RIPCK=G%RUCK=G%RUL=G%RUD=G) IE (R=Y%DFI=N%T=40%TOSI=S%CD=S%SI=S% OS : DLI=S ) Now you may look at this fingerpri nt and immediately understand what everything means. If so, you can simply skip this section. But I have never seen such a reaction. Many people probably think some sort of buffer overflow or unterminated string error is causing Nmap to spew garbage data at them. This section . helps you decode the information so you can immediately tell that blind TCP sequence prediction attacks against this machine are moderately hard, but it may make a good idle scan (-s I ) zombie. The first step in understanding this fingerprint is to fix the line wrapping. The tests are all squished together, with each line wrapped at 71 characters. Then OS : is prepended to each line, raising the length to 74 characters. This makes fingerprints easy to cut and paste into the Nmap fingerprint submission form (see Section 8 . 7. 2, "When Nmap Fails to Find a Match and Prints a Fingerprint" [20 1 ]). Removing the prefix and fixing the word wrapping (each line should end with a right parenthesis) leads to the cleaned-up version in Example 8.4. 1 92 8.5. Understanding an Nmap Fingerprint Example 8.4. A cleaned-up subject fingerprint SCAN (V=4 . 62%D=5/21%0T=80%CT=l%CU=36 069%PV=Y%DS=l%G=Y%M=001839% TM=483466 E 0%P=i6 86-pc-linux-gnu) SEQ (SP=C9%GCD=1%ISR=C E %TI=Z%I I=I %TS=8 ) OPS (Ol=M5B4ST11NW0%02=M5B4ST11NW0%03=M5B4NNT1 1NW0%04=M5B4ST1 1NW0% 05=M5B4ST11NW0%06=M5B4ST1 1 ) WIN (Wl=l6A0%W2=16A0%W3=16A0%W4=16A0%W5=16A0%W6=16A0 ) ECN (R=Y%DF=Y%T=40%W=l6D0%0=M5B4NNSNW0%CC=N%Q=) Tl (R=Y%DF=Y%T=40%S=0%A=S+%F=AS%RD=0%Q=) T2 (R=N) T3 (R=Y%DF=Y%T=40%W=l6A0%S=O%A=S+%F=AS%0=M5B4ST11NW0%RD=0%Q= ) T4 (R=Y%DF=Y%T=40%W=0%S=A%A=Z%F=R%0=%RD=0%Q=) T5 (R=Y%DF=Y%T=40%W=0%S=Z%A=S+%F=AR%0=%RD=0%Q= ) T6 (R=Y%DF=Y%T=40%W=0%S=A%A=Z%F=R%0=%RD=0%Q=) T7 (R=Y%DF=Y%T=40%W=0%S=Z%A=S+%F=AR%0=%RD=0%Q=) Ul (R=Y%DF=N%T=40%TOS=C0% I PL=l64%UN=0%RIPL=G%RID=G%RIPCK=G%RUCK=G%RUL=G%RUD=G) IE (R=Y%DFI=N%T=40%TOSI=S%CD=S%SI=S%DLI=S ) While this still isn't the world's most intuitive format (we had to keep it short), the format is much clearer now. Every line is a category, such as SEQ for the sequence generation tests, T 3 for the results from that particular TCP probe, and IE for tests related to the two ICMP echo probes. Following each test name is a pair of parentheses which enclose results for individual tests. The tests take the format = . All of the possible categories, tests, and values are described in Section 8.3, "TCP/IP Fingerprinting Methods Supported by Nmap" [ 1 76]. Each pair of tests are separated by a percentage symbol (%). Tests values can be empty, leading to a percentage symbol or category-termi nating right-parenthesis immediately following the equal sign. The string "0= %RD=0 %Q= ) " in T 4 of our example shows two of these empty tests. A blank test value must match another blank value, so this empty TCP quirks Q value wouldn't match a fingerprint with Q set to RU. In some cases, a whole test is missing rather than just its value. For example, T2 of our sample fingerprint has no W (TCP window), S (sequence number), A (acknowledgment number), T (TIL), or TG (TIL guess) tests. This is because the one test and value it does include, R=N, means that no response was returned for the T2 probe. So including a wi ndow value or sequence number would make little sense. Similarly, tests which aren't well supported on the system running Nmap are skipped. An example is the R I D (IP ID field returned in ICMP packet) test, which doesn't work well on Solaris because that system tends to corrupt the ID field Nmap sends out. Tests which are i nconclusive (such as failing to detect the IP ID sequence for the T I and I I tests) are also omitted. Decoding the SCAN line of a subject fingerprint The SCAN line is a special case i n a subject fingerprint. Rather than describe the target system, these tests describe various conditions of the scan. These help us integrate fingerprints submitted to Nmap.Org. The tests in this line are: • Nmap version number (V). • Date of scan (D) in the form month/day. 8.5. Understanding an Nmap Fingerprint 1 93 • Open and closed TCP ports (on target) used for scan (OT and CT). Unlike most tests, these are printed in decimal formal. If Nmap was unable to find an open or a closed port, the test is included with an empty value (even when Nmap guesses a possibly closed port and sends a probe there). • Closed UDP port (CU). This is the same as CT, but for UDP. Since the majority of scans don't include UDP, this test's value is usually empty. • Private IP space (PV) is Y if the target is on the 1 0 . 0 . 0 . 0 I 8, 1 7 2 . 1 6 . 0 . 0 I 1 2 , or 1 9 2 . 1 6 8 . 0 . 0 I 1 6 private networks (RFC 191 8). Otherwise i t is N . • Network distance ( D S ) is the network hop distance from the target. I t i s 0 i f the target i s localhost, 1 if directly connected on an ethernet network, or the exact distance if discovered by Nmap. If the distance is unknown, this test is omitted. • Good results (G) is Y if conditions and results seem good enough to submit this fingerprint to Nmap.Org. It i s N otherwise. Unless you force them by enabling debugging (-d) or extreme verbosity (-vv), G=N fingerprints aren't printed by Nmap. • Target MAC prefix (M) is the first six hex digits of the target MAC address, which correspond to the vendor name. Leading zeros are not included. This field is omitted unless the target is on the same ethernet network (DS= l ). • The OS scan time (TM) is provided i n Unix time_t format (in hexadecimal). • The platform Nmap was compiled for is given in the P field. 8.5.2. Decod i ng the Reference Fi ngerprint Format When Nmap scans a target to create a subject fingerprint, i t then tries to match that data against the thousands of reference fingerprints in the nmap - o s -db database. Reference fingerprints are initially formed from one or more subject fingerprints and thus have much in common. They do have a bit of extra information to facilitate matching and of course to describe the operating systems they represent. For example, the subject fingerprint we just looked at might form the basis for the reference fi ngerprint in Example 8.5. 1 94 8.5. Understanding an Nmap Fingerprint pie 8.5. A typical reference fingerprint rprint Sony PlayStation 3 game console Sony I embedded I I game console P•F7-101%GCD=<7%ISR=FC-1 06 %TI=RD%TS=21 ) l•M5B4NNSNW1NNT1 1%02=M5B4NNSNW1NNT1 1%03=M5B4NW1NNT11% 4•M5B4NNSNWlNNT1 1 %05=M5B4NNSNWlNNT11%06=M5B4NNSNNTl l ) l•FFFF%W2=FFFF%W3=FFFF%W4=FFFF%W5=FFFF%W6=FFFF) l•Y%DF=N%T=41%TG=41 %W=FFFF%0=M5B4NNSNW1%CC=N%Q=) Y%DF=N%T=41%TG=41%S=0%A=S+%F=AS%RD=0%Q= ) Y%DF=N%T=41%TG=41%W=0%S=Z%A=O J S%F=AR%0=%RD=0%Q=) Y%DF=N%T=41%TG=41%W=FFFF%S=0%A=S+%F=AS%0=M5B4NNSNW1NNT1 1%RD=0%Q=) •Y%DF=N%T=41%TG=41%W=0%S=A l 0%A=Z%F=R%0=%RD=0%Q=) Y%DF=N%T=40%TG=40%W=0%S=Z%A=O I S+%F=AR%0=%RD=0%Q= ) •Y%DF=N%T=40%TG=40%W=0%S=A I O%A=Z%F=R%0=%RD=0%Q=) •Y%DF=N%T=40%TG=40%W=0%S=Z%A=O I S%F=AR%0=%RD=0%Q=) F•N%T=FF%TG=FF%TOS=0% IPL=38%UN=0%RIPL=G%RID=G%RIPCK=G%RUCK=G%RUL=G%RUD=G) FI=N%T=FF%TG=FF%TOSI=S%CD=S%SI=S%DLI=S) Some differences are immediately obvious. Line wrapping is not done because that is only important for the mission process. The SCAN line is also removed, since that information describes a specific scan instance 1llher than general target OS characteristics. You probably also noticed the two new lines, F i ngerpr i nt and C l a s s , which are new to this reference lngerprint. A more subtle change is that some of the individual test results have been removed while others have been enhanced with logical expressions. Free-form OS description (Fingerprint line) The F ingerpr int line first serves as a token so Nmap knows to start loading a new fingerprint. Each fingerprint only has one such line. Immediately after the F i ngerpr i n t token (and a space) comes a textual description of the operating system(s) represented by this fingerprint. These are in free-form English text, designed for human interpretation rather than a machine parser. Nevertheless, Nmap tries to stick with a consistent format including the vendor, product name, and then version number. Version number ranges and comma-separated alternatives discussed previously can be found in this field. Here are some examples: qerprint qerprint qerprint qerprint qerprint qerprint HP LaserJet printer ( 4050, 4100, 4200, or 8150) Sun Solaris 9 or 10 ( SPARC) Linux 2 . 6 . 22 - 2 . 6 . 24 Microsoft Windows Server 2003 SPl Microsoft Windows XP Professional SPl Minolta Di550 laser printer In an ideal world, every different OS would correspond to exactly one unique fi ngerprint. Unfortunately, OS vendors don't make life so easy for us. The same OS release may fingerprint differently based on what aetwork drivers are in use, user-configurable options, patch levels, processor architecture, amount of RAM available, firewall settings, and more. Sometimes the fingerprints differ for no discernible reason. While the reference fingerprint format has an expression syntax for coping with slight variations, creating multiple fingerprints for the same OS is often preferable when major differences are discovered. Just as multiple fingerprints are often needed for one OS, sometimes a single fingerprint describes several systems. If two systems give the exact same results for every single test, Nmap has l ittle choice but to offer 8.5. Understanding an Nmap Fingerprint 1 95 up both as possibilities. This commonly occurs for several reasons. One is that vendors may release a new version of their OS without any significant changes to their IP stack. Maybe they made important changes elsewhere in the system, or perhaps they did little but want to make a bunch of money selling "upgrades". In these cases, Nmap often prints a range such as App l e Mac OS X 1 0 . 4 . 8 - 1 0 . 4 . 1 1 or Sun S o l ar i s 9 or 1 0 . Another cause of duplicate fingerprints i s embedded devices which share a common OS. For example, a printer from one vendor and an ethernet switch from another may actually share an embedded OS from a third vendor. In many cases, subtle differences between the devices still allow them to be disti nguished. But sometimes Nmap must simply list a group of possibilities such as C i s c o 1 2 0 0 - s e r i e s WAP , HP Pr oCurve 2 6 5 0 swi t ch , or Xerox Phaser 7 4 0 0 N or 8 5 5 0 DT p r i nter. There are also cases where numerous vendors private label the exact same OEM device with their own brand name and model number. Here again, Nmap must simply list the possibilities. But distinguishing these is less important because they are all fundamentally the same device. Tip If the description printed by Nmap (which comes from the Fi ngerprint line) isn't informative enough for you, more detailed information may be available in comments above the fingerprint itself in nmap - o s -db. You can find it installed on your system as described in Chapter 14, Understanding and Customizing Nmap Data Files [363], or look up the latest version at http://nmap. org/datalnmap-os-db. Search for the exact OS description that Nmap gives you. Keep i n mind that there may be several F i ngerpr i:n t lines with exactly the same description, so you may have to examine them all. Or use the Nmap XML output, which shows the line number of each match. Device and OS classification (Class l ines) · While the F i nge r p r i n t description works great for analysts reading Nmap output directly, many people run Nmap from other scripts and applications. Those applications might use the OS information to check for OS-specific vulnerabilities or just create a pretty graph or report. A more structured OS classification system exists for these purposes. It is also useful when there are multiple matches. If you only get a partial fingerprint (maybe no open ports were found on the target so many tests had to be skipped), it might match dozens of different fingerprints in the nmap-os -db database. Pri nting the details for all of those fingerprints would be a mess. But thanks to OS classification, Nmap can find commonality. If all of the matches are classified as Linux, Nmap will simply print that the target is a Linux box. Every fingerprint has one or more C l a s s lines. Each contains four well-defined fields: vendor, OS name, OS family, and device type. The fields are separated by the pipe symbol ( I ) . The device type is a broad classification such as router, pr i nter, or game c o n s o l e and was discussed previously in this chapter. General-purpose operati ng systems such as L i nux and W i ndows which can be used for just about anything are classi fied as general purpo se. 1 96 8.5. Understanding an Nmap Fingerprint The vendor is the company which makes an OS or device. Examples are App l e , C i s co, Mi c r o s o f t , and Linksys. For community projects such a s OpenBSD and Linux without a controlling vendor, the OS family name is repeated for the vendor col umn. OS family includes products such as Wi ndows, L i nux, IOS (for Cisco routers), S o l a r i s , and OpenBSD. There are also hundreds of devices such as switches, broadband routers, and printers which use undisclosed operating systems. When the underlying OS isn't clear, embedded is used. OS generation is a more granular description of the OS. Generations of Linux i nclude 2 . 4 x and 2 . 6 . x, while Windows generations include 9 5 , 9 8 , Me, 2 0 0 0 , XP, and V i s t a. FreeBSD uses generations such as 4 . X and 5 . x. For obscure operating systems which we haven't subdivided i nto generations (or whenever the OS is listed simply as embedded), this field is left blank. . Each field may contain just one value. When a fingerprint represents more than one possible combination of these four fields, multiple C l a s s lines are used. Example 8.6 provides some example F i ngerp r i nt lines followed by their corresponding classifications. Example 8.6. Some typical fingerprint descriptions and corresponding classifications Fingerprint D-Link DSL-500G ADSL router Class D-Link I embedded I I broadband router Fingerprint Linksys WRT54GC or TRENDnet TEW-431BRP WAP Class Linksys I embedded I I WAP Class TRENDnet I embedded I I WAP Fingerprint Apple Mac OS X 1 0 . 3 . 9 (Panther ) - 1 0 . 4 . 7 (Tiger ) Class Apple I Mac OS X I 1 0 . 3 . X I general purpose Class Apple I Mac OS X I 1 0 . 4 . X I general purpose Fingerprint Sony PlayStation 3 game console Class Sony I embedded I I game console I f these examples aren't enough, a listing o f classifications recognized b y the latest version o f Nmap is maintained at http://nmap.org/data/os-classes. txt. Test expressions The test expressions don't have to change between a subject and reference fingerprint, but they almost always do. The reference fingerprint often needs to be generalized a little bit to match all instances of a particular OS, rather than just the machine you are scanning. For example, some Wi ndows XP machines return a Window size of F 4 2 4 to the T 1 probe, while others return FAF 0 . This may be due to the particular ethernet device driver in use, or maybe how much memory is available. In any case, we would like to detect Windows XP no matter which window size is used. One way to generalize a fingerprint is to simply remove tests that produce i nconsistent results. Remove all of the window size tests from a reference fingerprint, and systems will match that print no matter what size they use. The downside is that you can lose a lot of important i nformation this way. If the only Window sizes that a particular system ever sends are F 4 2 4 and FAF O, you really only want to allow those two values, not all 65,536 possibilities. 8.5. Understanding an Nmap Fingerprint 1 97 While removing tests is overkill in some situations, it is useful in others. The R=Y test value, meaning there was a response, is usually removed from the U l and I E tests before they are added to nmap-os -db. These probes are often blocked by a firewall, so the lack of a response should not count against the OS match. When removing tests is undesirable, Nmap offers an expression syntax for allowing a test to match multiple values. For example, W=F 4 2 4 I FAF 0 would allow those two Windows XP window values without al lowing any others. Table 8.1 1 shows the permitted operators in test values. Table 8.11. Reference fingerprint test expression operators Op Name Symbol Example Description Or I O= I ME I MNNTNW Matches if the corresponding subject fingerprint test takes the value of any of the clauses. In this example, the initial pipe symbol means that an empty options list will match too. Range - SP= 7 -A Matches if the subject fingerprint's corresponding test produces a numeric value which falls within the range specified. Greater than > SP=> 8 Matches if the subject fingerprint's corresponding test produces a numeric value which is greater than the one specified. Less than GCD= < S Matches if the subject fingerprint's corresponding test produces a numeric value which is less than the one specified. < Expressions can combine operators, as in GC D=< 7 I 6 4 I 2 5 6 I > 1 0 2 4 , which matches if the GCD is than seven, exactly 64, exactly 256, or greater than 1 024. less ·a.6. OS Matc h i ng Algorith ms Nmap's algorithm for detecting matches is relatively simple. It takes a subject fingerprint and tests it against every single reference fingerprint in nmap - o s -db. When testing against a reference fi ngerprint, Nmap looks at each probe category line from the subject fingerprint (such as SEQ or T 1 ) in turn. Any probe lines which do not exist in the reference fingerprint are skipped. When the reference fi ngerprint does have a matching line, they are compared. For a probe line comparison, Nmap examines every individual test (R, OF, w, etc.) from the subject category line in turn. Any tests which do not exist i n the reference line are skipped. Whenever a matching test is found, Nmap increments the P o s s i b l ePo i n t s accumulator by the number of points assigned to this test. Then the test values are compared. If the reference test has an empty value, the subject test only matches if its value is empty too. If the reference test is j ust a plain string or number (no operators), the subject test must match it exactly. If the reference string contains operators ( I , -, >, or <), the subject must match as described in the section called "Test expressions" [ 1 97]. If a test matches, the NumMa t c hP o i n t s accumulator is incremented by the test's point value. 198 8.6. OS Matching Algorithms Once all of the probe lines are tested for a fingerprint, Nmap divides NumMa t chPo i n t s by Pos s iblePo int s . The result is a confidence factor describing the probabi lity that the subject fi ngerprint matches that particular reference fi ngerprint. It is treated as a percentage, so 1 0 0 is a perfect match while 0 . 9 5 is very close. . Test point values are assigned by a special Mat c hP o i n t s entry (which may only appear once) in nmap-os -db. This entry looks much like a normal fingerprint, but instead of providing results for each lest, it provides point values (non-negative integers) for each test. Tests listed in the Mat ch P o i n t s structure only apply when found in the same test they are listed in. So a value given for the W (Window size) test in Tl doesn't affect the w test in T 3 . An example Mat chPo i nt s structure is given in Example 8.7. Example 8.7. The MatchPoints structure tchPoints Q(SP=25%GCD=75%ISR=25%TI=l00%I I=l00%SS=80%TS=l00 ) (01=20%02=20%03=20%04=20%05=20%06=20 ) (Wl=l5%W2=15%W3=15%W4=15%W5=15%W6=15 ) (R•l00%DF=20%T=l5%TG=l5%W=l5%0=15%CC=l00%Q=20 ) (R=100%DF=20%T=l5%TG=l5%S=20%A=20%F=30%RD=20%Q=20 ) (R=80%DF=20%T=1 5%TG=l5%W=25%S=20%A=20%F=30%0=10%RD=20%Q=20 ) (R=80%DF=20%T=l5%TG=l5%W=25%S=20%A=20%F=30%0=10%RD=20%Q=20 ) • , where < t a rget > is the misidentified system in question. Look at the OS detection results to ensure that the misidentification is still present. If the Nmap output for the host OS results says ( JU S T GUE S S I NG ) , it is expected that results may be a little off. Don't submit a correction in this case. Otherwise, the map command should have produced results including the line OS f i ngerp r i nt : . Below that is the fingerprint (a series of lines which each start with OS : ). Check that OS detection works against other hosts Try scanning a couple other hosts on the target network which you know have a different OS. If they aren't detected properly, maybe there is some network obstruction between the systems which is corrupting the packets. If you have gotten this far and are still able to submit, good for you ! Please submit the information at http://insecure. orglcgi-binlsubmit. cgi ?corr-os When Nmap Fai ls to Find a Match and Pri nts a Fingerprint 8.7.2. When Nmap detects that OS detection conditions seem ideal and yet it finds no exact matches, it will print out a message like this: OS matches for host ( If you know what OS is running on it, see tp: //nmap . org/submit/ ) . P/IP fingerprint : :SCAN (V=4 . 62%D=5/20%0T=21%CT=1%CU=42293%PV=Y%DS=1%G=Y%M=008077%TM=48336D6 :D%P=i686-pc-l inux-gnu) S EQ ( SP=l1 %GCD=l E 848%ISR=A4%TI=I% I I=I%SS=S%TS=A) OPS : (Ol=MSB4NWONNSNNT11%02=M578NWONNSNNT11%03=M280NWONNT11%04=M5B4NWONNSNNT1 : 1%05=M218NW0NNSNNT11 %06=Ml 09NNSNNTll ) WIN (W1=21F0%W2=2088%W3=2258%W4=21FO : %W5=20C0%W6=209D) ECN (R=Y%DF=N%T=40%W=2238%0=M5B4NWONNS%CC=N%Q=) Tl (R=Y%DF :=N%T=40%S=O%A=S+%F=AS%RD=0%Q=) T2 (R=N ) T3 (R=Y%DF=N%T=40%W=209D%S=O%A=S+%F= :AS%0=Ml09NWONNSNNT1 1%RD=0%Q=) T4 (R=Y%DF=N%T=40%W=0%S=A%A=Z%F=R%0=%RD=0%Q= : ) T5 (R=Y%DF=N%T=40%W=0%S=Z%A=S+%F=AR%0=%RD=0%Q= ) T6 (R=Y%DF=N%T=40%W=0%S=A% :A=Z%F=R%0=%RD=0%Q= ) T7 ( R=Y%DF=N%T=4 0%W=0%S=Z%A=S+%F=AR%0=%RD=0%Q= ) Ul ( R=Y% :DF=N%T=FF%TOS=0%IPL=38%UN=0%RIPL=G%RID=G%RIPCK=G%RUCK=G%RUL=G%RUD=G) I E ( R :•Y%DFI=N%T=FF%TOSI=Z%CD=S%SI=S%DLI=S ) O e consider submitting the fingerprint so that all Nmap users can benefit. It only takes a minute or two it may mean you don't need to see that ugly message again when you scan the host with the next Nmap ion ! Simply visit the URL Nmap provides for instructions. Nmap finds no matches and yet prints no fingerprint, conditions were not ideal . Even if you obtain the gerprint through debug mode or XML output, please don't submit it unless Nmap asks you to (as in the ious example). 8.7. Deal ing with Misidentified and Unidentified Hosts 201 8. 7.3. Mod ifying the runap-os-db Database Yourself People often ask about integrating a fingerprint themselves rather than (or in addition to) submitting it to Nmap.Org. While we don't offer detailed instructions or scripts for this, it is certainly possible after you become intimately familiar with Section 8.5, "Understanding an Nmap Fingerprint" [ 1 9 1 ]. I hope this is useful for your purposes, but there is no need to send your own reference fingerpri nt creations to us. We can only integrate raw subject fingerprint submissions from the web form. 8.8. SOLUTION : Detect Rog ue Wi reless Access Poi nts on an Enterprise Network 8.8.1 . Problem With the ubiquity o f mobile devices and cheap commodity networking equipment, companies are increasingly finding that employees are extending their networks in undesirable ways. Among the most dangerous devices are 802. 1 1 wireless access points (WAPs). Users may install a $20 WAP in their cubicle so they can work from the break room, without realizing (or caring) that they just opened the protected corporate network to potential attackers in the parking lot or nearby buildings. Some WAP installations are even worse than those installed by naive users. Breaching a building's security is much riskier for an attacker than accessing corporate data from far away through a network. It carries the risk of being arrested on the spot. So attackers have been known to install compact WAPs so they can then intrude on the network at will from the relative safety of a car down the street. A WAP taped under a desk or otherwise hidden is unlikely to be noticed for a while. · While the focus of this solution is finding WAPs, the same strategy can be used to find just about anything. You might need to locate all Cisco routers to apply a new patch or Solaris boxes to determine whether you have enough systems to warrant paying for support. 6 One way to fi nd unauthorized wireless devices is to sweep the area with a wireless sniffer such as Kismet 7 or NetStumbler . Another approach is to scan the wired side with Nmap. Not surprisingly, this solution focuses exclusively on the latter approach. Each technique can miss certain WAPs, so the best approach is to do both and merge the results. 8.8.2. Solution Scan your whole address space using the - A option. You can speed i t u p by limiting scanned ports to 1 -85, 1 1 3, 443, and 8080-8 100. Those should find both an open and closed port on most WAPs, which improves OS detection accuracy. If your network spans multiple ethernet segments, scan each segment from a designated machine on the same segment. This speeds up the scan (especially since you can do them in parallel), and also gives you the MAC address of each device. Scanning from the same segment also allows you to spot stealth devices. Even a WAP with all ports filtered will generally respond to an ARP request. Results should be saved in at least normal and XML formats, so you might as well use -oA. Consider all of the 6 7 http://www.kismetwireless.net/ http://www.netstumbler.com/ 202 8.8. SOLUTION: Detect Rogue Wireless Access Points on an Enterprise Network performance-enhancing options described in Chapter 6, Optimizing Nmap Performance [ 1 35). A good and relatively safe start for performance options is - T 4 - -mi n-host group 5 0 - -max - r t t - t imeout 1 0 0 0 --i n i t i a l - r t t -t imeout 300 - -max-ret r i e s 3 --max- scan -de l ay 1 0 0 0 . Put this all together for a command like: - - ho s t - t imeout 2 0m nmap -A -oA -/nmap-logs/wapscan -p 1-85,113,443,8080-8100 -T4 --min-hostgroup 50 --max-rtt-timeout 1000 --initial-rtt-timeout 300 --max-retries 3 --host-timeout 20m --max-scan-delay 1000 When the scan completes, search for WAP characteristics. On a network of fewer than a couple hundred live hosts, your best bet is to look at each one i ndividually. For larger networks, you will likely need to automate the task. Searching for individual characteristics can be done with grep, though a Perl script which analyzes the XML output is preferable. This is pretty easy thanks to existing modules, such as Nmap::Scanner and Nmap::Parser, for parsing Nmap XML output. See Section 13.7, "Manipulating XML Output with Perl" [352) for examples. Once you determine a list of candidates, it is probably best to open the normal Nmap output file and examine each one to eliminate false positives. For example, a Linksys device may be flagged as a possible WAP even though it could be one of their plain switches without any wireless functionality. Once you find the WAPs, it is time to track them down. This can usually be done by querying the switch they connect to for their physical ethernet port number. 8.8.3. WAP Characteristics Now it is time to discuss the WAP characteristics to look for. Understanding these is useful for manual inspections or for modifying the WAP finder script to search for something else. You will probably see many of them immediately by looking at the scan of a typical WAP in Example 8.8. Example 8.8. Scan results against a consumer WAP -A -v wap . nmap . org Starting Nmap ( http : //nmap . org Interesting ports on wap . nmap . org ( 192 . 16 8 . 0 . 6 ) : Not shown : 999 closed ports PORT STATE SERV ICE VERSION 80/tcp open http Netgear MR-series WAP (MR814; Embedded HTTPD 1 . 00 ) MAC Address : 00 : 09 : 5B : 3F : 7D : 5E (Netgear ) Device type : WAP Running : Compaq embedded, Netgear embedded OS details : WAP : Compaq iPAQ Connection Point or Netgear MR81 4 Service Info : Device : WAP Nmap done : 1 IP address ( 1 host up) scanned in 1 0 . 90 seconds Raw packets sent : 1 703 ( 75 . 706KB) I Rcvd : 1686 ( 7 7 . 552KB) nmap This device shows many obvious clues to being a WAP (Device t ype : WAP is pretty blatant) and some more subtle ones. But WAPs aren't always so easy to discover. This section provides a list of WAP characteristics, starting with the most powerful and ending with heuristics that are long shots or more likely 8.8. SOLUTION: Detect Rogue Wireless Access Points on an Enterprise Network 203 to produce false positives. Each characteristic listed is accompanied by an XPath8 expression that shows where to find it in Nmap XML output. Since this is security related, I suggest trying all of them and removing false positives manually. TCP/IP fi ngerprinting device type As described in the section cal led "Device and OS classification (Class lines)" [ 1 96), every reference fi ngerprint has at least one classification (which incl udes device type) associated with it. Because WAPs are so controversial, we try to use that (or give two classifications) when multiple types would fit. So devices like the D-Link DI-624 wireless broadband router is classified as WAP rather than switch or r o u t e r . The device type can be found in XML output using the XPath expression / nmapr u n / h o s t / o s / o s c l a s s / @ t ype. (That is, the t ype attribute of the o s c l a s s element of the o s element of any of the h o s t elements inside the root nmaprl..i n element). TCP/IP fingerprinting details While devices with Wireless capability should be classified as device type WAP, it is worth searching the detailed OS description for terms such as w i re l e s s or wap just to be sure. The description is in / nmapr u n / ho s t / o s / o smat ch / @ name in XML output. Version detection device type Version detection also tries to determine device types, but by fingerprinting the target's running services rather than its IP stack. Check whether the XML dev i c e t ype attribute located at / nmapr u n / h o s t /po r t s / po r t / serv i c e / @de v i c e t ype is WAP. To be completely safe, checking the / nmaprun / h o s t /por t s /port / s erv i ce / @ ex t r a i n f o field for the substrings wap or w i r e l e s s is worthwhile. Vendor (from MAC address, TCP/IP fi ngerprinting, and version detection) Certain vendors specialize in producing the low-cost consumer networking devices which are most likely to covertly find their way onto office network s Examples are Linksys, Netgear, Belkin, SMC, D-Link, Motorola, Trendnet, Zyxel, and Gateway. You can check for these vendors based on the MAC address lookup (which is at / nmapr u n / h o s t / addre s s / @ vendor in XML output), OS detection in XML output), or version detection ( / nmapru n / ho s t / o s / o s c l a s s / @ vendor in XML output) results. Be sure to search @product / e ( / nmaprun / ho s t /port s /port I s e r v i c incorporation type (e.g. Inc.) or contain may field for the vendor as a substring of the fields, since the other information. . This test may lead to many false positives. If you use a vendor heavily for authorized devices, such as putting Netgear NICs in your desktop machines, you may have to remove that vendor and rerun the script. Hostname It doesn't hurt to check hostnames (reverse DNS resolution) for terms such as wap, wireless, or a i rp o r t . These can be found at / nmaprun / h o s t / ho s t n ame s / h o s t name / @ name in XML output. Non-administrative employees rarely change DNS names, but this can be useful for pen-testers, new administrators, and others who may be scanning a new network looking for authorized access points. 8 h11p:llwww. w3.org!TR/xpath 204 8.8. SOLUTION: Detect Rogue Wireless Access Points on an Enterprise Network Chapter 9. Nmap Scri pt ing Eng ine 9.1 . Introduction The Nmap Scripting Engine (NSE) i s one of Nmap's most powerful and flexible features. It allows users to write (and share) simple scripts to automate a wide variety of networking tasks. Those scripts are then executed in parallel with the speed and efficiency you expect from Nmap. Users can rely on the growing and diverse set of scripts distributed with Nmap, or write their own to meet custom needs. We designed NSE to be versatile, with the following tasks in mind: Network discovery This is Nmap's bread and butter. Examples i nclude looking up whois data based on the target domain, querying ARIN, RIPE, or APNIC for the target IP to determine ownership, performing identd lookups on open ports, SNMP queries, and listing available NFS/SMB/RPC shares and services. More sophisticated version detection The Nmap version detection system (Chapter 7, Service and Application Version Detection [ 1 45]) is able to recognize thousands of di fferent services through its probe and regular expression signature based matching system, but it cannot recognize everything. For example, identifying the Skype v2 service requires two independent probes, which version detection isn't flexible enough to handle. Nmap could. also recognize more SNMP services if it tried a few hundred different community names by brute force. Neither of these tasks are well suited to traditional Nmap version detection, but both are easily accomplished with NSE. For these reasons, version detection now calls NSE by default to handle some tricky services. This is described in Section 9.10, "Version Detection Using NSE" [25 1 ] . Vulnerability detection When a new vulnerability is discovered, you often want to scan your networks quickly to identify vulnerable systems before the bad guys do. While Nmap isn't a comprehensive vulnerability scanner, NSE is powerful enough to handle even demanding vulnerability checks. Many vulnerability detection scripts are already available and we plan to distribute more as they are written. Backdoor detection Many attackers and some automated worms leave backdoors to enable later reentry. Some of these can be detected by Nmap's regular expression based version detection. For example, within hours of the MyDoom worm hitting the Internet, Jay Moran posted an Nmap version detection probe and signature so that others could quickly scan their networks for MyDoom infections. NSE is needed to reliably detect more complex worms and backdoors. Vulnerability exploitation As a general scripting language, NSE can even be used to exploit vulnerabilities rather than just find them. The capability to add custom exploit scripts may be valuable for some people (particularly penetration testers), though we aren't planning to turn Nmap i nto an exploitation framework such as Metasploi t 1 • 1 htrp:l!www.metasploit.com 9. 1. Introduction 205 These listed items were our initial goals, and we expect Nmap users to come up with even more inventive uses for NSE. Scripts are written in the embedded Lua programming language2 . The language itself is well documented in the books Programming in Lua, Second Edition and Lua 5 .1 Reference Manual. The reference manual is also freely available online3 , as is the first edition of Programming in Lua4. Given the availability of these excellent general Lua programming references, this document only covers aspects and extensions specific to Nmap's scripting engine. NSE is activated with the - s c option (or - - s cr ipt if you wish to specify a custom set of scripts) and results are integrated into Nmap normal and XML output. Two types of scripts are supported: service and host scripts. Service scripts relate to a certain open port (service) on the target host, and any results they produce are included next to that port i n the Nmap output port table. Host scripts, on the other hand, run no more than once against each target IP and produce results below the port table. Example 9 . 1 shows a typical script scan. Service scripts producing output i n this example are s sh-hos tkey, which provides the system's RSA and DSA SSH keys, and rpc i n f o, which queries portmapper to enumerate available services. The only host script producing output in this example is smb-os-d i s covery, which collects a variety of information from SMB servers. Nmap discovered all of this information in a third of a second. Example 9.1. Typical NSE output nmap -sc -p22 , ll l , 139 -T4 localhost Starting Nmap ( http : //nmap . org ) Interesting ports on flog ( 1 27 . 0 . 0 . 1 ) : PORT STATE S ERV ICE 22/tcp open ssh I ssh-hostkey : 1024 bl : 36 : 0d : 3f : 50 : dc : l3 : 96 : b2 : 6e : 3 4 : 39 : 0d : 9b : la : 38 (DSA) I _ 2048 77 : d0 : 20 : lc : 44 : 1f : 87 : a0 : 3 0 : aa : 85 : cf : e8 : ca : 4c : l l (RSA) 111/tcp open rpcbind .1 rpcinfo : 1 00000 2 , 3 , 4 1 1 1/udp rpcbind I I 100024 1 56454/udp status I _ 100000 2 , 3, 4 1 1 1 /tcp rpcbind 139/tcp open netbios-ssn Host script results : I smb-os-discovery : Unix I LAN Manager : Samba 3 . 0 . 31-0 . fc8 I _ Name : WORKGROUP Nmap done : 1 IP address ( 1 host up) scanned in 0 . 33 seconds # 9.2. Usage and Examples While NSE has a complex implementation for efficiency, it is strikingly easy to use. Simply specify -sc to enable the most common scripts. Or specify the - - s cr i p t option to choose your own scripts to execute 2 3 http://www.lua.org/ http://www.lua.orglmanual/5.J/ 4 http://www.lua.org/pil/ 206 9.2. Usage and Examples providing categories, script file names, or the name of directories ful l of scripts you wish to execute. You customize some scripts by providing arguments to them via the - - s cr i p t - args option. The two ·ning options, - - s c r ip t - t r a c e and - - s cr ipt -updat edb, are generally only used for script gging and development. Script scanning is also included as part of the -A (aggressive scan) option . . 2.1 . Script Categories E scripts define a list of categories they belong to. Currently defined categories are a u t h, de f a u l t , iscovery, ext e r n a l , i n t r u s ive, malware, s a fe, ver s i on, and vuln. Category names are IOl case sensitive. The following list describes each category. These scripts try to determine authentication credentials on the target system, often through a brute-force attack. Examples include s nmp-brut e , http-auth, and ftp-anon. These scripts are the default set and are run when using the -sc or -A options rather than listing scripts with --sc r ipt. This category can also be specified explicitly like any other using --script=de f a u l t . Many factors are considered in deciding whether a script should be run by default: Speed A default scan must finish quickly, which excludes brute force authentication crackers, web spiders, and any other scripts which can take minutes or hours to scan a single service. Usefulness Default scans need to produce valuable and actionable information. If even the script author has trouble explaining why an average networking or security professional would find the output valuable, the script should not run by default. The script may still be worth including in Nmap so that administrators can run for those occasions when they do need the extra information. Verbosity Nmap output is used for a wide variety of purposes and needs to be readable and concise. A script which frequently produces pages full of output should not be added to the d e f a u l t category. When there is no important information to report, NSE scripts (particularly default ones) should return nothing. Checking for an obscure vulnerability may be OK by default as Jong as it only produces output when that vulnerability discovered. Reliability Many scripts use heuristics and fuzzy signature matching to reach conclusions about the target host or service. Examples include s n i f fe r -detect and s ql - i n j e ct i o n . If the script is often wrong, it doesn't belong in the de f a u l t category where it may confuse or mislead casual users. Users who specify a script or category directly are generally more advanced and likely know how the script works or at least where to find its documentation. Intrusiveness Some scripts are very intrusive because they use significant resources on the remote system, are likely to crash the system or service, or are likely to be perceived as an attack by the remote administrators. The more intrusive a script is, the less suitable it is for the d e f a u l t category. 9.2. Usage and Examples 207 Privacy Some scripts, particularly those in the external category described later, divulge information third parties by their very nature. For example, the whoi s script must divulge the target IP add to regional whois registries. We have also considered (and decided against) adding scripts whi check target SSH and SSL key fingerprints against Internet weak key databases. The mom privacy-invasive a script is, the less suitable it is for d e fau l t category inclusion. We don't have exact thresholds for each of these criteria, and many of them are subjective. All of these factors are considered together when making a decision whether to promote a script into the default category. A few default scripts are ident d-owner s (determines the username running remote services using identd), h t t p -auth (obtains authentication scheme and realm of web sites requiring authentication), and ftp-anon (tests whether an FfP server allows anonymous access). d i s co ve r y These scripts try to actively discover more about the network by querying public registries, SNMP-enabled devices, directory services, and the like. Examples include html -t i t l e (obtains the title of the root path of web sites), smb-enum- shares (enumerates Windows shares), and snmp-sysdescr (extracts system details via SNMP). external Scripts in this category may send data to a third-party database or other network resource. An example of this is who i s , which makes a connection to whois servers to learn about the address of the target. There is always the possibility that operators of the third-party database will record anything you send to them, which in many cases will include your IP address and the address of the target. Most scripts involve traffic strictly between the scanning computer and the client; any that do not are placed in this category. i n t r u s ive These are scripts that cannot be classified i n the safe category because the risks are too high that they will crash the target system, use up significant resources on the target host (such as bandwidth or CPU time), or otherwise be perceived as malicious by the target's system administrators. Examples are h t t p-open-proxy (which attempts to use the target server as an HTTP proxy) and snmp-brute (which tries to guess a device's SNMP community string by sending common values such as publ ic, pr i vate, and c i s c o). ma lware These scripts test whether the target platform is infected by malware or backdoors. Examples i nclude smtp- s t r angeport, which watches for SMTP servers running on unusual port numbers, and a u t h - s p o o f , which detects identd spoofing daemons which provide a fake answer before even receiving a query. Both of these behaviors are commonly associated with mal ware infections. safe Scripts which weren't designed to crash services, use large amounts of network bandwidth or other resources, or exploit security holes are categorized as s a fe. These are less likely to offend remote administrators, though (as with all other Nmap features) we cannot guarantee that they won't ever cause adverse reactions. Most of these perform general network discovery. Examples are s s h-hostkey (retrieves an SSH host key) and html - t i t l e (grabs the title from a web page). 208 9.2. Usage and Examples versi on The scripts in this special category are an extension to the version detection feature and cannot be selected explicitly. They are selected to run only if version detection (- sV) was requested. Their output cannot be distinguished from version detection output and they do not produce service or host script results. Examples are skypev2 -ver s i on, pptp-vers i o n, and i ax 2 - ve r s i o n . vuln These scripts check for specific known vulnerabilities and generally only report results if they are found. Examples include realvnc-aut h-bypa s s and xampp-de f a u l t -auth. 9.2.2. Com mand-l i ne Argu ments These are the five command line arguments specific to script-scanning: -sc Performs a script scan using the default set of scripts. It is equivalent to - - s c r i p t =de f a u l t . Some of the scripts in this default category are considered intrusive and should not be run against a target network without permission. --script