No Starch Press Absolute Free BSD The Guide To 2nd Edition

Absolute%20FreeBSD-The%20%20Guide%20to%20FreeBSD%202nd%20Edition

User Manual: Pdf

Open the PDF directly: View PDF .
Page Count: 745

Download
Open PDF In Browser	View PDF

President of

F R E E BSD:
NOT JUST FOR
ALPHA GEEKS

the FreeBSD Foundation

ANYMORE!

With a foreword by

ROBERT WATSON,

• Use advanced security features like packet filtering,
virtual machines, and host-based intrusion detection
• Build custom live FreeBSD CDs and bootable flash
• Manage network services and filesystems
• Use DNS and set up email, IMAP, web, and FTP
services for both servers and clients
• Monitor your system with performance-testing and
troubleshooting tools
• Run diskless systems

“ I L AY F L AT .”
This book uses RepKover — a durable binding that won’t snap shut.
Printed on recycled paper

• Integrate FreeBSD-specific SNMP into your network
management system
Whether you’re just getting started with FreeBSD or
you’ve been using it for years, you’ll find this book to
be the definitive guide to FreeBSD that you’ve been
waiting for.
ABOUT THE AUTHOR

Michael W. Lucas is a network engineer and system
administrator responsible for a network that stretches
across the Western Hemisphere. He is the author of the
critically acclaimed Absolute OpenBSD, Cisco Routers
for the Desperate, and PGP & GPG, all from No Starch
Press. Despite being from Detroit, Michigan, he knows
almost nothing about automobiles. He has been using
Unix systems for over 20 years and FreeBSD since 1995.
Fortunately for the rest of us, his writing keeps him too
busy to implement his plans for world domination.

$59.95 ($65.95 CDN)
SHELVE IN:
OPERATING SYSTEMS/UNIX

w w w.nostarch.com

• Implement redundant disks, without special hardware

LUCAS

T H E F I N E ST I N G E E K E N T E RTA I N M E N T ™

• Build custom network appliances with embedded
FreeBSD

2
2N
ND
D EE D
D II TT II O
ON
N

Absolute FreeBSD, 2nd Edition is your complete guide
to FreeBSD, written by FreeBSD committer Michael
W. Lucas. Lucas considers this completely revised and
rewritten second edition of his landmark work to be his
best work ever; a true product of his love for FreeBSD
and the support of the FreeBSD community. Absolute
FreeBSD, 2nd Edition covers installation, networking,
security, network services, system performance, kernel
tweaking, filesystems, SMP, upgrading, crash debugging,
and much more, including coverage of how to:

• Manage schedulers, remap shared libraries, and
optimize your system for your hardware and your
workload

ABSOLUTE
FREEBSD

FreeBSD—the powerful, flexible, and free Unix-like
operating system—is the preferred server for many
enterprises. But it can be even trickier to use than either
Unix or Linux, and harder still to master.

ABSOLUTE

F R2 NED E
B
S
D
EDITION
T H E

COMPLETE GUIDE TO FREEBSD
MICH A EL W. LUC A S

®

PRAISE FOR THE FIRST EDITION, ABSOLUTE BSD
“Even longtime users of FreeBSD may be surprised at the power and features
it can bring to bear as a server platform, and Absolute BSD is an excellent guide
to harnessing that power.”
—UNIXREVIEW.COM
“ . . . provides beautifully written tutorials and reference material to help you
make the most of the strengths of this OS.”
—LINUXUSER & DEVELOPER MAGAZINE
“ . . . a great resource for people new to BSD and those who have been using
it for years. Michael Lucas has a writing style which is very easy to read and
absorb.”
—FRESHMEAT
“A very fine piece of work, it isn’t about how to implement BSD solutions, but
it is about managing systems in situ.”
–;LOGIN:
“ . . . packed with a lot of information.”
—DAEMON NEWS

PRAISE FOR ABSOLUTE OPENBSD BY MICHAEL LUCAS
“Absolute OpenBSD by Michael Lucas is a broad and mostly gentle introduction
into the world of the OpenBSD operating system. It is sufficiently complete
and deep to give someone new to OpenBSD a solid footing for doing real
work and the mental tools for further exploration. . . . The potentially boring
topic of systems administration is made very readable and even fun by the
light tone that Lucas uses.”
—CHRIS PALMER, PRESIDENT, SAN FRANCISCO O PENBSD USERS GROUP
“ . . . a well-written book that hits its market squarely on target. Those new to
OpenBSD will appreciate the comprehensive approach that takes them from
concept to functional execution. Existing and advanced users will benefit from
the discussion of OpenBSD-specific topics such as the security features and
pf administration.”
—SLASHDOT
“I recommend Absolute OpenBSD to all programmers and administrators
working with the OpenBSD operating system (OS), or considering it.”
—UNIXREVIEW.COM

PRAISE FOR PGP & GPG BY MICHAEL LUCAS
“PGP & GPG is another excellent book by Michael Lucas. I thoroughly enjoyed
his other books due to their content and style. PGP & GPG continues in this
fine tradition. If you are trying to learn how to use PGP or GPG, or at least
want to ensure you are using them properly, read PGP & GPG.”
—TAOSECURITY
“The world’s first user-friendly book on email privacy. Unless you’re a
cryptographer, or never use email, you should read this book.”
—LEN SASSAMAN, CODECON FOUNDER
“ Excellent tutorial, quick read, and enough humor to make it enjoyable.”
—INFOWORLD
“An excellent book that shows the end-user in an easy to read and often
entertaining style just about everything they need to know to effectively and
properly use PGP and OpenPGP.”
—SLASHDOT

PRAISE FOR CISCO ROUTERS FOR THE DESPERATE BY MICHAEL LUCAS
“ . . . this book isn’t a reference—it’s a survival guide, a ‘break glass in case
of emergency’ safety harness. . . . What I found remarkable was how it was
obviously written for people like me—those of us who have little interest in
router management but whose jobs depend on the consistent, trusted functioning of such infrastructure.
—ASP.NETPRO
“If only Cisco Routers for the Desperate had been on my bookshelf a few years
ago! It would have definitely saved me many hours of searching for configuration help on my Cisco routers. . . . I would strongly recommend this book
for both IT Professionals looking to get started with Cisco routers, as well as
anyone who has to deal with a Cisco router from time to time but doesn’t
have the time or technological know-how to tackle a more in-depth book
on the subject.”
—BLOGCRITICS MAGAZINE

ABSOLUTE

FREEBSD
2ND EDITION
THE COMPLETE GUIDE TO FREEBSD

by Mi chael W. L u cas

®

San Francisco

ABSOLUTE FREEBSD, 2ND EDITION. Copyright © 2008 by Michael W. Lucas.
All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or
mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior
written permission of the copyright owner and the publisher.
Printed on recycled paper in the United States of America
11 10 09 08 07

123456789

ISBN-10: 1-59327-151-4
ISBN-13: 978-1-59327-151-0
Publisher: William Pollock
Production Editors: Christina Samuell and Megan Dunchak
Cover and Interior Design: Octopod Studios
Developmental Editor: William Pollock
Technical Reviewer: John Baldwin
Copyeditor: Dmitry Kirsanov
Compositor: Riley Hoffman
Proofreader: Alina Kirsanova
Indexer: Nancy Guenther
For information on book distributors or translations, please contact No Starch Press, Inc. directly:
No Starch Press, Inc.
555 De Haro Street, Suite 250, San Francisco, CA 94107
phone: 415.863.9900; fax: 415.863.9950; info@nostarch.com; www.nostarch.com
Librar y of Congress Cataloging-in-Publication Data
Lucas, Michael, 1967Absolute FreeBSD : the complete guide to FreeBSD / Michael W. Lucas. -- 2nd ed.
p. cm.
Includes index.
ISBN-13: 978-1-59327-145-9
ISBN-10: 1-59327-145-X
1. FreeBSD. 2. UNIX (Computer file) 3. Internet service providers--Computer programs.
servers--Computer programs. 5. Client/server computing. I. Title.
QA76.76.O63L83 2007
004'.36--dc22
2007036190

4. Web

No Starch Press and the No Starch Press logo are registered trademarks of No Starch Press, Inc. Other product and
company names mentioned herein may be the trademarks of their respective owners. Rather than use a trademark
symbol with every occurrence of a trademarked name, we are using the names only in an editorial fashion and to the
benefit of the trademark owner, with no intention of infringement of the trademark.
The mark “FreeBSD” is a registered trademark of The FreeBSD Foundation and is used by Michael W. Lucas with the
permission of The FreeBSD Foundation.
The FreeBSD Logo is a trademark of The FreeBSD Foundation and is used by Michael W. Lucas with the permission
of The FreeBSD Foundation.
The BSD Daemon is copyright Marshall Kirk McKusick and is used with permission.
The information in this book is distributed on an “As Is” basis, without warranty. While every precaution has been
taken in the preparation of this work, neither the author nor No Starch Press, Inc. shall have any liability to any
person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the
information contained in it.

For Liz. With luck, this one is the right size to plug that dang gopher hole.

BRIEF CONTENTS
Foreword by Robert N.M. Watson ............................................................................... xxvii
Acknowledgments ....................................................................................................... xxix
Introduction ....................................................................................................................1
Chapter 1: Getting More Help ........................................................................................19
Chapter 2: Installing FreeBSD .........................................................................................33
Chapter 3: Start Me Up! The Boot Process........................................................................61
Chapter 4: Read This Before You Break Something Else! (Backup and Recovery)...................89
Chapter 5: Kernel Games ............................................................................................117
Chapter 6: The Network ..............................................................................................145
Chapter 7: Securing Your System ..................................................................................177
Chapter 8: Disks and Filesystems ..................................................................................209
Chapter 9: Advanced Security Features .........................................................................261
Chapter 10: Exploring /etc ..........................................................................................301
Chapter 11: Making Your System Useful ........................................................................315
Chapter 12: Advanced Software Management ...............................................................343
Chapter 13: Upgrading FreeBSD ..................................................................................371

Chapter 14: The Internet Road Map: DNS .....................................................................411
Chapter 15: Small System Services ...............................................................................439
Chapter 16: Spam, Worms, and Viruses (Plus Email, If You Insist) .....................................467
Chapter 17: Web and FTP Services...............................................................................499
Chapter 18: Disk Tricks with GEOM..............................................................................529
Chapter 19: System Performance and Monitoring ...........................................................569
Chapter 20: The Fringe of FreeBSD ...............................................................................603
Chapter 21: System (and Sysadmin) Panics and Crashes .................................................637
Afterword...................................................................................................................655
Appendix: Some Interesting sysctl MIBs..........................................................................661
Index .........................................................................................................................675

viii

B ri ef C on ten t s

CONTENTS IN DETAIL
F O R E W O R D B Y R O B E R T N. M . W AT SO N
A CK N O W LE D G M E N T S
I NT R O D U C T I O N

xxvii
xxix
1

What Is FreeBSD? .................................................................................................... 2
BSD: FreeBSD’s Granddaddy ....................................................................... 2
The BSD License .......................................................................................... 3
The AT&T/CSRG/BSDi Iron Cage Match ....................................................... 4
The Birth of FreeBSD .................................................................................... 4
FreeBSD Development ............................................................................................... 5
Committers ................................................................................................. 5
Contributors ................................................................................................ 6
Users ......................................................................................................... 7
Other BSDs ............................................................................................................. 7
NetBSD ...................................................................................................... 7
OpenBSD ................................................................................................... 7
Mac OS X .................................................................................................. 8
FreeBSD’s Children ...................................................................................... 8
Other Unixes ........................................................................................................... 8
Solaris/OpenSolaris .................................................................................... 8
AIX ............................................................................................................ 9
Linux .......................................................................................................... 9
IRIX, HP/UX, and So On .............................................................................. 9
FreeBSD’s Strengths ................................................................................................ 10
Portability ................................................................................................. 10
Power ...................................................................................................... 10
Simplified Software Management ................................................................ 10
Optimized Upgrade Process ....................................................................... 11
Advanced Filesystem ................................................................................. 11
Who Should Use FreeBSD? ..................................................................................... 11
Who Should Run Another BSD? ............................................................................... 11
Who Should Run a Proprietary Operating System? ..................................................... 12
How to Read This Book ........................................................................................... 12
What Must You Know? ........................................................................................... 12
For the New System Administrator ............................................................................ 13
Desktop FreeBSD ....................................................................................... 13
How to Think About Unix ........................................................................... 14
Notes on the Second Edition .................................................................................... 16
Contents of This Book ............................................................................................. 16

1
G E T T I NG M O R E H E L P

19

Why Not Just Email for Help? .................................................................................. 20
The FreeBSD Attitude ................................................................................. 20
Support Options ........................................................................................ 20

Man Pages ............................................................................................................ 21
Manual Sections ....................................................................................... 22
Navigating Man Pages .............................................................................. 23
Finding Man Pages ................................................................................... 23
Section Numbers and Man ......................................................................... 24
Man Page Contents ................................................................................... 24
FreeBSD.org .......................................................................................................... 25
Web Documents ....................................................................................... 25
The Mailing List Archives ............................................................................ 26
Other Websites ...................................................................................................... 26
Using FreeBSD Problem-Solving Resources ................................................................. 26
Checking the Handbook/FAQ .................................................................... 27
Checking the Man Pages ........................................................................... 27
Checking the Mailing List Archives .............................................................. 28
Using Your Answer .................................................................................... 28
Emailing for Help ................................................................................................... 29
Writing Your Email .................................................................................... 29
Sending Your Email ................................................................................... 30
Responding to Email .................................................................................. 31
Email Is Forever ........................................................................................ 31

2
I NS T AL L IN G F R E E B S D

33

FreeBSD Hardware ................................................................................................ 34
Sample Hardware ..................................................................................... 35
Proprietary Hardware ................................................................................ 35
What We Won’t Cover ............................................................................. 36
Hardware Requirements ............................................................................. 36
Preinstall Decisions ................................................................................................. 37
Partitioning ............................................................................................... 37
Multiple Hard Drives .................................................................................. 40
Partition Block Size .................................................................................... 41
Choosing Your Distribution(s) ...................................................................... 42
The FreeBSD FTP Site .............................................................................................. 43
FTP Server Content .................................................................................... 43
The Install Process .................................................................................................. 45
Choosing Boot Media ................................................................................ 45
Choosing Installation Media ....................................................................... 46
Preparing Boot Floppies .......................................................................................... 47
Preparing Boot CDs ................................................................................................ 47
FTP Media Setup .................................................................................................... 48
Actually Installing FreeBSD ...................................................................................... 49
Configuring the Network ............................................................................ 54
Miscellaneous Network Services ................................................................. 56
Time Zone ................................................................................................ 56
Linux Mode .............................................................................................. 56
PS/2 Mouse ............................................................................................. 56
Adding Packages ...................................................................................... 56
Adding Users ............................................................................................ 57
Root Password .......................................................................................... 58
Post-Installation Setup ................................................................................. 58
Restart! ................................................................................................................. 59
x

C on t en ts in D et ai l

3
S T A R T M E U P ! T H E B O O T P R O CE S S

61

Power-On and the Loader ........................................................................................ 62
Single-User Mode ................................................................................................... 63
Disks in Single-User Mode .......................................................................... 64
Programs Available in Single-User Mode ...................................................... 64
The Network in Single-User Mode ............................................................... 65
Uses for Single-User Mode ......................................................................... 65
The Loader Prompt ................................................................................................. 66
Default Files ........................................................................................................... 68
Loader Configuration .............................................................................................. 69
Serial Consoles ...................................................................................................... 70
Hardware Serial Consoles .......................................................................... 71
Software Serial Consoles ........................................................................... 71
Serial Console Physical Setup ..................................................................... 73
Serial Console Use .................................................................................... 73
Serial Console Disconnection ...................................................................... 75
Startup Messages ................................................................................................... 76
Multi-User Startup ................................................................................................... 79
/etc/rc.conf and /etc/defaults/rc.conf ........................................................ 79
The rc.d Startup System .............................................................................. 87
Shutdown ................................................................................................. 88

4
R E A D T H I S B E F O R E Y O U B R E A K S O M E T HI N G E L S E !
( B A CK U P A ND R E C O VE R Y )

89

System Backups ..................................................................................................... 90
Backup Tapes ........................................................................................................ 90
Tape Drive Device Nodes, Rewinding, and Ejecting ...................................... 91
The $TAPE Variable ................................................................................... 91
Tape Status with mt(1) ................................................................................ 92
Other Tape Drive Commands ..................................................................... 93
To Rewind or Not? .................................................................................... 93
Backup Programs ................................................................................................... 94
tar ........................................................................................................................ 94
tar Modes ................................................................................................ 94
Other tar Features ..................................................................................... 96
gzip ........................................................................................................ 97
dump .................................................................................................................... 98
User Control ............................................................................................. 98
dump Levels .............................................................................................. 98
dump, Tape Drives, and Files ...................................................................... 99
dump and Live Filesystems .......................................................................... 99
Timestamps and dump ............................................................................. 100
Running dump ........................................................................................ 100
Throwing Data Overboard with nodump .................................................... 101
Restoring from a dump .......................................................................................... 101
Checking the Contents of an Archive ......................................................... 101
Restoring dump Data ............................................................................... 102
Multiple Backups on One Tape .............................................................................. 105
C on t en ts in D et ail

xi

Revision Control ................................................................................................... 106
Initializing Revision Control ...................................................................... 107
Editing Files in RCS ................................................................................. 108
Checking Back In .................................................................................... 108
Viewing RCS Logs ................................................................................... 109
Reviewing a File’s Revision History ............................................................ 110
Getting Older Versions ............................................................................ 111
Breaking Locks ........................................................................................ 112
Recording What Happened ................................................................................... 114
The Fixit Disk ....................................................................................................... 114

5
K E R N E L G AM E S

117

What Is the Kernel? .............................................................................................. 118
sysctl .................................................................................................................. 119
sysctl MIBs .............................................................................................. 120
sysctl Values ........................................................................................... 121
Viewing sysctls ........................................................................................ 121
Changing sysctls ..................................................................................... 122
Kernel Modules .................................................................................................... 124
Viewing Loaded Modules ......................................................................... 124
Loading and Unloading Modules .............................................................. 125
Loading Modules at Boot ......................................................................... 125
Build Your Own Kernel ......................................................................................... 126
Preparations ........................................................................................... 126
Buses and Attachments ............................................................................ 127
Back Up Your Working Kernel .................................................................. 128
Configuration File Format ......................................................................... 128
Configuration Files .................................................................................. 129
Trimming a Kernel ................................................................................................ 131
CPU Types ............................................................................................. 131
Basic Options ......................................................................................... 131
Multiple Processors .................................................................................. 134
Device Drivers ........................................................................................ 134
Pseudodevices ........................................................................................ 135
Removable Hardware .............................................................................. 136
Building a Kernel ................................................................................................. 136
Troubleshooting Kernel Builds ................................................................... 137
Booting an Alternate Kernel ...................................................................... 137
Inclusions, Exclusions, and Expanding the Kernel ..................................................... 138
NOTES .................................................................................................. 138
Inclusions and Exclusions ......................................................................... 139
How Kernel Options Fix Problems ............................................................. 139
Sharing Kernels .................................................................................................... 140
Testing Kernels Remotely ....................................................................................... 141
Kernel Stuff You Should Know ................................................................................ 142
ACPI ...................................................................................................... 142
PAE ....................................................................................................... 142
Symmetric Multiprocessing ....................................................................... 143
Lock Order Reversals ............................................................................... 143

xii

C on te nt s i n De ta il

6
THE NETWORK

145

Network Layers .................................................................................................... 146
The Physical Layer ................................................................................... 146
Datalink: The Physical Protocol .................................................................. 146
The Network Layer .................................................................................. 147
Heavy Lifting: The Transport Layer ............................................................. 147
Applications ........................................................................................... 148
The Network in Practice ........................................................................................ 148
Getting Bits and Hexes ......................................................................................... 150
Remedial TCP/IP .................................................................................................. 152
IP Addresses and Netmasks ...................................................................... 152
ICMP ..................................................................................................... 155
UDP ....................................................................................................... 155
TCP ....................................................................................................... 156
How Protocols Fit Together ....................................................................... 157
Transport Protocol Ports ............................................................................ 157
Understanding Ethernet ......................................................................................... 158
Protocol and Hardware ............................................................................ 159
Ethernet Speed and Duplex ...................................................................... 160
MAC Addresses ...................................................................................... 160
Configuring Your Ethernet Connection .................................................................... 161
ifconfig(8) .............................................................................................. 161
Adding an IP to an Interface ..................................................................... 162
Testing Your Interface .............................................................................. 163
Set Default Route ..................................................................................... 163
Multiple IP Addresses on One Interface ...................................................... 163
Renaming Interfaces ................................................................................ 164
DHCP .................................................................................................... 165
Reboot! .................................................................................................. 166
Network Activity .................................................................................................. 166
Current Network Activity .......................................................................... 166
What’s Listening on What Port? ................................................................ 167
Port Listeners in Detail .............................................................................. 168
Network Capacity in the Kernel ................................................................ 169
Optimizing Network Performance .......................................................................... 170
Optimizing Network Hardware ................................................................. 170
Memory Usage ....................................................................................... 171
Maximum Incoming Connections ............................................................... 173
Polling ................................................................................................... 174
Changing Window Size .......................................................................... 174
Other Optimizations ................................................................................ 175
Network Adapter Teaming .................................................................................... 175
Aggregation Protocols ............................................................................. 175
Configuring lagg(4) ................................................................................. 176

7
S E CU R I N G Y O U R S Y S T E M

177

Who Is the Enemy? .............................................................................................. 178
Script Kiddies ......................................................................................... 178
Botnets ................................................................................................... 179
C o nt en t s in D et ai l

xiii

Disaffected Users .................................................................................... 179
Motivated Skilled Attackers ...................................................................... 179
FreeBSD Security Announcements ........................................................................... 180
User Security ....................................................................................................... 181
Creating User Accounts ........................................................................... 181
Editing Users: passwd(1), chpass(1), and Friends ........................................ 183
Shells and /etc/shells ........................................................................................... 188
root, Groups, and Management ............................................................................. 189
The root Password ................................................................................... 189
Groups of Users ...................................................................................... 190
Using Groups to Avoid Root ..................................................................... 191
Tweaking User Security ......................................................................................... 195
Restricting Login Ability ............................................................................ 195
Restricting System Usage .......................................................................... 197
File Flags ............................................................................................................ 201
Setting and Viewing File Flags .................................................................. 203
Securelevels ......................................................................................................... 204
Securelevel Definitions ............................................................................. 204
Which Securelevel Do You Need? ............................................................ 205
What Won’t Securelevels and File Flags Accomplish? ................................. 206
Living with Securelevels ............................................................................ 206
Network Targets .................................................................................................. 207
Putting It All Together ............................................................................................ 208

8
D I S K S A ND F I LE SY S T E M S

209

Disk Drives 101 ................................................................................................... 209
Device Nodes ...................................................................................................... 210
Hard Disks and Partitions ......................................................................... 211
The Filesystem Table: /etc/fstab ............................................................................. 212
What’s Mounted Now? ........................................................................................ 214
Mounting and Unmounting Disks ............................................................................ 214
Mounting Standard Filesystems ................................................................. 214
Mounting at Nonstandard Locations .......................................................... 215
Unmounting a Partition ............................................................................ 215
How Full Is a Partition? ......................................................................................... 215
The Fast File System .............................................................................................. 217
Vnodes .................................................................................................. 218
FFS Mount Types ..................................................................................... 218
FFS Mount Options .................................................................................. 220
Soft Updates and Journaling with FFS ........................................................ 220
Write Caching ........................................................................................ 221
Snapshots .............................................................................................. 222
Dirty Disks .............................................................................................. 222
Forcing Read-Write Mounts on Dirty Disks .................................................. 224
FFS Syncer at Shutdown ........................................................................... 224
Background fsck, fsck -y, Foreground fsck, Oy Vey! ..................................... 225
Using Foreign Filesystems ...................................................................................... 225
Supported Foreign Filesystems .................................................................. 226
Permissions and Foreign Filesystems .......................................................... 228

xiv

C on te nt s i n De ta il

Removable-Media Filesystems ................................................................................ 228
Formatting FAT32 Media ......................................................................... 228
Using Removable Media .......................................................................... 230
Ejecting Removable Media ....................................................................... 231
Removable Media and /etc/fstab ............................................................. 231
Other FreeBSD Filesystems .................................................................................... 231
Memory Filesystems ................................................................................. 232
Mounting Disk Images ............................................................................. 235
Filesystems in Files ................................................................................... 235
Miscellaneous Filesystems ......................................................................... 238
Wiring Down Devices ........................................................................................... 238
Adding New Hard Disks ....................................................................................... 240
Creating Slices ....................................................................................... 240
Creating Partitions ................................................................................... 241
Configuring /etc/fstab ............................................................................. 241
Installing Existing Files onto New Disks ...................................................... 241
Stackable Mounts .................................................................................... 242
Network Filesystems ............................................................................................. 243
FreeBSD and CIFS ................................................................................................ 248
Prerequisites ........................................................................................... 248
Kernel Support ........................................................................................ 249
Configuring CIFS .................................................................................... 249
nsmb.conf Keywords ............................................................................... 249
CIFS Name Resolution ............................................................................. 250
Other smbutil(1) Functions ........................................................................ 250
Mounting a Share ................................................................................... 251
Other mount_smbfs Options ..................................................................... 251
Sample nsmb.conf Entries ......................................................................... 252
CIFS File Ownership ................................................................................ 252
Serving CIFS Shares ............................................................................................. 252
devfs .................................................................................................................. 253
devfs at Boot: devfs.conf .......................................................................... 253
Global devfs Rules ................................................................................... 255
Dynamic Device Management with devd(8) ................................................ 256

9
A D V A NC E D S E C U R IT Y F E A T U R E S

261

Unprivileged Users ............................................................................................... 261
The nobody Account ................................................................................ 263
A Sample Unprivileged User ..................................................................... 263
Network Traffic Control ......................................................................................... 263
Default Accept vs. Default Deny ............................................................................. 264
TCP Wrappers ..................................................................................................... 265
Configuring Wrappers ............................................................................. 265
Wrapping Up Wrappers .......................................................................... 271
Packet Filtering .................................................................................................... 272
Enabling PF ............................................................................................ 273
Default Accept and Default Deny in Packet Filtering ..................................... 273
Basic Packet Filtering and Stateful Inspection .............................................. 274
Configuring PF ........................................................................................ 275
Complete PF Rule Sample ......................................................................... 278
Activating PF Rules .................................................................................. 279
C on t en ts i n D et ail

xv

Public Key Encryption ........................................................................................... 280
Configuring OpenSSL .............................................................................. 281
Certificates ............................................................................................. 282
SSL Trick: Connecting to SSL-Protected Ports ................................................ 285
Jails .................................................................................................................... 286
Jail Host Server Setup .............................................................................. 287
Jail and the Kernel ................................................................................... 289
Client Setup ............................................................................................ 290
Decorating Your Cell: In-Jail Setup ............................................................. 291
Jail and /etc/rc.conf ............................................................................... 293
Jail Startup and Shutdown ........................................................................ 293
Managing Jails ....................................................................................... 294
Jail Shutdown ......................................................................................... 295
What’s Wrong with Jails .......................................................................... 295
Preparing for Intrusions with mtree(1) ...................................................................... 296
Running mtree(1) ..................................................................................... 297
Saving the Spec File ................................................................................ 298
Reacting to an Intrusion ............................................................................ 299
Monitoring System Security ................................................................................... 299
If You’re Hacked .................................................................................................. 300

10
E X P LO R I N G / E T C

301

/etc Across Unix Species ...................................................................................... 302
/etc/adduser.conf ................................................................................................ 302
/etc/amd.map .................................................................................................... 302
/etc/bluetooth, /etc/bluetooth.device.conf, and
/etc/defaults/bluetooth.device.conf ................................................................. 302
/etc/crontab ....................................................................................................... 302
/etc/csh.* .......................................................................................................... 303
/etc/devd.conf .................................................................................................... 303
/etc/devfs.conf, /etc/devfs.rules, and
/etc/defaults/devfs.rules ................................................................................ 303
/etc/dhclient.conf ................................................................................................ 303
/etc/disktab ........................................................................................................ 303
/etc/freebsd-update.conf ...................................................................................... 304
/etc/fstab ........................................................................................................... 304
/etc/ftp.* ........................................................................................................... 304
/etc/group .......................................................................................................... 304
/etc/hosts ........................................................................................................... 304
/etc/hosts.allow .................................................................................................. 304
/etc/hosts.equiv .................................................................................................. 304
/etc/hosts.lpd ...................................................................................................... 305
/etc/inetd.conf .................................................................................................... 305
/etc/localtime ..................................................................................................... 305
/etc/locate.rc ...................................................................................................... 305
/etc/login.* ........................................................................................................ 306
/etc/mail/mailer.conf .......................................................................................... 306

xvi

C on te nt s i n De ta il

/etc/make.conf ................................................................................................... 306
CFLAGS ................................................................................................. 307
COPTFLAGS ........................................................................................... 307
CXXFLAGS ............................................................................................. 307
CPUTYPE=i686 ....................................................................................... 307
INSTALL=install -C ................................................................................... 308
/etc/master.passwd ............................................................................................. 308
/etc/motd ........................................................................................................... 308
/etc/mtree .......................................................................................................... 308
/etc/namedb ...................................................................................................... 309
/etc/netstart ........................................................................................................ 309
/etc/network.subr ................................................................................................ 309
/etc/newsyslog.conf ............................................................................................ 309
/etc/nscd.conf ..................................................................................................... 309
/etc/nsmb.conf .................................................................................................... 309
/etc/nsswitch.conf ............................................................................................... 309
/etc/opie* .......................................................................................................... 309
/etc/pam.d/* ..................................................................................................... 310
/etc/pccard_ether ................................................................................................ 310
/etc/periodic.conf and /etc/defaults/periodic.conf ................................................. 310
daily_output=”root” ................................................................................. 310
daily_show_success=”YES” ...................................................................... 310
daily_show_info=”YES” ........................................................................... 310
daily_show_badconfig=”NO” .................................................................. 311
daily_local=”/etc/daily.local” .................................................................. 311
/etc/pf.conf ........................................................................................................ 311
/etc/pf.os ........................................................................................................... 311
/etc/phones ........................................................................................................ 311
/etc/portsnap.conf ............................................................................................... 311
/etc/ppp ............................................................................................................ 311
/etc/printcap ...................................................................................................... 312
/etc/profile ......................................................................................................... 312
/etc/protocols ..................................................................................................... 312
/etc/rc* ............................................................................................................. 312
/etc/remote ........................................................................................................ 312
/etc/rpc ............................................................................................................. 313
/etc/security/ ...................................................................................................... 313
/etc/services ....................................................................................................... 313
/etc/shells .......................................................................................................... 313
/etc/snmpd.config ............................................................................................... 313
/etc/src.conf ....................................................................................................... 313
/etc/sysctl.conf .................................................................................................... 313
/etc/syslog.conf .................................................................................................. 313
/etc/termcap ....................................................................................................... 314
/etc/ttys ............................................................................................................. 314

11
M A KI N G YO U R S Y S T E M U S E F U L

315

Making Software ................................................................................................. 316
Source Code and Software ................................................................................... 316

C o nt en t s in D et ai l

xvii

The Ports and Packages System .............................................................................. 317
Ports ...................................................................................................... 318
Finding Software .................................................................................................. 320
Finding by Name .................................................................................... 321
Finding by Keyword ................................................................................ 321
Legal Restrictions ..................................................................................... 322
Using Packages ................................................................................................... 322
CD Packages .......................................................................................... 323
FTP Packages .......................................................................................... 324
Installing Packages .................................................................................. 325
pkg_add(1) Environment Settings .............................................................. 326
What Does a Package Install? .................................................................. 327
Uninstalling Packages .............................................................................. 328
Package Information ................................................................................ 329
Package Problems ................................................................................... 330
Using Ports .......................................................................................................... 331
Installing a Port ....................................................................................... 332
Integrated Port Customizations .................................................................. 334
Port Makefiles ......................................................................................... 336
Uninstalling and Reinstalling ..................................................................... 337
Tracking Port Build Status ......................................................................... 338
Cleaning Up Ports ................................................................................... 338
Building Packages ................................................................................... 339
Changing the Install Path .......................................................................... 339
Setting make Options Permanently ............................................................ 340
Ports and Package Security ................................................................................... 340

12
A D V A NC E D S O F T W A R E M AN A G E M E N T

343

Using Multiple Processors: SMP ............................................................................. 344
Kernel Assumptions ................................................................................. 344
SMP: The First Try .................................................................................... 345
Today’s SMP .......................................................................................... 346
Processors and SMP ................................................................................ 347
Using SMP ............................................................................................. 348
Schedulers ........................................................................................................... 349
Startup and Shutdown Scripts ................................................................................ 350
rc Script Ordering ................................................................................... 350
A Typical rc Script ................................................................................... 351
Special rc Script Providers ........................................................................ 352
Using Scripts to Manage Running Programs ............................................... 353
Vendor Startup/Shutdown Scripts .............................................................. 353
Debugging Custom rc Scripts .................................................................... 353
Managing Shared Libraries ................................................................................... 354
Shared Library Versions and Files .............................................................. 354
Attaching Shared Libraries to Programs ...................................................... 355
LD_LIBRARY_PATH ................................................................................... 357
What a Program Wants ........................................................................... 358
Threads, Threads, and More Threads ..................................................................... 358
Userland Threading Libraries ................................................................................. 359
Remapping Shared Libraries .................................................................................. 360
xviii

C ont en t s in D et a il

Running Software from the Wrong OS .................................................................... 361
Recompilation ......................................................................................... 362
Emulation ............................................................................................... 363
ABI Reimplementation .............................................................................. 363
Binary Branding ...................................................................................... 364
Supported ABIs ....................................................................................... 364
Foreign Software Libraries ........................................................................ 365
Using Linux Mode ................................................................................................ 365
The Linuxulator Userland .......................................................................... 366
Testing Linux Mode .................................................................................. 366
Identifying and Setting Brands .................................................................. 367
linprocfs ................................................................................................. 367
Debugging Linux Mode with truss(1) .......................................................... 368
Running Software from the Wrong Architecture ........................................................ 369

13
UP G R A D I N G F R E E B S D

371

FreeBSD Versions ................................................................................................. 372
Releases ................................................................................................. 372
FreeBSD-current ....................................................................................... 373
FreeBSD-stable ........................................................................................ 374
Snapshots .............................................................................................. 375
FreeBSD and Testing ................................................................................ 376
Which Version Should You Use? ............................................................... 376
Upgrade Methods ................................................................................................ 377
Binary Updates .................................................................................................... 378
/etc/freebsd-update.conf ......................................................................... 378
Running freebsd-update(8) ........................................................................ 379
Scheduling Binary Updates ....................................................................... 380
Upgrading via sysinstall ........................................................................................ 380
Upgrading via Source ........................................................................................... 382
Selecting Your Supfile .............................................................................. 383
Modifying Your Supfile ............................................................................ 384
A Complete Supfile ................................................................................. 386
Blocking Updates: The Refuse File ............................................................. 386
Updating System Source Code .................................................................. 387
Using csup to Get the Whole Source Tree .................................................. 387
Building FreeBSD from Source ............................................................................... 388
Build the World ...................................................................................... 388
Build, Install, and Test a Kernel ................................................................. 389
Optimization with Parallel Builds ............................................................... 390
Prepare to Install the New World .............................................................. 390
Installing the World ................................................................................. 393
mergemaster Revisited ............................................................................. 395
Upgrades and Single-User Mode .............................................................. 395
Shrinking FreeBSD ................................................................................................ 396
Updating with csup and make ............................................................................... 398
Cross-Building FreeBSD ......................................................................................... 399
Building a Local CVSup Server ............................................................................... 399
Controlling Access ................................................................................... 402

C on te nt s i n De ta il

xix

Upgrading the Ports Collection ............................................................................... 403
Configuring portsnap ............................................................................... 403
Using portsnap(8) ................................................................................... 404
Updating Installed Ports ....................................................................................... 404
Initial portmaster Setup ............................................................................ 405
Identifying Unneeded Software ................................................................. 406
Identifying and Upgrading Software .......................................................... 406
Forcing a Rebuild .................................................................................... 407
Rebuilding Upward Dependencies ............................................................. 408
Changing Dependencies .......................................................................... 408
Ignoring Ports ......................................................................................... 408
Other portmaster Features ........................................................................ 409
Reducing the Size of the Ports Tree ............................................................ 409

14
T H E I N T E R N E T R O A D M A P: D NS

411

How DNS Works ................................................................................................. 412
Basic DNS Tools .................................................................................................. 413
The host(1) Command .............................................................................. 413
Digging for Detail ................................................................................... 414
Finding Hostnames with dig ..................................................................... 416
More dig Options ................................................................................... 417
in-addr.arpa ........................................................................................... 418
Configuring the Resolver ....................................................................................... 419
Host/IP Information Sources ..................................................................... 419
Setting Local Domain Names .................................................................... 420
The Nameserver List ................................................................................ 421
Local DNS Overrides with /etc/hosts ..................................................................... 422
Building a Nameserver ......................................................................................... 422
Masters and Slaves ................................................................................. 423
BIND Configuration Files .......................................................................... 423
Configuring BIND with named.conf ........................................................................ 424
Options ................................................................................................. 424
Zones in named.conf ............................................................................... 425
Configuring a Slave Domain .................................................................... 426
Configuring a Master Domain .................................................................. 427
Master and Slave File Storage .................................................................. 427
Zone Files ........................................................................................................... 428
A Real Sample Zone ................................................................................ 432
Dots and Termination in Zone Files ............................................................ 433
Reverse DNS Zones ................................................................................. 433
Managing named ................................................................................................ 434
Configuring rndc ..................................................................................... 434
Using rndc ............................................................................................. 435
Checking DNS ..................................................................................................... 436
Nameserver Security ............................................................................................ 436
Controlling Zone Transfers ........................................................................ 436
Securing named(8) .................................................................................. 437
More on BIND ..................................................................................................... 437

xx

C o nt en t s in D et ai l

15
S M AL L S Y S T E M S E R V I C E S

439

SSH .................................................................................................................... 439
The SSH Server: sshd(8) ........................................................................... 440
Configuring the SSH Daemon ................................................................... 442
Managing SSH User Access ..................................................................... 444
SSH Clients ............................................................................................ 445
Network Time ...................................................................................................... 447
Setting the Time Zone .............................................................................. 447
Network Time Protocol ............................................................................. 448
Name Service Switching and Caching .................................................................... 450
/etc/nsswitch.conf .................................................................................. 450
Name Query Caching with nscd(8) ........................................................... 451
inetd ................................................................................................................... 453
/etc/inetd.conf ....................................................................................... 453
Configuring inetd Servers ......................................................................... 454
Starting inetd(8) ...................................................................................... 455
Changing inetd’s Behavior ....................................................................... 456
DHCP ................................................................................................................. 456
How DHCP Works .................................................................................. 457
Managing dhcpd(8) ................................................................................ 457
Configuring dhcpd(8) .............................................................................. 457
Printing and Print Servers ...................................................................................... 459
/etc/printcap ......................................................................................... 460
TFTP ................................................................................................................... 461
Root Directory ......................................................................................... 461
tftpd and Files ......................................................................................... 462
File Ownership ....................................................................................... 462
tftpd(8) Configuration .............................................................................. 462
Scheduling Tasks .................................................................................................. 463
User Crontabs vs. /etc/crontab ................................................................ 463
cron and Environment .............................................................................. 464
Crontab Format ....................................................................................... 464

16
S PA M , W O R M S , A N D V IR US E S
( PL U S E M A I L, I F Y O U I N SI S T )

467

Email Overview ................................................................................................... 468
Finding Mail Servers for a Domain ............................................................ 468
Undeliverable Email ................................................................................ 469
The SMTP Protocol ................................................................................... 470
Relay Control .......................................................................................... 472
Stopping Bad Email ................................................................................. 472
Sendmail ............................................................................................................. 473
mailwrapper(8) ....................................................................................... 474
Submission vs. Reception ......................................................................... 474
Sendmail Logging ................................................................................... 476
Configuring Sendmail ........................................................................................... 476
The access File ........................................................................................ 476
The aliases File ....................................................................................... 478
C on te nt s i n De ta il

xxi

The mailertable File ................................................................................. 479
The relay-domains File ............................................................................. 480
Making Changes Take Effect .................................................................... 480
Virtual Domains ................................................................................................... 481
The /etc/mail/local-host-names File ........................................................... 481
User Mapping ........................................................................................ 481
Changing sendmail.cf ........................................................................................... 483
Custom .mc Files ..................................................................................... 484
Rejecting Spam Sources ........................................................................... 485
Greylisting .......................................................................................................... 487
Configuring milter-greylist ......................................................................... 488
Attaching milter-sendmail to Sendmail ........................................................ 490
Sendmail Authentication with SASL ......................................................................... 491
saslauthd(8) ............................................................................................ 492
mailer.conf and Your New Sendmail ......................................................... 492
Building sendmail.cf ................................................................................ 492
Testing SASL ........................................................................................... 493
IMAP and POP3 ................................................................................................... 493
Installing Dovecot .................................................................................... 494
Configuring Dovecot ................................................................................ 494
Creating a Dovecot SSL Certificate ............................................................ 495
Running Dovecot ..................................................................................... 496
Testing POP3S ........................................................................................ 496
Testing IMAPS ........................................................................................ 497

17
W E B A N D F T P S E R VI C E S

499

How a Web Server Works .................................................................................... 500
The Apache Web Server ....................................................................................... 500
Apache Configuration Files ...................................................................... 501
Core Apache Configuration ..................................................................... 501
Apache Logs .......................................................................................... 503
Apache Modules .................................................................................................. 505
Directories and Permissions ................................................................................... 507
Controlling Access by IP Address .............................................................. 507
Directory Options .................................................................................... 508
Configuration by Users ............................................................................ 510
Other Directory Settings ........................................................................... 511
Password Protection and Apache .............................................................. 512
Including Other Configuration Files ........................................................................ 515
Virtual Hosting ..................................................................................................... 517
Configuring Virtual Hosts ......................................................................... 517
Tuning Virtual Hosts ................................................................................. 518
HTTPS Websites ................................................................................................... 520
Controlling Apache .............................................................................................. 521
File Transfer ......................................................................................................... 522
FTP Security ............................................................................................ 522
The FTP Client ......................................................................................... 522
Binary and ASCII Transfers ....................................................................... 523
The FTP Server ........................................................................................ 524
FTP User Control ..................................................................................... 524
xxii

C on t en ts in D et ail

FTP Server Messages ............................................................................... 525
Setting Up Anonymous FTP Servers ............................................................ 526
Chrooting sftp(1) and scp(1) .................................................................................. 527

18
D I S K T R I CK S W I T H G E O M

529

GEOM Essentials ................................................................................................. 530
Disk Drives 102 ................................................................................................... 530
Slicing Disks ........................................................................................................ 531
Viewing the Slice Table with fdisk(8) .......................................................... 532
Backing Up the Slice Table ....................................................................... 533
Changing the Slice Table ......................................................................... 533
Partitioning Slices .................................................................................... 536
Reading Disklabels .................................................................................. 537
Backing Up and Restoring Disklabels ......................................................... 538
Editing Disklabels .................................................................................... 538
Replicating Drive Slicing and Partitioning ................................................... 539
Missing Disklabels ................................................................................... 540
Building Filesystems .............................................................................................. 540
RAID ................................................................................................................... 541
Hardware vs. Software RAID .................................................................... 541
GEOM RAID and Disk Size ...................................................................... 542
Parity and Stripe Size .............................................................................. 542
RAID Types ............................................................................................. 543
Generic GEOM Commands .................................................................................. 544
Striping Disks ....................................................................................................... 545
Creating a Striped Provider ...................................................................... 546
gstripe Destruction ................................................................................... 546
Daily Status Check .................................................................................. 547
Mirroring Disks .................................................................................................... 547
Creating a Mirror .................................................................................... 547
Repairing Mirrors .................................................................................... 548
Mirrored Boot Disks ................................................................................. 549
Destroying Mirrored Disks ........................................................................ 550
Daily Status Check .................................................................................. 550
RAID-3 ................................................................................................................ 550
Creating a RAID-3 ................................................................................... 551
Repairing a RAID-3 ................................................................................. 551
Destroying a RAID-3 ............................................................................... 553
RAID-10 .............................................................................................................. 553
RAID-10 Setup ........................................................................................ 553
RAID-10 Status ........................................................................................ 554
Destroying a RAID-10 ............................................................................. 554
Journaling Filesystems with gjournal(8) .................................................................... 554
Configuring gjournal(8) ........................................................................... 556
Using a Separate Journal Device ............................................................... 557
De-Journaling Partitions ............................................................................ 557
Filesystem Encryption ............................................................................................ 558
Kernel Configuration ............................................................................... 559
Generating and Using a Cryptographic Key ............................................... 559
Filesystems on Encrypted Devices .............................................................. 560
C on t en ts in D et ai l

xxiii

Deactivating Encrypted Disks .................................................................... 560
Encrypting Swap Space with geli(8) .......................................................... 561
Disk Device Network Exports ................................................................................. 561
geom_gate Security ................................................................................. 562
geom_gate Server Setup .......................................................................... 562
geom_gate Client Setup ........................................................................... 563
Identifying geom_gate Devices ................................................................. 564
Shutting Down geom_gate ....................................................................... 564
Oops! Rescuing geom_gate ...................................................................... 564
Mirroring Disks Across the Network ........................................................................ 565
Backup Server Setup ................................................................................ 565
Primary Server Setup ............................................................................... 566
Mirror Failover and Recovery ................................................................... 567

19
S Y ST E M P E R FO R M A N CE AN D M O N I T O R I NG

569

Computer Resources ............................................................................................. 570
Checking the Network .......................................................................................... 571
General Bottleneck Analysis with vmstat(8) .............................................................. 571
Processes ............................................................................................... 572
Memory ................................................................................................. 572
Paging ................................................................................................... 572
Disks ..................................................................................................... 573
Faults ..................................................................................................... 573
CPU ...................................................................................................... 573
Using vmstat ........................................................................................... 573
Continuous vmstat ................................................................................... 574
Disk I/O ............................................................................................................. 574
CPU, Memory, and I/O with top(1) ........................................................................ 575
PID Values .............................................................................................. 576
Load Average ......................................................................................... 576
Uptime ................................................................................................... 576
Process Counts ........................................................................................ 576
Process Types ......................................................................................... 577
Memory ................................................................................................. 577
Swap ..................................................................................................... 578
Process List ............................................................................................. 578
top(1) and I/O ....................................................................................... 579
Following Processes .............................................................................................. 580
Paging and Swapping .......................................................................................... 581
Paging ................................................................................................... 582
Swapping .............................................................................................. 582
Performance Tuning .............................................................................................. 582
Memory Usage ....................................................................................... 583
Swap Space Usage ................................................................................. 583
CPU Usage ............................................................................................ 583
Rescheduling .......................................................................................... 584
Reprioritizing with Niceness ..................................................................... 584
Investigating Software .............................................................................. 586
Status Mail .......................................................................................................... 586

xxiv

C on t en t s in D et ai l

Logging with syslogd ............................................................................................ 587
Facilities ................................................................................................. 587
Levels ..................................................................................................... 588
Processing Messages with syslogd(8) ......................................................... 589
syslogd Customization ............................................................................. 592
Log File Management ........................................................................................... 593
Log File Path ........................................................................................... 594
Owner and Group .................................................................................. 594
Permissions ............................................................................................. 594
Count .................................................................................................... 594
Size ....................................................................................................... 595
Time ...................................................................................................... 595
Flags ..................................................................................................... 596
Pidfile .................................................................................................... 597
Signal .................................................................................................... 597
Sample newsyslog.conf Entry .................................................................... 597
FreeBSD and SNMP ............................................................................................. 598
SNMP 101 ............................................................................................. 598
Configuring bsnmpd ................................................................................ 600

20
T H E F R IN G E O F F R E E B S D

603

/etc/ttys ............................................................................................................. 604
/etc/ttys Format ...................................................................................... 604
Insecure Console ..................................................................................... 605
Diskless FreeBSD .................................................................................................. 606
Diskless Clients ....................................................................................... 607
DHCP Server Setup ................................................................................. 607
tftpd and the Boot Loader ......................................................................... 609
The NFS Server and the Diskless Client Userland ......................................... 609
Diskless Farm Configuration .................................................................................. 611
The /conf/base Directory ......................................................................... 611
The /conf/default Directory ................................................................................... 612
Per-Subnet and Per-Client Directories ......................................................... 612
Diskless Packages and Files ................................................................................... 613
Installing Packages .................................................................................. 613
Diskless Configuration Files ...................................................................... 613
NanoBSD: Building Your Own Appliances .............................................................. 615
What Is NanoBSD? ................................................................................. 616
Your Hardware and Your Flash Drive ........................................................ 617
The NanoBSD Toolkit ............................................................................... 618
Expanding FlashDevice.sub ...................................................................... 618
NanoBSD Configuration Options .............................................................. 619
A Sample NanoBSD Configuration ............................................................ 621
Building NanoBSD .................................................................................. 624
Customizing NanoBSD ............................................................................ 627
Using NanoBSD ...................................................................................... 629
Live Media with FreeSBIE ...................................................................................... 630
Installing the FreeSBIE Toolkit .................................................................... 631
Configuring FreeSBIE ............................................................................... 631
FreeSBIE Plug-ins ..................................................................................... 634
C on te nt s in D e ta il

xxv

Choosing Packages ................................................................................. 635
Building a FreeSBIE Image ....................................................................... 636
Rebuilding FreeSBIE ................................................................................. 636

21
S Y ST E M ( A N D S YS A D M IN ) P A N IC S A N D CR AS H E S

637

What Causes Panics? ........................................................................................... 637
Recognizing Panics .............................................................................................. 638
Responding to a Panic .......................................................................................... 639
Preparations ........................................................................................... 640
The Crash Dump in Action ........................................................................ 640
Configuring Crash Dumps ........................................................................ 640
Debugging Kernels .................................................................................. 641
When Panic Strikes: Manual Crash Dumps .............................................................. 642
Using the Dump ................................................................................................... 643
Getting a Backtrace ................................................................................. 643
vmcore and Security ................................................................................ 645
Submitting Problem Reports ................................................................................... 646
Before Filing a PR .................................................................................... 647
Bad PRs ................................................................................................. 648
Good PRs ............................................................................................... 649
A Sample PR .......................................................................................... 652
Submitting the PR .................................................................................... 653
After Submitting the PR ............................................................................. 653

A FT E R W O R D

655

The Community .................................................................................................... 655
Why Do We Do It? .............................................................................................. 656
What Can You Do? .............................................................................................. 657
If Nothing Else . . . ............................................................................................... 658
Getting Things Done ............................................................................................. 658

A PP E N D I X
S O M E I N T E R E S T I N G S Y S CT L M I B S

661

I ND E X

675

xxvi

C on t en t s in D et ai l

FOREWORD

It gives me great pleasure to write the foreword to
Michael Lucas’s Absolute FreeBSD. For five years, Michael’s
Absolute series has provided the definitive guide to BSD
software, not just as a reference, but also as a narrative for real human beings.
This is an important distinction, because while there is no lack of excellent
reference material on FreeBSD, this book provides a nuts-and-bolts tutorial
that readers will find an invaluable companion.
Michael is an active long-term contributor in the FreeBSD community.
Absolute FreeBSD draws on his experience with the many ways in which people
use FreeBSD in the real world—what they want to do, what works, and what
doesn’t. Apart from covering the use of FreeBSD, Michael will tell you about
the thousands of software developers—from hobbyists to professional developers and university professors—who write FreeBSD and about the evolution
of this community and its software. What I would like to do is invite you to
become a part of that community.
FreeBSD is a powerful network operating system with state-of-the-art
features that make it not only one of the most widely used pieces of software
in the world, but also an easy and practical tool on which to build and
provision services. From the Yahoo! and Verio websites to NetApp storage

products, from Cisco anti-spam appliances and Juniper routers to the root
nameservers—it’s hard to throw a rock on the Internet without hitting
FreeBSD. However, FreeBSD is not the product of any one company, but of a
large open source community: the FreeBSD Project, made up of developers,
users, and countless supporters and advocates. While you can, as many people
do, use FreeBSD simply as a piece of software without ever interacting with
that community, you can significantly enrich your FreeBSD experience by
becoming a part of that community.
Whether you are a first-time user or a kernel hacker, the resources available via the http://www.freebsd.org website, countless mailing lists, regional
user groups, and conferences can be invaluable. Have a question? Just email
questions@FreeBSD.org, and one or more of the hundreds of volunteers will
undoubtedly answer it. Want to learn more about the exciting new features
coming in future FreeBSD versions? Read the Project’s quarterly status
reports, development mailing lists, or attend one of the many regional BSD
conferences taking place around the world; at the time of writing, the most
recent addition is the first BSDConTR in Istanbul, Turkey.
These resources are a product of the FreeBSD Project and its community,
a large number of collaborating individuals and companies, as well as the
FreeBSD Foundation, a nonprofit organization coordinating funding, legal
resources, and support for development work and community activities.
Michael’s easy-to-use book provides a gateway for newbies to benefit from
this community’s expertise and to become active users of FreeBSD themselves.
FreeBSD is open source software, available for you to use and distribute
at no charge. By helping to support, advocate, or even develop FreeBSD, you
can give back to the FreeBSD Project and help this community grow.
Whether you are a new user of FreeBSD or an experienced one, I am
confident you will find Absolute FreeBSD a book you want to keep close at hand.
Robert N.M. Watson
FreeBSD Core Team Member
President, FreeBSD Foundation
Cambridge, UK
September 2007

xxviii

F orew ord

ACKNOWLEDGMENTS

I would like to thank all the members of the FreeBSD
community for their hard work, dedication, and
friendship. FreeBSD has saved my hide on numerous
occasions, and I’m delighted to give something back. In that community,
however, there are a few people who I want to specifically thank by name.
Doug Barton, Ceri Davies, Alex Dupre, Max Laier, Alexander Leidinger,
Remko Lodder, Benno Rice, Tom Rhodes, Gleb Smirnoff, and Robert Watson
all provided valuable feedback on this book. Some of them read individual
chapters that they have special expertise in, while others read the whole
manuscript, whether they knew about the topics or not. Wilko Bulte not
only did a review of this book, he volunteered to do so after reviewing the
entire first edition of this book back in 2001. He certainly deserves some
sort of “iron man” award! John Baldwin did an excellent final technical
review, catching an astonishing variety of errors ranging from subtle to
blatant. Any errors in this book were introduced by myself despite these
people’s best efforts.
I’d like to thank David Boyd, David O’Brien, and Wilko Bulte for donating
a variety of hardware that made it possible for me to write this book. I’d
especially like to thank Matt Olander of iXSystems, who sent me a complete

amd64 server when I really, really needed one. Speaking of hardware, as I was
finishing this book, I was wondering where I would find a good kernel panic
to write about in the last chapter. FreeBSD obliged me. Thanks to Scott Long
for fixing that panic, so I could actually write Chapter 18.
As always, the folks at No Starch Press have worked their butts off to
bring this to you. You all deserve a long vacation after putting up with me—
tell Bill I said it’s okay. Similarly, the fine staff at the School of Chinese
Martial Arts deserve a vacation from me. Sadly, now that this book is done
I’ll be spending some quality time on the mats, so they don’t get any time
off. Sorry, folks.
And, as always, I’m grateful that my wife did not succumb to the temptation to bash me over the head with a shovel and bury me and my laptop
behind the garage while I was finishing this book. She’s been more than
patient waiting for me to finish up so I could take out the trash. Last March’s
trash, that is . . .
Michael Lucas
St. Clair Shores, Michigan
September 2007

xxx

Ac kn owl edg m en ts

INTRODUCTION

Welcome to Absolute FreeBSD! This book is
a one-stop shop for system administrators
who want to build, configure, and manage
FreeBSD servers. It will also be useful for those folks
who want to run FreeBSD on their desktops, servers,
diskless system farms, and so on. By the time you finish
this book, you should be able to use FreeBSD to provide network services.
You should also understand how to manage, patch, and maintain your
FreeBSD systems and have a basic understanding of networking, system
security, and software management. We’ll discuss FreeBSD version 7, which
is the version recommended for production use at the time this book is
being released; however, most of this book applies to earlier and later
versions as well.

What Is FreeBSD?
FreeBSD is a freely available Unix-like operating system, used widely by
Internet service providers, in appliances and embedded systems, and
anywhere that reliability on commodity hardware is paramount. One day
last week, FreeBSD miraculously appeared on the Internet, fully formed,
extruded directly from the mutant brain of its heroic creator’s lofty intellect.
Just kidding; the truth is far more impressive. FreeBSD is a result of almost
three decades of continuous development, research, and refinement. The
story of FreeBSD begins in 1979, with BSD.

BSD: FreeBSD’s Granddaddy
Many years ago, AT&T needed a lot of specialized, custom-written computer
software to run its business. It was not allowed to compete in the computer
industry, however, so it could not sell its software. Instead, AT&T licensed
various pieces of software and the source code for that software to universities
at low, low prices. The universities could save money by using this software
instead of commercial equivalents with pricey licenses, and university students
with access to this nifty technology could read the source code to see how
everything worked. In return, AT&T got exposure, some pocket change, and
a generation of computer scientists who had cut their teeth on AT&T technology. Everyone got something out of the deal. The best-known software
distributed under this licensing plan was Unix.
Compared with modern operating systems, the original Unix had a lot of
problems. Thousands of students had access to its source code, however, and
hundreds of teachers needed interesting projects for their students. If a program behaved oddly, or if the operating system itself had a problem, the
people who lived with the system on a day-to-day basis had the tools and the
motivation to fix it. Their efforts quickly improved Unix and created many
features we now take for granted. Students added the ability to control
running processes, also known as job control. The Unix S51K filesystem made
system administrators cry like small children, so they replaced it with the Fast
File System, whose features have spread into every modern filesystem. Many
small, useful programs were written over the years, gradually replacing entire
swaths of Unix.
The Computer Science Research Group (CSRG) at the University of
California, Berkeley, participated in these improvements and also acted as
a central clearinghouse for Unix code improvements. The CSRG collected
changes from other universities, evaluated them, packaged them, and
distributed the compilation for free to anyone with a valid AT&T UNIX
license. The CSRG also contracted with the Defense Advanced Research
Projects Agency (DARPA) to implement various features in Unix, such as
TCP/IP. The resulting collection of software came to be known as the
Berkeley Software Distribution, or BSD.

2

I n t rod uct io n

BSD users took the software and improved it further, then fed their
enhancements back into BSD. Today, we consider this to be a fairly standard
way for an open source project to run, but in 1979 it was revolutionary. BSD
was also quite successful; if you check the copyright statement on an old BSD
system, you’ll see this:
Copyright 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.

Yep, 15 years of work—a lifetime in software development. How many
other pieces of software are not only still in use, but still in active development,
15 years after work began? In fact, so many enhancements and improvements
went into BSD that the CSRG found that over the years, it had replaced almost
all of the original Unix with code created by the CSRG and its contributors.
You had to look hard to find any original AT&T code.
Eventually, the CSRG’s funding ebbed, and it became clear that the BSD
project would end. After some political wrangling within the University of
California, in 1992 the BSD code was released to the general public under
what became known as the BSD license.

The BSD License
BSD code is available for anyone to use under what is probably the most
liberal license in the history of software development. The license can be
summarized as follows:
Don’t claim you wrote this.
Don’t blame us if it breaks.
Don’t use our name to promote your product.
This means that you can do almost anything you want with BSD code.
(The original BSD license did require that users be notified if a software
product included BSD-licensed code, but that requirement was later
dropped.) There’s not even a requirement that you share your changes
with the original authors! People were free to take BSD and include it in
proprietary products, open source products, or free products—they could
even print it out on punch cards and cover the lawn with it. You want to run
off 10,000 BSD CDs and distribute them to your friends? Enjoy. Instead of
copyright, the BSD license is sometimes referred to as copycenter, as in Take this
down to the copy center and run off a few for yourself. Not surprisingly, companies
such as Sun Microsystems jumped right on it: It was free, it worked, and plenty
of new graduates had experience with the technology. One company, BSDi,
was formed specifically to take advantage of BSD Unix.

I n tr od uct io n

3

The AT&T/CSRG/BSDi Iron Cage Match
At AT&T, UNIX work continued apace even as the CSRG went on its merry
way. AT&T took parts of the BSD Unix distribution and integrated them with
its UNIX, then relicensed the result back to the universities that provided
those improvements. This worked well for AT&T until the company was
broken up and the resulting companies were permitted to compete in the
computer software business. AT&T had one particularly valuable property: a
high-end operating system that had been extensively debugged by thousands of
people. This operating system had many useful features, such as a variety of
small but powerful commands, a modern filesystem, job control, and TCP/IP.
AT&T started a subsidiary, Unix Systems Laboratories (USL), which happily
started selling Unix to enterprises and charging very high fees for it, all the
while maintaining the university relationship that had given it such an
advanced operating system in the first place.
Berkeley’s public release of the BSD code in 1992 was met with great
displeasure from USL. Almost immediately, USL sued the university and the
software companies that had taken advantage of the software, particularly
BSDi. The University of California claimed that the CSRG had compiled BSD
from thousands of third-party contributors unrelated to AT&T, and so it was
the CSRG’s intellectual property to dispose of as it saw fit.
This lawsuit motivated many people to grab a copy of BSD to see what all
the fuss was about, while others started building products on top of it. One of
these products was 386BSD, which would eventually be used as the core of
FreeBSD 1.0.
In 1994, after two years of legal wrangling, the University of California
lawyers proved that the majority of AT&T UNIX was actually taken in its
entirety from BSD, rather than the other way around. To add insult to injury,
AT&T had actually violated the BSD license by stripping the CSRG copyright
from files it had assimilated. (Only a very special company can violate the
world’s most liberal software license!) A half-dozen files were the only sources
of contention, and to resolve these outstanding issues, USL donated some of
them to BSD while retaining some as proprietary information.
Once the dust settled, a new version of BSD Unix was released to the
world as BSD 4.4-Lite. A subsequent update, BSD 4.4-Lite2, is the grandfather
of the current FreeBSD, as well as ancestor to every other BSD variant in use
today.

The Birth of FreeBSD
One early result of BSD was 386BSD, a version of BSD designed to run on the
cheap 386 processor.1 The 386BSD project successfully ported BSD to Intel’s
386 processor, but it stalled. After a period of neglect, a group of 386BSD
users decided to branch out on their own and create FreeBSD so that they

1
At the time, several thousand dollars for a computer was dirt cheap. You young punks have no
idea how good you have it.

4

I n t rod uct io n

could keep the operating system up to date. (Several other groups started
their own branches off of 386BSD around the same time, of which only
NetBSD remains.)
386BSD and FreeBSD 1 were derived from 1992’s BSD release, the
subject of AT&T’s wrath. As a result of the lawsuit, all users of the original
BSD were requested to base any further work on BSD 4.4-Lite2. BSD 4.4-Lite2
was not a complete operating system—in particular, those few files AT&T
had retained as proprietary were vital to the system’s function. (After all, if
those files hadn’t been vital, AT&T wouldn’t have bothered!) The FreeBSD
development team worked frantically to replace those missing files, and
FreeBSD 2.0 was released shortly afterward. Development has continued
ever since.
Today, FreeBSD is used across the Internet by some of the most vital
and visible Internet-oriented companies. Yahoo! runs almost entirely on
FreeBSD. IBM, Nokia, Juniper, NetApp, and many other hardware companies use FreeBSD in embedded systems where you’d never even know it
unless someone told you. The fact is, if a company needs to pump serious
Internet bandwidth, it’s probably running FreeBSD or one of its BSD relatives.
Like smog, spiders, and corn syrup, FreeBSD is all around you; you simply
don’t see it because FreeBSD just works. The key to FreeBSD’s reliability is
the development team and user community—which are really the same thing.

FreeBSD Development
There’s an old saying that managing programmers is like herding cats. Despite
the fact that the FreeBSD development team is scattered across the world
and speaks dozens of languages, for the most part, the members work well
together as parts of the FreeBSD team. They’re more like a pride of lions
than a collection of house cats. Unlike some other projects, all FreeBSD
development happens in public. Three groups of people are responsible
for FreeBSD’s progress: committers, contributors, and users.

Committers
FreeBSD has about 500 developers, or committers. Committers have read-andwrite access to the FreeBSD master source code repository and can develop,
debug, or enhance any piece of the system. (The term committer comes from
their ability to commit changes to the source code.) Because these commits
can break the operating system in both subtle and obvious ways, committers
carry a heavy responsibility. Committers are responsible for keeping FreeBSD
working or, at worst, not breaking it as they add new features and evaluate
patches from contributors. Most of these developers are volunteers; only a
handful are actually paid to do this painstaking work, and most of those
people are paid only as it relates to other work. For example, Intel employs
a committer to ensure that FreeBSD properly supports its network cards.
FreeBSD has a high profile in the Internet’s heavy-lifting crowd, so Intel
needs its cards to work on FreeBSD.

I n tr od uct io n

5

To plug yourself into the beehive of FreeBSD development, consider
subscribing to the mailing list FreeBSD-hackers@FreeBSD.org, which contains
most of the technical discussion. Some of the technical talk is broken out
into more specific mailing lists—for example, fine details of the networking
implementation are discussed in FreeBSD-net@FreeBSD.org.
Every few years, the committer team elects a small number of its members
to serve as a core team, or Core. Core’s work is simultaneously vital, underrated,
and misunderstood. Core is theoretically responsible for the overall management of FreeBSD, but in practice, it manages little other than resolving
personality disputes and procedural conflicts among committers. Core also
approves new committers and delegates responsibility for large parts of
FreeBSD to individuals or groups. For example, it delegates authority over
the ports and packages system to the ports management team. Core does
not set architectural direction for FreeBSD, nor does it dictate processes or
procedures; that’s up to the committers, who must agree en masse. Core
does suggest, cajole, mediate, and inspire, however.
Core also experiences the worst part of management. Some of the key
functions of management in a company are oversight, motivation, and
handling problems between people. Oversight is provided by the millions
of users who will complain loudly when anything breaks or behaves unexpectedly, and FreeBSD committers are self-motivated. The ugly part of
management is settling a squabble between two people, and that’s the part
that Core has its hands full of. The status one gets from saying, “I’m in Core”
is an insufficient reward for having to manage an argument between two
talented developers who have gotten on each other’s nerves.

Contributors
In addition to the committer team, FreeBSD has thousands of contributors.
Contributors don’t have to worry about breaking the main operating system
source code repository; they just submit patches for consideration by
committers. Committers evaluate contributor submissions and decide what
to accept and what to reject. A contributor who submits many high-quality
patches is often asked to become a committer himself.
For example, I spent several years contributing to FreeBSD whenever
the urge struck me. Any time I feel that I’ve wasted my life, I can look at the
FreeBSD website and see where my work was accepted by the committers and
distributed to thousands of people. After I submitted the first edition of this
book to the publisher, I spent my spare time submitting patches to the FreeBSD
FAQ. Eventually, some members of the FreeBSD Documentation Project
approached me and asked me to become a committer. As a reward, I got an
email address and the opportunity to humiliate myself before thousands of
people, once again demonstrating that no good deed goes unpunished.
If I had never contributed anything, I’d remain a user. Nothing’s wrong
with that, either.

6

I n t rod uct io n

Users
Users are the people who run FreeBSD systems. It’s impossible to realistically
estimate the number of FreeBSD users, although organizations such as the
BSDstats Project (http://www.bsdstats.org) are making an effort. After all,
you can download the whole of FreeBSD for free and never register, upgrade,
or email a mailing list. Companies such as Netcraft estimate that between 5
and 15 percent of all computers attached to the Internet are BSD-based. If
you remove all the Windows boxes on corporate desktops, the percentage
rises considerably.
Since FreeBSD is by far the most popular open source BSD, that’s not
an inconsiderable number of machines. And since one FreeBSD server can
handle hundreds of thousands of Internet domains, a disproportionate
number of sites use FreeBSD as their supporting operating system. This
means that there are hundreds of thousands, if not millions, of FreeBSD
system administrators out in the world today.

Other BSDs
FreeBSD might be the most popular BSD, but it’s not the only one.
BSD 4.4-Lite2 spawned several different projects, each with its own focus
and purpose. Those projects in turn had their own offspring, several of
which thrive today.

NetBSD
NetBSD is similar to FreeBSD in many ways, and NetBSD and FreeBSD share
developers and code. NetBSD’s main goal is to provide a secure and reliable
operating system that can be ported to any hardware platform with minimal
effort. As such, NetBSD runs on VAXes, PocketPC devices, and high-end
SPARC and Alpha servers. I run NetBSD on my HP Jornada handheld
computer.2

OpenBSD
OpenBSD branched off from NetBSD in 1996 with the goal of becoming the
most secure BSD. OpenBSD was the first to support hardware-accelerated
cryptography, and its developers are rightfully proud of the fact that their
default installation was largely immune to remote exploits for several years.
The OpenBSD team has contributed several valuable pieces of software to
the world, the most notable being the OpenSSH suite used by almost every
operating system and hardware vendor today.

2
If you’re ever in a position where you need to prove that you are Alpha Geek amongst the pack,
running Unix on your palmtop will almost certainly do it.

I n tr od uct io n

7

Mac OS X
Mac OS X? That’s right. Apple incorporates large chunks of FreeBSD into
its Mac OS X on an ongoing basis. If you’re looking for a stable operating
system with a friendly face and a powerful core, Mac OS X is unquestionably
for you. While FreeBSD makes an excellent desktop for a computer professional, I wouldn’t put it in front of Grandma. I would put Mac OS X in
front of Grandma without a second thought, however, and I’d even feel that
I was doing the right thing. But Mac OS X includes many things that aren’t
at all necessary for an Internet server, and it only runs on Apple hardware,
so I don’t recommend it as an inexpensive general-purpose server.
What’s more, code goes both ways. FreeBSD has incorporated code
originally developed for Mac OS X. And while you cannot view the user
interface source code for Mac OS X, you can get the source code to its BSD
core and Mach kernel. Apple has released both under the code name
Darwin.

FreeBSD’s Children
Several projects have taken FreeBSD and built other projects or products
on top of it. The award-winning FreeNAS transforms an x 86 system into a
network fileserver with just a simple menu. FreeSBIE is a bootable CD that
lets you run FreeBSD without installing it. The m0n0wall project is also a
bootable CD, but it transforms your system into a firewall with a nice web
management interface. PC-BSD puts a friendly face on FreeBSD, trying to
make FreeBSD usable by Grandma. Other projects like this appear from time
to time; while not all are successful, I’m sure by the time this book comes out,
we’ll have one or two more solid members of this group.

Other Unixes
Several other operating systems derive from or emulate primordial Unix in
one way or another. This list is by no means exhaustive, but I’ll touch on the
high points.

Solaris/OpenSolaris
The best-known Unix is Sun Microsystems’ Solaris and its new offspring,
OpenSolaris. Solaris runs on high-end hardware that supports dozens of
processors and gobs of disk. (Yes, gobs is a technical term, meaning more than
you could possibly ever need, and I know very well that you need more disk than I
think you need.) Solaris, especially early versions of Solaris, had strong BSD
roots. Many enterprise-level applications run on Solaris. Solaris runs mainly
on the SPARC hardware platform manufactured by Sun, which allows Sun to
support interesting features such as hot-swappable memory and mainboards.
OpenSolaris increasingly targets commodity hardware, however.

8

I n t rod uct io n

WH Y UNIX-LIKE?
One thing to note is that FreeBSD, Linux, and so on are called Unix-like instead of
Unix. The term Unix is a trademark of The Open Group. For an operating system to
receive the right to call itself Unix, the vendor must prove that the OS complies with the
current version of the Single Unix Specification. While FreeBSD generally meets the
standard, continuous testing and re-certification cost money, which the FreeBSD
Project doesn’t have to spare. Certification as Unix also requires that someone sign
a paper stating that not only is he or she responsible for FreeBSD’s conformance to
the Single Unix Specification, but that he or she will fix any deviations from the
standard that are found in the future. FreeBSD’s development model makes this even
more difficult—bugs are found and deviations are fixed, but there’s nobody who
can sign a piece of paper that guarantees 100 percent standards compliance.

AIX
Another Unix contender is IBM’s entry, AIX. AIX’s main claim to fame is its
journaling filesystem, which records all disk transactions as they happen and
allows for fast recovery from a crash. It was also IBM’s standard Unix for
many years, and anything backed by Big Blue shows up all over the place.
AIX is largely based on BSD.

Linux
Linux is a close cousin of Unix, written from the ground up. Linux is similar
to FreeBSD in many ways, though FreeBSD has a much longer heritage and
is more friendly to commercial use than Linux. Linux includes a requirement
that any user who distributes Linux must make his or her changes available
to the end user, while BSD has no such restriction. Of course, a Linux fan
would say, “FreeBSD is more vulnerable to exploitation than Linux.” Linux
developers believe in share-and-share-alike, while BSD developers offer a nostrings-attached gift to everyone. It all depends on what’s important to you.
Many new Unix users have a perception of conflict between the BSD and
Linux camps. If you dig a little deeper, however, you’ll find that most of the
developers of these operating systems communicate and cooperate in a friendly
and open manner. It’s just a hard fringe of users and developers that generate
friction, much like different soccer teams’ hooligans or fans of different Star
Trek series.

IRIX, HP/UX, and So On
Other Unixes include Silicon Graphics’ IRIX, a solid Unix for graphics
applications, and Hewlett-Packard’s HP/UX, popular in large enterprises.
A quick web search uncovers many smaller contenders, such as Tru64 Unix
and the suicidal SCO Group’s UnixWare. You’ll also find old castoffs such as
Apple’s A/UX and Microsoft’s Xenix. (Yes, Microsoft was a licensed Unix
vendor, back in that age when dinosaurs watched the skies nervously and my
I n tr od uct io n

9

dad hunted mammoth for tribal rituals.) Many high-end applications are
designed to run best on one particular flavor of Unix. All modern Unixes
have learned lessons from these older operating systems, and today’s Unixes
and Unix-like operating systems are remarkably similar.

FreeBSD’s Strengths
After all this, what makes FreeBSD unique?

Portability
The FreeBSD Project’s goal is to provide a freely redistributable, stable, and
secure operating system that runs on the computer hardware that people are
most likely to have access to. Today this means Intel x 86-compatible systems
such as the 486, the various Pentiums, AMD, and so on, as well as AMD’s
amd64 architecture (copied by Intel as EM64T). Older x86 systems no
longer work out of the box with newer versions of FreeBSD, but most of
those systems are either long dysfunctional or aren’t about to change
operating systems any time soon.
The ARM platform used in embedded devices is a new addition to
FreeBSD and is well supported on specific embedded boards. FreeBSD also
supports Sun’s SPARC systems and Intel’s Itanium (IA64), as well as the
PowerPC processor recently used by Apple. While these other platforms
are not afterthoughts, they don’t receive the same level of attention that
x 86 and amd64 do.

Power
Since FreeBSD runs adequately on 386 hardware, it runs extremely well on
modern computers. It’s rather nice to have an operating system that doesn’t
demand a Pentium III and half a gig of RAM just to run the user interface. As
a result, you can actually dedicate your hardware to accomplishing real work
rather than tasks you don’t care about. If you choose to run a pretty graphical
interface with all sorts of spinning geegaws and fancy whistles, FreeBSD will
support you; it just won’t penalize you if you don’t want that. FreeBSD will
also support you on the latest n-CPU hardware.

Simplified Software Management
FreeBSD also simplifies software management through the Ports Collection.
Traditionally, running software on a Unix-like system required a great deal
of expertise. The Ports Collection simplifies this considerably by automating
and documenting the install, uninstall, and configuration processes for
thousands of software packages.

10

In t ro duc ti on

Optimized Upgrade Process
Unlike operating systems that require painful and risky upgrade procedures,
FreeBSD’s simple upgrade process builds an operating system optimized for
your hardware and applications. This lets FreeBSD use every feature supported
by your hardware, instead of just the lowest common denominator. If you
change hardware, you can rebuild your operating system to best handle that
particular hardware. Vendors such as Sun and Apple do exactly this, but they
control both the hardware and the software; FreeBSD pulls off the same trick
on commodity hardware.

Advanced Filesystem
A filesystem is how information is stored on the physical disk—it is what maps
the file My Resume to a series of zeroes and ones on a hard drive. FreeBSD
supports very sophisticated filesystems and can support files up to a petabyte
(one thousand thousand gigabytes). Its default filesystem is highly damage
resistant and reads and writes files extremely quickly. The BSD filesystem is
advanced enough that many commercial Unix vendors have used it as a basis
for their own filesystems.

Who Should Use FreeBSD?
While FreeBSD can be used as a powerful desktop or development machine,
its history shows a strong bias towards web, mail, file, and support services.
FreeBSD is most famous for its strengths as an Internet server, and it is an
excellent choice as an underlying platform for any network service. If major
firms such as Yahoo! count on FreeBSD to provide reliable service, it will
work as well for you.
If you’re thinking of running FreeBSD (or any Unix) on your desktop,
you’ll need to understand how your computer works. FreeBSD is not your
best choice if you need point-and-click simplicity. If that’s your goal, get a
Mac so you can use the power of Unix when you need it and not worry about
it the rest of the time. If you want to learn FreeBSD, though, running it on
your desktop is the best way—as we’ll discuss later.

Who Should Run Another BSD?
NetBSD and OpenBSD are FreeBSD’s closest competitors. Unlike competitors
in the commercial world, this competition is mostly friendly. FreeBSD,
NetBSD, and OpenBSD freely share code and developers; some people
even maintain the same subsystems in multiple operating systems.
If you want to use old or oddball hardware, NetBSD is a good choice for
you. For several years I ran NetBSD on an ancient SGI workstation that I used
as a Domain Name System (DNS) and fileserver. It did the job well until the
hardware finally released a cloud of smoke and stopped working.

I nt r odu ct ion

11

OpenBSD has implemented an impressive variety of security features.
Many of the tools are eventually integrated into FreeBSD, but that takes
months or years. If you have real security concerns but don’t need sophisticated multiprocessor support, you might look at OpenBSD.
If you’re just experimenting to see what’s out there, any BSD is good!

Who Should Run a Proprietary Operating System?
Operating systems such as Solaris, Windows, AIX, and their ilk are still quite
popular, despite the open source operating systems gnawing at their market
share. High-end enterprises are pretty tightly shackled to these operating
systems. While this is slowly changing, you’re probably stuck with commercial
operating systems in such environments. But slipping in an occasional
FreeBSD machine to handle basic services such as monitoring and department file serving can make your life much easier at much lower cost. Yahoo!
and NetApp have built entire businesses using FreeBSD instead of commercial
operating systems.
Of course, if the software you need only runs on a proprietary operating
system, your choice is pretty clear. Still, always ask a vendor if a FreeBSD
version is available; you might be pleasantly surprised.

How to Read This Book
Many computer books are thick and heavy enough to stun an ox, if you have
the strength to lift them high enough. Plus, they’re either encyclopedic in
scope or so painfully detailed that they’re difficult to actually read. Do you
really need to reference a screenshot when you’re told click OK or accept the
license agreement? And when was the last time you actually sat down to read
the encyclopedia?
Absolute FreeBSD is a little different. It’s designed to be read once, from
front to back. You can skip around if you want to, but each chapter builds on
what comes before it. While this isn’t a small book, it’s smaller than many
popular computer books. After you’ve read it once, it makes a decent
reference.
If you’re a frequent buyer of computer books, please feel free to insert
all that usual crud about “read a chapter at a time for best learning” and so
on. I’m not going to coddle you—if you picked up this book, you either have
two brain cells to rub together or you’re visiting someone who does. (If it’s
the latter, hopefully your host is smart enough to take this book away from
you before you learn enough to become dangerous.)

What Must You Know?
This book is aimed at the new Unix administrator. Two decades ago, the
average Unix administrator had kernel programming experience and was
working on his master’s degree in computer science. Even a decade ago, he
was already a skilled Unix user with real programming skills and most of a
bachelor’s degree in comp sci. Today, Unix-like operating systems are freely
12

In t ro duc ti on

available, computers are cheaper than food, and even 12-year-old children
can run Unix, read the source code, and learn enough to intimidate older
folks. As such, I don’t expect you to know a huge amount about Unix before
firing it up.
To use this book to its full potential, you need to have familiarity with
some basic tasks, such as how to change directories, list files in a directory,
and log in with a username and password. If you’re not familiar with basic
commands and the Unix shell, I recommend you begin with a book like
UNIX System Administration Handbook by Evi Nemeth and friends (Prentice
Hall PTR, 2006). To make things easier on newer system administrators,
I include the exact commands needed to produce the desired results. If you
learn best by example, you should have everything you need right here.
You’ll also need to know something about computer hardware—not a
huge amount, mind you, but something. For example, it helps to know how
to recognize an IDE, SCSI, or SATA cable. Your need for this knowledge
depends on the hardware you’re using, but if you’re interested enough to
pick up this book and read this far, you probably know enough.

For the New System Administrator
If you’re new to Unix, the best way to learn is to eat your own dog food. No,
I’m not suggesting that you dine with Rover. If you ran a dog food company,
you’d want to make a product that your own dog eats happily. If your dog
turns his nose up at your latest recipe, you have a problem. The point here is
that if you work with a tool or create something, you should actually use it.
The same thing applies to any Unix-like operating system, including FreeBSD.

Desktop FreeBSD
If you’re serious about learning FreeBSD, I suggest wiping out the operating
system on your main computer and running FreeBSD instead. Yes, I know,
now that dog food doesn’t sound so bad. But learning an operating system is
like learning a language; total immersion is the quickest and most powerful
way to learn. That’s what I did, and today I can make a Unix-like system do
anything I want. In fact, this book was composed entirely on a FreeBSD
laptop, using the open source text editor XEmacs and the OpenOffice.org
business suite. I also use FreeBSD to watch movies, rip and listen to MP3s,
balance my bank accounts, process my email, and surf the Web. As I write
this, I have a dozen animated BSD daemons running around on top of my
desktop windows, and I occasionally take a break to zap them with my mouse.
If this doesn’t count as a Stupid Desktop Trick, I don’t know what does.3
Many Unix system administrators these days come from a Windows
background. They’re beavering away in their little world when management
swoops by and says, “You can handle one more system, can’t you? Glad to

3
In the first edition of this book, I neglected to mention exactly how to do a similar Stupid
Desktop Trick, which generated more questioning email than any other topic in the whole
book. That’s a mistake I won’t make again!

I nt r odu ct ion

13

hear it! It’s a Unix box, by the way,” and then vanishes into the managerial
ether. Once the new Unix administrator decides to not slit his wrists, the
boss’s wrists, or start a fresh and exciting career as a whale autopsy technician,
he tentatively pokes at the system. He learns that ls is like dir and that cd is
the same on both platforms. He can learn the commands by rote, reading,
and experience. What he cannot learn, coming from this background, is how
a Unix machine thinks. Unix will not adjust to you; you must adjust to it.
Windows and OS X require similar adjustments, but they hide this behind a
glittering facade. With that in mind, let’s spend a little time learning how to
think about Unix.

How to Think About Unix
These days, most Unix systems come with pretty GUIs out of the box, but
they’re just eye candy. The real work happens on the command line, no
matter how many tools purport to hide it. The command line is actually
one of Unix’s strengths, and it is responsible for its unparalleled flexibility.
Unix’s underlying philosophy is many small tools, each of which does a
single job well. My laptop’s local programs directory (/usr/local/bin) has
662 programs in it. I have installed every one of them, either directly or
indirectly. Most are small, simple programs that only do one task, with
occasional exceptions, such as the office suite. This array of small tools
makes Unix extremely flexible and adaptable. Many commercial software
packages try to do everything; they wind up with all sorts of capabilities but
only mediocre performance in their core functions. Remember, at one time
you needed to be a programmer to use a Unix system, let alone run one.
Programmers don’t mind building their own tools. The Unix concept of
channels encouraged this.
Channels of Communication
People used to GUI environments such as Windows and Mac OS X are
probably unfamiliar with how Unix handles output and input. They’re used
to clicking something and seeing either an OK message, an error, nothing,
or (all too often) a pretty blue screen with nifty high-tech letters explaining
in the language called Geek why the system crashed. Unix does things a little
differently.
Unix programs have three channels of communication: standard input,
standard output, and standard error. Once you understand how each of
these channels works, you’re a good way along to understanding the whole
system.
Standard input is the source of information. When you’re at the console
typing a command, the standard input is the data coming from the keyboard.
If a program is listening to the network, the standard input is the network.
Many programs can rearrange standard input to accept data from the
network, a file, another program, the keyboard, or any other source.
The standard output is where the program’s output is displayed. This is
frequently the console (screen). Network programs usually return their
14

In t ro duc ti on

output to the network. Programs might send their output to a file, another
program, over the network, or anywhere else available to the computer.
Finally, standard error is where the program sends its error messages.
Frequently, console programs return their errors to the console; others log
errors in a file. If you set up a program incorrectly, it just might discard all
error information.
These three channels can be arbitrarily arranged, a concept that is
perhaps the biggest hurdle for new Unix users and administrators. For
example, if you don’t like the error messages appearing on the terminal,
you can redirect them to a file. If you don’t want to repeatedly type a lot
of information into a command, you can put the information into a file
(so you can reuse it) and dump the file into the command’s standard input.
Or, better still, you can run a command to generate that information and put
it in a file, or just pipe (send) the output of the first command directly to the
second, without even bothering with a file.
Small Programs, Channels, and the Command Line
Taken to its logical extreme, these input/output channels and the variety of
tools seem overwhelming. When I saw a sysadmin type something like the
following during my initial Unix training session, I gave serious consideration
to changing careers.
$ tail -f /var/log/messages | grep -v popper | grep -v named &

Lines of incomprehensible text began spilling across the screen, and
they kept coming. And worse still, my mentor kept typing as gibberish
poured out! If you’re from a point-and-click computing environment, a long
string of commands like this is definitely intimidating. What do all those
funky words mean? And an ampersand? You want me to learn what ?
Think of learning to use the command line as learning a language.
When learning a language, we start with simple words. As we increase our
vocabulary, we also learn how to string the words together. We learn that
placing words in a certain order makes sense, and that a different order
makes no sense at all. You didn’t speak that well at three years old—give
yourself some slack and you’ll get there.
Smaller, simpler programs and channels of communication provide
almost unlimited flexibility. Have you ever wished you could use a function
from one program in another program? By using a variety of smaller programs and arranging the inputs and outputs as you like, you can make
a Unix system behave in any manner that amuses you. Eventually, you’ll
feel positively crippled if you can’t just run a command’s output through
| sort -rnk 6 | less. 4

4
This ugly thing takes the output of the last command, sorts it in reverse order by the contents
of the sixth column, and presents it one screen at a time. If you have hundreds of lines of output,
and you want to know which entries have the highest values in the sixth column, this is how you
do it. Or, if you have lots of time, you can dump the output to a spreadsheet and fiddle with
equally obscure commands for a much longer time.

I nt r odu ct ion

15

Everything Is a File
You can’t be around Unix for very long before hearing that everything is a
file. Programs, account information, and system configuration are all stored
in files. Unix has no Windows-style registry; if you back up the files, you have
the whole system.
What’s more, the system identifies system hardware as files! Your CD-ROM
drive is a file, /dev/acd0. Network cards appear as files in /dev/net. Even virtual
devices, such as packet sniffers and partitions on hard drives, are files.
When you have a problem, keep this fact in mind. Everything is a file or
is in a file, somewhere on your system. All you have to do is find it!

Notes on the Second Edition
When I wrote my first technical book, the members of the BSD family had
huge amounts in common. A system administrator familiar with one BSD
could sit down at a different one and have the environment tuned nicely in
an hour or two. Some tools were in a different place, the boot sequences
were slightly different, and some features didn’t quite match, but on the
whole, each was just another derivative of BSD 4.4. That was five years ago,
and in the meantime each BSD has marched down a different path. While
they still have a lot in common, the differences are broad enough that I no
longer feel comfortable saying that much of this book is largely applicable to
all three BSDs. As such, this is Absolute FreeBSD, 2nd Edition, instead of just
Absolute BSD, 2nd Edition.
You’ll find other changes from the first edition, of course. The differences
between FreeBSD 4 and FreeBSD 7 vary from the dramatic to the subtle, and
either can trip you up if you’re not careful. Many tools for making Sendmail
manageable and friendly have been integrated into the system, so I cover
Sendmail instead of Postfix. (I still like Postfix, but this is a FreeBSD book.)
In 2000, it was unthinkable to have a computer without a floppy disk drive;
now, some computers ship without any integrated removable-media drives
whatsoever. This makes diskless work much more important, because for
some hardware, it’s the only way to get an operating system on the machine!
Lastly, FreeBSD has evolved greatly in the last five years, and I’ve learned more
in that time than I would have believed possible. Hopefully, this combination
makes Absolute FreeBSD, 2nd Edition a quantum leap better than its predecessor.

Contents of This Book
Absolute FreeBSD, 2nd Edition contains the following chapters.
Chapter 1: Getting More Help
This chapter discusses the information resources the FreeBSD Project
and its devotees provide for users. No one book can cover everything,
but knowing how to use the many FreeBSD resources on the Internet
helps fill any gaps you find here.

16

In t ro duc ti on

Chapter 2: Installing FreeBSD
This chapter gives you an overview of installing FreeBSD and offers
advice on an optimal install.
Chapter 3: Start Me Up! The Boot Process
This chapter teaches you about the FreeBSD boot process and how to
make your system start, stop, and reboot in different configurations.
Chapter 4: Read This Before You Break Something Else!
Here we discuss how to back up your data on both a system-wide and a
file-by-file level, and how to make your changes so that they can be easily
undone.
Chapter 5: Kernel Games
This chapter describes configuring the FreeBSD kernel. Unlike some
other operating systems, you are expected to tune FreeBSD’s kernel to
best suit your purposes. This gives you tremendous flexibility and lets you
optimize your hardware’s potential.
Chapter 6: The Network
Here we discuss the network and how it works in FreeBSD.
Chapter 7: Securing Your System
This chapter teaches you how to make your computer resist attackers
and intruders.
Chapter 8: Disks and Filesystems
This chapter covers some of the details of working with hard drives in
FreeBSD, support for other filesystems, and a few network filesystems.
Chapter 9: Advanced Security Features
Here we discuss some of the more interesting security features found in
FreeBSD.
Chapter 10: Exploring /etc
This chapter describes the many configuration files in FreeBSD and how
they operate.
Chapter 11: Making Your System Useful
Here I describe the ports and packages system that FreeBSD uses to
manage add-on software.
Chapter 12: Advanced Software Management
This chapter discusses some of the finer points of running software on
FreeBSD systems.
Chapter 13: Upgrading FreeBSD
This chapter teaches you how to use FreeBSD’s upgrade process. The
upgrade system is among the most remarkable and smooth of any operating system.
Chapter 14: The Internet Road Map: DNS
This chapter describes DNS and teaches you how to install and troubleshoot it.
I nt r odu ct ion

17

Chapter 15: Small System Services
Here we discuss some of the small programs you’ll need to manage in
order to use FreeBSD properly.
Chapter 16: Spam, Worms, and Viruses (Plus Email, If You Insist)
This chapter describes how to set up an email system on FreeBSD to reliably deliver mail and repel spam and viruses.
Chapter 17: Web and FTP Services
This chapter teaches you how to set up and secure these two vital
Internet services.
Chapter 18: Disk Tricks with GEOM
This chapter goes over some of the fancy techniques FreeBSD supports
for mirroring disks, exporting disk devices across the network, and generally having a good old time protecting and manipulating your data.
Chapter 19: System Performance and Monitoring
This chapter covers some of FreeBSD’s performance-testing and troubleshooting tools and shows you how to interpret the results. We also discuss
system logging and FreeBSD’s SNMP implementation.
Chapter 20: The Fringe of FreeBSD
This chapter teaches you some of the more interesting tricks you can do
with FreeBSD, such as running systems without disks and with tiny disks,
as well as some live failover and redundancy setups.
Chapter 21: System (and Sysadmin) Panics and Crashes
This chapter teaches you how to deal with those rare occasions when a
FreeBSD system fails, how to debug problems, and how to create a useful
problem report.
Appendix: Some Interesting sysctl MIBs
This appendix provides basic information about some of the kerneltuning options available for your use.
Okay, enough with the introductory stuff. Onward!

18

In t ro duc ti on

1
GETTING MORE HELP

As thick as this book is, it still can’t possibly
cover everything you must know about
FreeBSD. After all, Unix has been kicking
around for close to four decades, BSD is over a
quarter-century old, and FreeBSD is already a teenager.
Even if you memorize this book, it won’t cover every
situation you might encounter—especially when FreeBSD starts acting like a
typical teenager and needs a good smack. The FreeBSD Project supports a
huge variety of information resources, including numerous mailing lists and
the FreeBSD website, not to mention the official manual and Handbook. Its
users maintain even more documentation. The flood of information can be
overwhelming in itself, and it can make you want to just email the world and
beg for help. But before you send a question to a mailing list, confirm that
the information you need isn’t already available.

Why Not Just Email for Help?
The FreeBSD mailing lists are the best-known support resources. Many mailing list participants are very knowledgeable and can answer your questions
very quickly. But remember, when you mail a question to a FreeBSD mailing
list, you are asking tens of thousands of people all over the world to take a
moment to read your email. You’re also asking that one or more of them
take the time to help you instead of watching a favorite movie, enjoying
dinner with their families, or catching up on sleep. Problems arise when
these experts answer the same question 10, 50, or even hundreds of times.
They become grumpy. Some get downright tetchy.
What makes matters worse is that these same people have spent a
great deal of time and effort making the answers to most of these questions
available elsewhere. If you make it clear that you have already searched the
resources and your answer really doesn’t appear therein, you will probably
receive a polite, helpful answer. If you ask a question that has already been
asked several hundred times, however, the expert on that subject just might
snap and go ballistic on you. Do your homework, and chances are you’ll get
an answer more quickly than a fresh call to the mailing list could provide.

The FreeBSD Attitude
“Homework? What do you mean? Am I back in school? What do you want,
burnt offerings on bended knee?” Yes, you are in school. The information
technology business is nothing but lifelong, self-guided learning. Get used
to it or get out. Burnt offerings, on the other hand, are difficult to transmit
via email and are not quite so useful today.
Most commercial operating systems conceal their inner workings. The
only access you have to them is through the options presented by the vendor.
Even if you want to learn how something works, you probably can’t. When
something breaks, you have no choice but to call the vendor and grovel for
help. Worse, the people paid to help you frequently know little more than
you do.
If you’ve never worked with open source software vendors, FreeBSD’s
support mechanism might surprise you. There is no toll-free number to call
and no vendor to escalate within. No, you may not speak to a manager, for a
good reason: You are the manager. Congratulations on your promotion!

Support Options
Having said that, you’re not entirely on your own. The FreeBSD community
includes numerous developers, contributors, and users who care very deeply
about FreeBSD’s quality, and they’re happy to work with you. FreeBSD
provides everything you need: complete access to the source code used to
create the system, the tools needed to turn that source code into programs,
and the same debuggers used by the developers. Nothing is hidden; you can
see the innards, warts and all. You can view FreeBSD’s development history

20

C ha pt er 1

since the beginning, including every change ever made and the reason for it.
These tools might be beyond your abilities, but that’s not the Project’s problem. Various community members are even happy to provide guidance as you
develop your own skills so you can use those tools yourself. You’ll have lots of
help fulfilling your responsibilities.
As a grossly overgeneralized rule, people help those like themselves.
If you want to use FreeBSD, you must make the jump from eating what the
vendor gives you to learning how to cook. Every member of the FreeBSD
user community learned how to use it, and they welcome interested new
users with open arms. If you just want to know what to type without really
understanding what’s going on behind the scenes, you’ll be better off
reading the documentation: The general FreeBSD support community
simply isn’t motivated to help those who won’t help themselves or who
can’t follow instructions.
If you want to use FreeBSD but have neither the time nor the inclination
to learn more, invest in a commercial support contract. It might not be able to
put you in touch with FreeBSD’s owner, but at least you’ll have someone
to yell at. You’ll find several commercial support providers listed on the
FreeBSD website.
It’s also important to remember that the FreeBSD Project only maintains FreeBSD. If you’re having trouble with some other piece of software, a
FreeBSD mailing list is not the place to ask for help. FreeBSD developers are
generally proficient in a variety of software, but that doesn’t mean that they
want to help you, say, configure KDE.
The first part of your homework, then, is to learn about the resources
available beyond this book. These include the integrated manual, the
FreeBSD website, the mailing list archives, and other websites.

Man Pages
Man pages (short for manual pages) are the primordial way of presenting
Unix documentation. While man pages have a reputation for being obtuse,
difficult, or even incomprehensible, they’re actually quite friendly—for
particular users. When man pages were first created, the average system
administrator was a C programmer and, as a result, the pages were written
by programmers, for programmers. If you can think like a programmer, man
pages are perfect for you. I’ve tried thinking like a programmer, but I only
achieved real success after remaining awake for two days straight. (Lots of
caffeine and a high fever help.)
Over the last several years, the skill level required for system administration has dropped; no longer must you be a programmer. Similarly, man pages
have become more and more readable. Man pages are not tutorials, however;
they explain the behavior of one particular program, not how to achieve a
desired effect. While they’re neither friendly nor comforting, they should be
your first line of defense. If you send a question to a mailing list without
checking the manual, you’re likely to get a terse man whatever in response.

Get t in g M ore Hel p

21

Manual Sections
The FreeBSD manual is divided into nine sections. Roughly speaking, the
sections are:
1.
2.
3.
4.
5.

General user commands
System calls and error numbers
C programming libraries
Devices and device drivers
File formats

6.
7.
8.
9.

Game instructions
Miscellaneous information
System maintenance commands
Kernel interfaces

Each man page starts with the name of the command it documents
followed by its section number in parenthesis, like this: reboot(8). When you
see something in this format in other documents, it’s telling you to read that
man page in that section of the manual. Almost every topic has a man page.
For example, to see the man page for the editor vi, type this command:
$ man vi

In response, you should see the following:
VI(1)

VI(1)

NAME
ex, vi, view - text editors
SYNOPSIS
ex [-eFGRrSsv] [-c cmd] [-t tag] [-w size] [file ...]
vi [-eFGlRrSv] [-c cmd] [-t tag] [-w size] [file ...]
view [-eFGRrSv] [-c cmd] [-t tag] [-w size] [file ...]
LICENSE
The vi program is freely redistributable. You are welcome to copy,
modify and share it with others under the conditions listed in the
LICENSE file. If any company (not individual!) finds vi sufficiently
useful that you would have purchased it, or if any company wishes to
redistribute it, contributions to the authors would be appreciated.
DESCRIPTION
Vi is a screen oriented text editor. Ex is a line-oriented text editor
Ex and vi are different interfaces to the same program, and it is
possible to switch back and forth during an edit session. View is the
:

The page starts with the title of the man page (vi) and the section
number (1), and then it gives the name of the page. This particular page
has three names: ex, vi, and view. Typing man ex or man view would take you
to this same page.

22

C ha pt er 1

Navigating Man Pages
Once you’re in a man page, pressing the spacebar or the PGDN key takes you
forward one full screen. If you don’t want to go that far, pressing ENTER or the
down arrow scrolls down one line. Typing B or pressing the PGUP key takes
you back one screen. To search within a man page, type / followed by the word
you’re searching for. You’ll jump down to the first appearance of the word,
which will be highlighted. Typing N subsequently takes you to the next
occurrence of the word.
This assumes that you’re using the default BSD pager, more(1). If you’re
using a different pager, use that pager’s syntax. Of course, if you know so
much about Unix that you’ve already set your preferred default pager, you’ve
probably skipped this part of the book entirely.

Finding Man Pages
New users often say that they’d be happy to read the man pages, if they
could find the right one. You can perform basic keyword searches on the
man pages with apropos(1) and whatis(1). apropos(1) searches for any man page
name or description that includes the word you specify. whatis(1) does the
same search, but only matches whole words. For example, if you’re interested
in the vi command, you might try the following:
$ apropos vi
BUS_ADD_CHILD(9)
- add a device node to the tree with a given priority
BUS_PRINT_CHILD(9)
- print information about a device
BUS_READ_IVAR(9), BUS_WRITE_IVAR(9) - manipulate bus-specific device instance
variables
DEVICE_ATTACH(9)
- attach a device
...

This continues for a total of 581 entries, which is probably far more
than you want to look at. Most of these have nothing to do with vi(1),
however; the letters vi just appear in the name or description. Device driver is
a fairly common term in the manual, so that’s not surprising. On the other
hand, whatis(1) gives more useful results in this case.
$ whatis vi
ex(1), vi(1), view(1)
etags(1), ctags(1)
$

- text editors
- generate tag file for Emacs, vi

There are only two results, and both clearly have relevance to vi(1). On
other searches, apropos(1) gives better results than whatis(1). Experiment with
both and you’ll quickly learn how they fit your style.

Get t in g M ore Hel p

23

Section Numbers and Man
You might find cases where a single command appears in multiple parts of
the manual. For example, every man section has an introductory man page
that explains the contents of the section. To specify a section to search for a
man page, give the number immediately after the man command.
$ man 3 intro

This pulls up the introduction to section 3 of the manual. I recommend
you read the intro pages to each section of the manual, if only to help you
understand the breadth and depth of information available.

Man Page Contents
Man pages are divided into sections. While the author can put just about any
heading he likes into a man page, several are standard. See mdoc(7) for a
partial list of these headings as well as other man page standards:
NAME gives the name(s) of a program or utility. Some programs have
multiple names—for example, the vi(1) text editor is also available as
ex(1) and view(1).
SYNOPSIS lists the possible command-line options and their arguments,
or how a library call is accessed. If I’m already familiar with a program
but just can’t remember the option I’m looking for, I find that this
header is sufficient to remind me of what I need.
DESCRIPTION contains a brief description of the program, library, or
feature. The contents of this section vary widely depending on the topic,
as programs, files, and libraries all have very different documentation
requirements.
OPTIONS gives a program’s command-line options and their effects.
BUGS describes known problems with the code and can frequently save
a lot of headaches. How many times have you wrestled with a computer
problem only to learn that it doesn’t work the way you would expect
under those circumstances? The goal of the BUGS section is to save you
time and describe known errors and other weirdness.1
SEE ALSO is traditionally the last section of a man page. Remember that
Unix is like a language, and the system is an interrelated whole. Like duct
tape, the SEE ALSO links hold everything together.
If you don’t have access to the manual pages at the moment, many
websites offer them. Among them is the main FreeBSD website.

1

24

C ha pt er 1

It’s called honesty. IT professionals may find this term unfamiliar, but a dictionary can help.

FreeBSD.org
The FreeBSD website (http://www.freebsd.org) contains a variety of information about general FreeBSD administration, installation, and management.
The most useful portions are the Handbook, the FAQ, and the mailing list
archives, but you’ll also find a wide number of articles on dozens of topics.
In addition to documents about FreeBSD, the website also contains a great
deal of information about the FreeBSD Project’s internal management and
the status of various parts of the Project.
If you find that the main website works slowly for you, try using a mirror
site. The main site offers a drop-down box with a choice of national mirrors,
or you can just try http://www..freebsd.org. Almost every country
has a local site that provides a duplicate of the FreeBSD website. I frequently
find that a mirror is more responsive than the main website.

Web Documents
The FreeBSD documentation is divided into articles and books. The difference
between the two is highly arbitrary: As a rule, books are longer than articles
and cover broader topics, while articles are short and focus on a single topic.
The two books that should most interest new users are the Handbook and
the Frequently Asked Questions (FAQ).
The Handbook is the FreeBSD Project’s tutorial-style manual. It is continuously updated, describes how to perform basic system tasks, and is an
excellent reference when you’re first starting on a project. In fact, I have
deliberately chosen not to include some topics in this book because they
have adequate coverage in the Handbook.
The FAQ is designed to provide quick answers to the questions most
frequently asked on the FreeBSD mailing lists. Some of the answers aren’t
suitable for inclusion in the Handbook, while others just point to the proper
Handbook chapter or article.
Several other books cover a variety of topics, from kernel debugging to
Project organization.
Of the 50 or so articles available, some are kept only for historical reasons
(such as the road map to releasing FreeBSD version 5), while others discuss
the subtleties of specific parts of the system such as serial ports or CVSup.
A few are old enough that they’re retained for only a handful of users who
are still stuck with 20th-century systems.
These documents are very formal, and they require preparation. As such,
they always lag a bit behind the real world. When a new feature is first rolled
out, the appropriate Handbook entry might not appear for weeks or months.
If the web documentation seems out of date, your best resource for up-to-theminute answers is the mailing list archive.

Get t in g M ore Hel p

25

The Mailing List Archives
Unless you’re really on the bleeding edge, someone has probably struggled
with your problem before and posted a question about it to the mailing lists.
After all, the archives go back to 1994 and contain close to two million messages. The only problem is that there are two million pieces of email, any one
of which might contain the answer you seek. (When the first edition of this
book came out, the archives contained only one million messages; they have
nearly doubled in size in the last few years!)
While FreeBSD provides a search facility for its web pages and the
mailing list archive, it pales beside the one offered by Google. Google has
a BSD-specific search site at http://www.google.com/bsd. Search for your error
message on Google, both in the regular web search and the Groups search.
Google Groups also indexes the FreeBSD mailing lists, and you can search
the FreeBSD.org website on Google by including the search term site:freebsd.org
in your query. Additionally, the Rambler search engine has a very good
FreeBSD-specific search engine at http://freebsd.rambler.ru. Rambler runs on
FreeBSD, and it employs at least one FreeBSD committer.

Other Websites
FreeBSD’s users have built a plethora of websites that you might check for
answers, help, education, products, and general hobnobbing. Here are some
of my favorites:
Daemon News (http://bsdnews.com)
This site provides links to news postings on all BSD topics, not just
FreeBSD.
FreeBSD Mall (http://www.freebsdmall.com)
The people who run FreeBSD Mall have been commercial supporters of
FreeBSD since the beginning. They sell FreeBSD on CD and DVD and
offer training and support contracts, as well as FreeBSD paraphernalia
such as clothes and toys. FreeBSD Mall is owned by IX Systems.
O’Reilly Network BSD Developer Center (http://www.onlamp.com/bsd )
This site hosts a variety of BSD articles, as well as content of interest to
BSD users. In my utterly unbiased opinion, the most fascinating thing
on the site is the Big Scary Daemons column on BSD, but everything else
there is also pretty good.

Using FreeBSD Problem-Solving Resources
Okay, let’s pick a common problem and use the FreeBSD resources to solve it.
I’ve seen this question more than once, on several different FreeBSD mailing
lists, so we’ll start with it.

26

C ha pt er 1

I’ve just installed FreeBSD on my 486 and the network isn’t
working. When I try to ping anything, the console shows
ed0: timeout. What’s wrong?

We’ll use several different methods to find an answer.

Checking the Handbook/FAQ
The Handbook doesn’t have anything relevant to the problem. In the FAQ,
however, this entry appears under Troubleshooting:
I keep seeing messages like "ed1: timeout". What's wrong?

That looks pretty darn close. Read the entry and try the solution presented.

Checking the Man Pages
As we go on, you’ll see that the numbers after device names are simply
instances of a particular device. If you see ed0, it just means device ed, unit
number 0. Every device driver has a man page, so if you type man ed to bring
up the manual entry for this device, you’ll see the following:
ED(1)

FreeBSD General Commands Manual

ED(1)

NAME
ed, red --

text editor

SYNOPSIS
ed [-] [-sx] [-p string] [file]
red [-] [-sx] [-p string] [file]
DESCRIPTION
The ed utility is a line-oriented text editor. It is used to create,
...

A text editor? What? My text editor is fine! Something obviously isn’t
right. Look closely at this man page; it’s from section 1 of the manual, the
General Commands section. You need to search the manual for other entries
containing ed. As the letters ed appear in an awful lot of manual pages, use
the more specific whatis(1) search.
$ whatis ed
ed(1), red(1)
ed(4)

- text editor
- NE-2000 and WD-80x3 Ethernet driver

Bingo! The text editor ed(1) is a general-purpose command. We want
the ed in section 4 of the manual. Type man 4 ed to bring up the manual page
for the network device. It’s pretty long, though, about 500 lines. Being lazy,

Get t in g M ore Hel p

27

I’d rather not read the whole thing—I’d rather just search for the part that
has the information I need. Looking at the error message, I guess that timeout
might be a good keyword to look for. Type /timeout and press ENTER.
ed%d: device timeout Indicates that an expected transmitter interrupt
did not occur. Usually caused by an interrupt conflict with another card
on the ISA bus.

Bingo again! Here we have a terse explanation of the problem and a
probable cause (interrupt timeout). We have a good old-fashioned IRQ
conflict, and if you’re actually on a 486, you know more about this problem
than you want to.

Checking the Mailing List Archives
You could use the FreeBSD website search engine to search the mailing list
archives, but I prefer either Google or Rambler. A search for ed0: timeout
site:FreeBSD.org spits out a whole bunch of results. Some of them date from
1994. When I did this right now, the first response answered the question.
When I did this for the first edition, the first result was correct then, as well.
Now, isn’t that faster than composing an email to a mailing list?

Using Your Answer
Any answer you get for our ed0 timeout example assumes that you know what
an IRQ is and how to adjust one on your hardware. This is fairly typical of the
level of expertise required for basic problems. If you get an answer that is
beyond your comprehension, you need to do the research to understand it.
While an experienced developer or system administrator is probably not
going to be interested in explaining IRQs to you, he or she might be willing
to point you to a web page that explains them, if you ask nicely.

A S K I N G AG A I N . . . A N D AG AI N . . .
A N D AG AIN . . .
Some of the emails answering this problem date from 1994. Yes, that’s right, over a
dozen years ago! Remember when I mentioned people being sick of answering the
same questions over and over again? Some of these questions have been asked
many times over the years. Be sure you’ve checked all the resources where you
might find assistance for your problem. If you truly can’t find any other help, then
perhaps your problem is unique enough to warrant broadcasting it to the world.

28

C ha pt er 1

Emailing for Help
When you finally decide to ask for help, do so in a way that allows people to
actually provide the assistance you need. You must include all the information
you have at your disposal, as we will soon discuss. There’s a lot of suggested
information to include, and you can choose to skip some or all of it. If you
slack off and fail to provide all the necessary information, one of the following
things will happen:
Your question will be ignored.
You will receive a barrage of email asking you to gather this information.
On the other hand, if you actually want help solving your problem,
include the following pieces of information in your message:
A complete problem description. A message like How do I make my modem
work? only generates a multitude of questions: What do you want your
modem to do? What kind of modem is it? What are the symptoms? What
happens when you try to use it? How are you trying to use it?
The output of uname -a. This gives the operating system version and
platform.
If you have upgraded your system via csup, give the date and time of your
last update. (This is the date of the newest files in /usr/src.)
Any error output. Be as complete as possible, and include any messages
from the console or from your logs, especially /var/log/messages and any
application-specific logs. Messages about hardware problems should
include a copy of /var/run/dmesg.boot.
It’s much better to start with a message like My modem isn’t dialing my ISP.
The modem is a BastardCorp v.90 model BOFH667. My OS is version 7.2 on a dualcore Opteron. There are no error messages in /var/log/messages or /var/log/ppp.log.
You’ll skip a whole round of email with a message like this, and you’ll get
better results more quickly.

Writing Your Email
First, be polite. People often say things in email that they wouldn’t dream
of saying to someone’s face. These lists are staffed by volunteers who are
answering your message out of sheer kindness. Before you click that Send
button, ask yourself, Would I be late for my date with the hot twins down the hall
to answer this message? 2 The fierce attitude that is occasionally necessary when
working with corporate telephone-based support only makes these knowledgeable people delete your emails unread. Their world doesn’t have to include

2
Several developers have assured me that they absolutely would accept a date with said hot twins
in lieu of politeness. Large sacks of money also suffice, preferably large, unmarked bills.

Get t in g M ore Hel p

29

surly jerks. Screaming until someone helps you is a valuable skill when dealing with commercial software support, but it will actively hurt your ability to
get FreeBSD support.
Send your email in plaintext, not HTML. Many FreeBSD developers read
their email with a text-only email program such as mutt or elm. These are
very powerful tools for handling large amounts of email, but they do not
display HTML messages without contortions. To see for yourself what this is
like, install /usr/ports/mail/mutt and read some HTML email with it. If you
are using a graphic mail client such as Microsoft Outlook, either send your
email in plaintext or make sure that your messages include both a plaintext
and an HTML version. All mail clients can do this; it’s just a question of
discovering where your GUI hides the buttons. What’s more, be sure to wrap
your text at 72 characters. Sending email in HTML, or without decent linewrapping, is an invitation to have your email discarded unread.
Harsh? Not at all, once you understand whom you’re writing to. Most
email clients are poorly suited to handling thousands of messages a day,
scattered across dozens of mailing lists, each containing a score of simultaneous conversations. The most popular email clients make reading email
easy, but they do not make it efficient; when you get that much email, efficiency is far more important than ease. As most people on those mailing lists
are in a similar situation, plaintext mail is very much the standard for them.
On a similar note, most attachments are unnecessary. You do not need
to use OpenPGP on messages sent to a public mailing list, and those businesscard attachments just demonstrate that you aren’t a system administrator.
Don’t use a long email signature. The standard for email signatures is four
lines. That’s it; four lines, each no longer than 72 characters. Long ASCII
art signatures are definitely out.
Second, stay on topic. If you are having a problem with X.org, check the
X.org website. If your window manager isn’t working, ask the people responsible for the window manager. Asking the FreeBSD folks to help you with
your Java Application Server configuration is like complaining to hardware
salespeople about your fast-food lunch. They might have an extra ketchup
packet, but it’s not really their problem. On the other hand, if you want
your FreeBSD system to no longer start the mail system at boot time, that’s
a FreeBSD issue.

Sending Your Email
When you’ve composed your nicely detailed and polite question, send it to
FreeBSD-questions@FreeBSD.org. Yes, there are other FreeBSD mailing lists,
some of which are probably dedicated to what you’re having trouble with.
As a new user, however, your question is almost certainly best suited to the
general questions mailing list. I’ve lurked on many of the other mailing lists
for a decade now, and have yet to see a new user ask a question on any of
them that wouldn’t have been better served by FreeBSD-questions. Generally, the
questioner is referred back to FreeBSD-questions anyway.

30

C ha pt er 1

This goes back to the first point about politeness. Sending a message to
the architectural mailing list asking about what architectures FreeBSD runs
on is only going to annoy the people who are trying to work on architectural
issues. You might get an answer, but you won’t make any friends. Conversely,
the people on FreeBSD-questions are there because they are volunteering to
help people just like you. They want to hear your intelligent, well-researched,
well-documented questions. Quite a few are FreeBSD developers, and some
are even Core members. Others are slightly more experienced users who
have transcended what you’re going through now and are willing to give
you a hand up, as well.

Responding to Email
Your answer might be a brief note with a URL, or even just two words: man
such-and-such. If that’s what you get, that’s where you need to go. Don’t ask
for more details until you’ve actually checked that resource. If you have a
question about the contents of the reference you’re given, or if you’re
confused by the reference, treat it as another problem. Narrow down the
source of your confusion, be specific, and ask about that. Man pages and
tutorials are not perfect, and some parts appear contradictory or mutually
exclusive until you understand them.
Finally, follow through. If someone asks you for more information,
provide it. If you don’t know how to provide it, learn how. If you develop
a bad reputation, nobody will want to help you.

Email Is Forever
Those of us who were on the Internet back in the ’80s remember when we
treated it as a private playground. We could say whatever we wanted, to whomever we wanted. After all, it was purely ephemeral. Nobody was keeping this
stuff; like CB radio, you could be a total jackass and get away with it.
That’s no longer true. In fact, it’s the exact opposite of true. Potential
employers, potential dates, even family members might scan the Internet for
your postings to mailing lists or message boards, trying to learn what sort of
person you are. I have rejected hiring more than one person based on their
postings to a mailing list. I want to work with a system administrator who sends
polite, professional messages to support forums, not childish and incoherent
rants without sufficient detail to offer any sort of guidance. And I’d think a
lot less of my in-laws if I stumbled across a message from one of them on
some message board where they acted like fools. The FreeBSD mailing lists
are widely archived; choose your words well, because they will haunt you for
decades.
Now that you know how to get more help when things go wrong, let’s
install FreeBSD.

Get t in g M ore Hel p

31

2
INSTALLING FREEBSD

Just getting FreeBSD running on your
computer isn’t enough, no matter how
satisfying it might be the first time. It’s important that your install be successful. Successful means
that your system must be configured appropriately for
its purpose. A web server, an email server, a desktop
system, or a database server all have different operational requirements, and
meeting those requirements can be greatly eased by planning before you
ever boot the hardware. Proper planning makes installing FreeBSD much
less painful. On the downside, you’ll get much less experience in reinstalling
FreeBSD, because you’ll only have to do it once. If mastering the installation
program is your only goal, you can skip all this boring stuff about “thinking
ahead” and go right to the middle of this chapter.
I’m assuming that you want to run FreeBSD in the real world, doing real
work, in a real environment. This environment might even be your laptop—
while you might argue that your laptop isn’t a real production system, I
challenge you to erase all the data on it without backing it up and tell me

that again. If you’re just using a test machine that you truly don’t care about,
then I still recommend following the best practices so that you develop good
habits.
Consider what hardware you need or have. Then, decide how to best use
that hardware, what parts of FreeBSD you need to install, and how to divide
your hard disk. Only after all of that can you actually boot your computer
and install FreeBSD. Finally, do some brief post-install setup, and your system
is ready to go!

FreeBSD Hardware
FreeBSD supports a lot of different hardware, including both different
architectures and devices for each architecture. One of the project’s goals is
to support the most widely available hardware, and the list of that hardware
has broadened over the last few years to include far more than the “personal
computer.” Today, the supported hardware includes:
amd64 AMD’s 64-bit extensions to the 32-bit i386, copied by Intel as
EM64T, and sometimes called x64. This hardware can run both the
32-bit i386 and 64-bit amd64 versions of FreeBSD. (Linux calls this the
x 86-64 platform.)
i386

The good old-fashioned Intel-compatible personal computer.

powerpc The PowerPC processor found in older Apple computers and
many embedded devices.
pc98

Similar to i386, but popular in Japan.

sparc64
xbox

Used in high-end servers from Sun Microsystems.

Yes, FreeBSD can run on Microsoft’s Xbox.

FreeBSD supports many network cards, hard drive controllers, and other
add-ons for each architecture. Since many of these architectures use similar
interfaces and hardware, this isn’t as much of a challenge as you might think:
SCSI is SCSI anywhere, and an Intel Ethernet card doesn’t become magically
different just by putting it in a sparc64 machine.
For the most part, FreeBSD doesn’t care about the supporting hardware
so long as it works. Most readers are primarily familiar with the i386 architecture, so that’s where we’ll spend a fair amount of time. The amd64
platform is quickly becoming popular, however, so we’ll touch on that, as
well as sparc64.
FreeBSD has been ported to a variety of other platforms, such as the ARM
architecture and Intel’s Itanium. These ports are either incomplete or of little
utility to anyone except a developer. While it’s nifty that many ARM boards
run FreeBSD, you can’t go to a computer shop and buy one to play with.
Although FreeBSD runs just fine on ancient hardware, that hardware
must be in acceptable condition. If your old Pentium crashes because it has
bad RAM, using FreeBSD won’t stop the crashes.

34

C ha pt er 2

Sample Hardware
This book was written using the following sample hardware:
Dual-core amd64 SATA Sager 9750
laptop
Dual-CPU Opteron rackmount
Pentium 800 i386 system

Soekris net4801 board
and case
Sun Ultra 1
External SCSI array

B U Y D R I N K S F O R T H ES E P EO P LE
Much of this hardware was a gift from people who liked the first edition of this book.
Their names all appear in the opening credits. If you find this book useful, I heartily
encourage you to buy any of them a drink, a meal, or a Maserati. I would have had
no crash boxes without them. Without crash boxes to test to destruction, I wouldn’t
have had the ability to learn FreeBSD’s real limits, especially after my boss forcefully
explained to me that paying customers do not appreciate being research subjects.

Proprietary Hardware
Some hardware vendors believe that keeping their hardware interfaces
secret prevents competitors from copying their designs and breaking into
their market. This has generally been proven to be a bad idea, especially
as the flood of generic parts has largely trampled these secretive hardware
manufacturers over the last few years. Yet a few vendors, especially video
and sound card makers, still cling to this strategy.
Developing device drivers for a piece of hardware without its interface
specifications is quite difficult. Some hardware can be well-supported without
full documentation and is common enough to make struggling through this
lack of documentation worthwhile. The FreeBSD sound driver team, in particular, has done an excellent job of reverse-engineering sound cards’ interfaces
and now provides generic sound card infrastructure that works well even for
poorly documented cards. Other hardware, such as the chipset used on the
PCI bus in Sun UltraSPARC III systems, cannot be supported without full
and complete documentation.
If a FreeBSD developer has specifications for a piece of hardware and
interest in that hardware, he’ll probably implement support for it. If not,
that hardware won’t work with FreeBSD. In most cases, unsupported proprietary hardware can be replaced with less expensive and more open
options.
Some hardware vendors provide closed-source binary drivers for
their hardware. For example, Nvidia offers a binary-only driver for their
video hardware. FreeBSD also employs some clever tricks to use Windows

I ns t al li ng F re eB SD

35

network drivers, notably those for the wireless Ethernet cards supported by
“Project Evil.”1 For the most part, however, the best support comes from
open-source FreeBSD drivers.

I S M Y H AR D W AR E S U P P O RT ED ?
The easiest way to tell if your particular hardware is supported is to check the
release notes for the release of FreeBSD you plan to install. The release notes are
available at http://www.freebsd.org.

What We Won’t Cover
We won’t cover ISA cards; PCI has been around for a decade now, and I
strenuously doubt that anyone uses ISA cards in a production setting.2 The
FreeBSD Handbook has decent instructions for making your ISA cards work.
PowerPC and pc98 are all older systems, generally in decline, so we won’t
bother discussing them specifically. Like a dinosaur, older server-grade hardware tends to be difficult to kill with anything short of a meteor strike. And
running FreeBSD on an Xbox, while fun, is more of a stunt than an idea
worth implementing in production.

Hardware Requirements
While FreeBSD has minuscule hardware requirements, you’ll get the best
results out of it if you give it enough to work with. The following recommendations are for i386 systems, but other platforms have similar requirements.
Chapter 19 discusses how to measure your system’s performance so that
you can maximize your hardware utilization.
Processor
Your brand of CPU is irrelevant. FreeBSD doesn’t care if you’re running an
Intel, AMD, IBM, or Cyrix/Via CPU. During the boot process, the FreeBSD
kernel probes the CPU and uses whatever chip features it finds. I’ve run
effective servers on 486 machines before—in fact, I’ve filled an Internet T1
with a 486. For you folks who are just learning, I recommend that you get a
Pentium or faster system. Some of the techniques in this book take days on a
486, and I’m no longer that patient. Those same operations take less than an
hour on my dual-core laptop.

1
Yes, this really is called Project Evil. And implementing the Windows kernel interface in the
FreeBSD kernel makes the project worthy of the name.
2
And if you are, you either have been in this business long enough that you probably aren’t even
reading this book, or you are a total nut job. Mind you, the latter is not a disadvantage in this
field.

36

C ha pt er 2

Memory
Memory (as in RAM) is good. Adding more RAM accelerates a system better
than anything else. I recommend at least 64MB of RAM, but if you have a
system with 256MB or greater you’ll find FreeBSD easier going. If you are
really trying to shrink your system, you can run a carefully crafted kernel in
16MB—but you can’t run the installer in that amount of memory.
Hard Drives
Hard drives can be a big performance bottleneck. While IDE drives are dirt
cheap, they don’t perform as well as SAS, SCSI, or even SATA drives. A SAS
or old-fashioned SCSI system transfers data to and from each drive at the full
controller speed, while IDE and SATA drives split their throughput between
all of the drives on the channel. A SCSI controller can have up to 15 drives
on a channel, while a standard IDE controller can have no more than 2.
SATA controllers tend to put only one drive on a channel, taking the easy
route to good throughput. While you can use splitters to attach more than
one drive to a SATA channel, multiple SATA drives on a single channel have
no greater throughput than a single drive. 15 drives, each running at full
speed, versus 2 drives averaging half speed, make a huge difference in the
amount of data throughput!
If you have IDE or SATA drives, put your hard disks on separate controllers if possible. Many systems now have a hard drive on one IDE controller
and a CD drive on the other. When you add a second hard drive, put it on
the same controller as the CD drive. Most likely, you won’t be using the CD
nearly as often as the hard drive, and this way each drive will have a dedicated
controller.
The base FreeBSD system can fit into 500MB, and stripped-down versions
can fit into 32MB. You’ll be happiest with at least 5GB of disk space on your
test system, although I’m assuming that you have at least 10GB. Some add-on
software requires far more disk space—building the OpenOffice.org suite,
for example, takes 10GB of /usr all on its own! Again, any hard drive new
enough to be workable will probably be at least that large.

Preinstall Decisions
Before installing your server, decide what you’ll use it for. Is this a web server?
Database server? Network logging server? We’ll discuss the requirements for
each in the appropriate section.

Partitioning
Partitions are logical divisions of a hard drive. FreeBSD can handle different
partitions in different ways, and can even allow different filesystems or different operating systems on different partitions. If you’re doing your first
FreeBSD install, and you really don’t know how you want to partition your
disk, you can just use the automated partitioning suggested by the installer.
If you have more complicated needs, I suggest that you write down your
desired partitioning on a piece of paper before you begin.
I ns t al li ng F re eB SD

37

Partitioning might seem like a pain. If you’re familiar with some other
Unix-like operating systems, such as some distributions of Linux, you might
want to create a single large root partition and put everything on it. If Windows
or Linux let you dump everything on one big disk, why divide your FreeBSD
disk into smaller, less flexible pieces? What are the advantages of partitioning?
On a physical level, different parts of the disk move at different speeds.
By putting frequently accessed data on the fastest parts of the disk, you
optimize system performance. The only way to arrange this is by using
partitions. On a logical level, FreeBSD handles each partition separately.
This means that you can set each partition to have different operating rules.
Partitions that contain user data should not have setuid programs (programs
that run as root), and you might not want them to have programs at all. You
can enforce that easily with partitions.
If the disk is damaged, chances are the damage is limited to a single
partition. You can boot the system from an intact partition and attempt to
recover data from the damaged partition. With a single large partition, any
damage to that partition becomes damage to your entire system, reducing or
eliminating chances of recovery.
Partitions can limit problems caused by poor system administration.
Unattended programs can completely fill a hard drive with logs. Larger hard
drives don’t mean that the problem takes longer to show up; they just mean
that software writes more logs. While Chapter 19 discusses ways to contain logs,
a full hard drive can even prevent you from connecting to the system to fix the
problem! Partitioning confines such problems to a subset of the system.
Finally, many backup programs—i.e., dump(8)—work at the partition
level. On a production system, you’ll want to set different backup strategies
for different types of data. FreeBSD’s standard partitions are / (root), swap
space, /var, /tmp, and /usr.
/ (root)
The root partition holds the core system configuration files, the kernel, and
the most essential Unix utilities. Every other partition lies “under” the root
partition or is subordinate to it. With an intact root partition, you can boot
the system to the bare-bones single-user mode and perform repairs on the
rest of the system. Your system needs fast access to the root partition, so put it
first on the disk. Because root holds only the basic utilities and configuration
files, it doesn’t need to be large; FreeBSD defaults to configuring 512MB for
a root partition, which is more than sufficient.
Swap Space
The next partition on your drive should be the swap space—the disk space
used by virtual memory. When FreeBSD uses up all the physical RAM, it
moves information that has been sitting idle from memory into swap. If
things go well, your system doesn’t need swap space—but if you do need
swap, it must be fast.
So, how much swap space do you need? This is a matter of long debates
between system administrators. The short answer is, “it depends.” Longrunning wisdom says that you should have at least twice as much swap as you
have physical memory. Long-running wisdom has become obsolete, however,
38

C ha pt er 2

and the capacity of modern systems has invalidated this rule of thumb.
When a process runs out of control and starts allocating memory (say, in an
infinite loop), the kernel will kill the process once the system runs out of
virtual memory. If your system has 6GB RAM and 9GB swap, this process
will need to consume 15GB of memory before the kernel kills it! i386
systems have about 3GB of virtual address space, and they must share that
with the kernel, shared libraries, the stack, and so on. The i386 platform
limits memory usage to 512MB per process, which means that the kernel
will stop a runaway process fairly quickly. 64-bit systems, like amd64, have
vast virtual memory space and a process could conceivably devour gigabytes
of memory. If a system is thrashing gigabytes of memory between disk and
RAM, it will be unresponsive, slow, and generally troubled. Today, you
should have enough swap to do your work. I recommend provisioning as
much swap space as you have RAM, perhaps even a few megabytes more.
The main use for swap on modern systems is for a dump in case of a
system panic and crash. For maximum safety, you want enough swap space
to dump the entire contents of your RAM to swap. This is a worst-case crash
dump. FreeBSD 7.0 and later defaults to using a kernel minidump, however,
which only dumps the kernel memory. A minidump is much smaller than a
full dump—a system with 8GB RAM has an average minidump size of about
250MB. You can probably get away with only providing 1GB of swap, which
leaves plenty of room for even a bloated kernel minidump.
/tmp
The /tmp directory is the system-wide temporary space, open to all system
users. If you do not create a separate /tmp partition, it will be included on
your root partition. This means that your system-wide temporary space will
be subject to the same conditions as the rest of your root drive. This probably
isn’t what you want, especially if you plan to mount your root partition
read-only.
Requirements for a /tmp directory are generally a matter of opinion—
after all, you can always just use a chunk of space in your home directory as
temporary space, and there’s always the /var/tmp directory if you have large
files that you need to work with temporarily. On a modern hard drive, I like
to have at least 512MB in a /tmp directory. Automated software installers
frequently want to extract files in /tmp, and having to work around these
installers when /tmp fills up is possible but tedious.
On systems where you don’t expect /tmp to use much space (for example,
web servers and database servers), you might want to use a memory filesystem
for /tmp. We’ll discuss memory filesystems in Chapter 8. If you intend to use a
memory filesystem, do not create a separate /tmp partition.
/var
The /var partition contains frequently changing logs, mail spools, temporary
run files, upgrade files from tools such as portsnap and FreeBSD-update,
and so on. If your server is a web server, your website logs go to this partition.
You might need to make it 2GB or more. On a small “generic” mail server or
web server I’d use a third of my remaining disk space for /var. If the server
I ns t al li ng F re eB SD

39

handles only email, databases, or logs, I’d kick this up to 70 percent or more,
or just assign sufficient space to the other partitions and throw everything
left on /var. If you’re really cramped for space, you might assign as little as
30MB to /var.
Make /var larger than physical memory. By default, FreeBSD writes crash
dumps to /var/crash. We’ll discuss crash dumps in Chapter 21, but for now,
take my word for it; if you have enough empty space in /var to write the
contents of your physical memory, that will help should you ever start having
serious system trouble.
/usr
The /usr partition holds the operating system programs, system source code,
compilers and libraries, add-on software, and all the other little details that
make the system actually do anything. Much of this changes only when you
upgrade your system. It also holds users’ home directories, which change
regularly and rapidly. If you have many users, consider creating a separate
/home partition. While you can assign quotas to control disk space, a separate
partition will protect your all-important OS files.
On a modern hard drive, I recommend using at least 6GB for /usr. This
provides enough room to run the operating system, store the main system
source code, and build upgrades to the next version of FreeBSD. On a web
server where users upload website files to their home directories, I suggest
giving this partition the majority of your hard drive.
Other Partitions
Experienced system administrators always have their favorite partitions;
also, some companies have standards on how systems should be partitioned.
Different Unix vendors have attempted to impose their partitioning standards
on the world. You’ll see partitions like /opt and /u1 on different Unix systems.
If you have a preferred partitioning scheme, use it. You can steer FreeBSD
to install add-on software in a different partition if you like. Or, you can have
users’ home directories in /gerbil if it makes you happy. The best advice I
have to offer to readers whom I’ll never meet and whose systems I will never
log on to is this: You are the one who must live with your partitioning, so
think first!

Multiple Hard Drives
If you have more than one hard drive of comparable quality, and you are not
using them for RAID, you can still make excellent use of them: Put your data
on one hard drive and the operating system on another. One of your partitions will contain the information that makes your server special. Database
servers store their data in /var, so put /var on its own hard drive. If it’s a web
server, put /usr on the second hard drive.

40

C ha pt er 2

If you have a special function for this server, consider making a private
partition just for that function. There’s nothing wrong with creating a /home,
/www, or /data partition on the second hard drive and dedicating that entire
drive to the system’s primary purpose.
In general, segregating your operating system from your data increases
system efficiency. Like all rules of thumb, this is debatable. But no system
administrator will tell you that this is an actively bad idea.
With multiple hard drives, you can improve the efficiency of your swap
space by splitting it amongst the drives. Put the first swap partition on the
second slot of the drive with your root partition, and the other swap partitions on the first slots of the other drives. This splits reads and writes among
multiple disk controllers and thus gives you some redundancy at the controller
level. Remember, however, that a crash dump must fit entirely within a single
swap partition.
For swap splitting to work best, however, the drives must be SAS or SCSI.
If you have IDE or SATA drives, they must be on different IDE controllers for
best results. Remember that each IDE controller splits its total data throughput
among all the hard drives connected to it. If you have two hard drives on the
same IDE controller and you’re accessing both drives simultaneously, each
disk works, on average, only half as fast as it would work alone on the same
channel. The major bottleneck in using swap space is disk speed, and you
won’t gain anything by creating contention on your IDE bus.
Another option is to gain some resiliency by implementing a softwarebased RAID. This provides protection against a hard drive failure by sharing
and mirroring the data amongst multiple hard drives. We discuss FreeBSD’s
RAID features in Chapter 18. Your slices on each drive must be of identical
size to use software RAID. This is easiest to accomplish if all your drives are
the same size, but that’s not strictly necessary.

Partition Block Size
This section describes options that can really impair system performance.
If you’re new to FreeBSD, read this section only for your information—don’t
actually try it! This is for experienced Unix administrators who know exactly
what they’re doing, or at least know enough to be leery of the whole topic.
Block size refers to the size of the filesystem building blocks used to store
files. Each block can be divided into fragments. FreeBSD defaults to 16KB
block sizes (16,384 bytes) and 2KB (2,048 bytes) fragments. Files use a combination of fragments and blocks. For example, a 15KB file would be assigned
to one block, while a 17KB file would be assigned to one block and one fragment. We’ll discuss blocks and fragments in Chapter 18.
If you know exactly what you’re doing, and you want to change the
block size, you can do that in the installer. Be warned that FreeBSD behaves
optimally if each block contains eight fragments; you can choose ratios other
than 1:8 but only at a performance cost.

I ns t al li ng F re eB SD

41

Choosing Your Distribution(s)
A distribution is a particular subset of FreeBSD. You’ll choose one or more
distributions during the installation process. While you can add pieces later,
it’s best and easiest just to make the right choice in the beginning. The
installer offers nine distribution sets:
All This contains absolutely everything that is considered part of
FreeBSD, including the X Window System. (FreeBSD uses the X.org
implentation of X.) If this is a test machine, definitely choose this option.
Developer
X-Developer

This includes everything except the games and X.
This includes everything except the games.

Kern-Developer This includes the FreeBSD programs and documentation, but only the kernel source code.
X-Kern-Developer
Window System.

This is the Kern-Developer distribution plus the X

User This includes the FreeBSD operating system programs and
documentation only—no source code, no X.
X-User

This is the User distribution plus X.

Minimal This contains only the core FreeBSD programs, without
documentation or source code of any sort. This is a good choice if your
disk is really, really small.
Custom

Define your own distribution set.

If you’re installing a test machine to learn FreeBSD on, definitely choose
All. An Internet server is probably best served by the User distribution, or
perhaps X-User if you’re already familiar with the X Window System. Power
users might want the Custom distribution.
Games?
Yes, FreeBSD includes very simple games. These are small, text-based games
that were typical on systems of 20 years ago. New users will find the FreeBSD
tips provided by fortune(6) useful, but if you want to play modern games
look in /usr/ports/games and read Chapter 11.

X W I N DO W S Y S T EM
The X Window System is the standard graphic interface for Unix-like operating
systems. If you expect to sit at the console of your machine on a regular basis and
do day-to-day work, you probably want the X Window System. If you don’t expect
to be using this system to browse the Web or perform other graphics-oriented tasks,
you probably don’t need the X Window System. You can always add the X Window
System later.

42

C ha pt er 2

The FreeBSD FTP Site
Just as the main source of information about FreeBSD is the FreeBSD
website, the main source of FreeBSD itself is the FreeBSD FTP server. You
can purchase CDs of FreeBSD, and while they’re a decent investment, many
people prefer to just use the Internet to grab what they need. Even if you
have a CD, you’ll interact with the FTP servers eventually.
The primary FreeBSD FTP server is ftp.freebsd.org, but many servers
mirror it to reduce the load on the primary server and provide speedy,
reliable access. You’ll find a comprehensive list of FreeBSD FTP servers
at http://www.freebsd.org, although you can also pick mirrors easily enough
without the list. Every mirror server has a name following this pattern:
ftp..freebsd.org

The country code is optional; if there’s no country code, it’s usually
assumed to be in the continental United States. For example, we have
ftp14.freebsd.org, ftp2.uk.freebsd.org, ftp5.ru.freebsd.org, and so on.
As a rule, the FTP mirrors with lower numbers are more heavily loaded
than those with higher numbers. Try a site around ftp12.freebsd.org, or some
high-numbered server under your country code, to see if you can get a speedy
connection.

FTP Server Content
Many FreeBSD mirrors also mirror other software, but all FreeBSD content
can be found under /pub/FreeBSD. While the contents of the FTP server vary
over time, let’s take a look at the important files found there:
CERT
ERRATA
ISO-IMAGES-amd64
ISO-IMAGES-i386
ISO-IMAGES-ia64
ISO-IMAGES-pc98
ISO-IMAGES-ppc
ISO-IMAGES-sparc64
README.TXT
distfiles
doc
ports
releases
snapshots
tools
torrents

I ns t al li ng F re eB SD

43

Lot of stuff, isn’t it? Fortunately, you don’t have to dig through all this to
get everything you need to install, but a few directories merit particular
attention:
CERT This directory contains all FreeBSD security advisories since the
project’s inception. We’ll discuss security advisories in Chapter 7.
ERRATA This directory contains all errata for different releases of
FreeBSD. We’ll discuss errata in Chapter 13.
ISO-IMAGES All of the directories that begin with ISO-IMAGES contain CD disc images for different architectures of FreeBSD. For example,
ISO-IMAGES-i386 contains ISO images for installing FreeBSD on the
i386 architecture. You can burn these images to CD to perform a CD
install. (See your CD recorder documentation for help in doing so.)
README.TXT These are the various subdirectories on the FTP site
and their contents. You might want to consult this file for the changes
since this book was written.
distfiles This directory contains quite a few source code and binary
files for the many third-party applications that run on FreeBSD. This is
definitely the largest directory on the FreeBSD.org FTP server; don’t just
download everything here or your hard drive might burst.
doc This directory contains the latest set of FreeBSD documentation,
subdivided by language. If you’re reading this book in English, you probably want the en (English) subdirectory. You’ll find all the articles and
books there in a variety of formats, compressed for easy downloading.
ports Within this directory you’ll find all the infrastructure and packages for the ports system. We’ll discuss ports in Chapter 11.
releases This directory contains the most recent versions of FreeBSD
released along each development track. Older versions can be found
on the server ftp-archive.freebsd.org. We’ll discuss development tracks in
Chapter 13.
snapshots This directory contains recent versions of FreeBSD-current
and FreeBSD-stable. This is where you’ll find the latest testing release of
the bleeding-edge and production versions of FreeBSD.
tools Here you’ll find various Windows programs that can be used to
prepare a multiboot system to run FreeBSD.
torrents BitTorrent users will find this directory useful; it contains torrent seeds for the most recent release(s) of FreeBSD. (If you don’t use
BitTorrent yet, you should check it out.)
Now that you know how to find everything you’ll need, let’s go on to the
install process itself.

44

C ha pt er 2

W H I CH V E R S I O N D O I W AN T ?
FreeBSD has released many different versions, and more are coming. We’ll
discuss FreeBSD versions and release numbers in detail in Chapter 13. In the
meantime, I suggest that you check out http://www.freebsd.org. On its front
page you’ll see a note that says Production Release and gives a version number.
Use that version.

The Install Process
One of the more interesting3 parts of a new operating system is figuring out
how to get the OS running on your computer in the first place. On many
modern systems it’s pretty straightforward: throw the CD into the system and
boot from it. However, FreeBSD can be used on systems so old that they don’t
support booting from CD. That’s no problem; you can boot just as well from
floppy disk. FreeBSD can also be used on systems so new that they don’t have
either floppy or CD drives. What then?
Any OS installation process has three parts: booting the installer program,
accessing the installation media, and copying the software onto the hard
drive. Even a Windows installer boots a “mini-Windows” to install Windows
proper. FreeBSD provides options for each of these stages. Once your computer is booted and you have a usable installation media, running through
the program to install the software to disk is straightforward.

Choosing Boot Media
If you have a system that boots from CD, this is probably the easiest way to go.
You can get FreeBSD CDs from a variety of vendors or from the FTP site. Make
sure that your computer’s BIOS is set to boot from CD before the hard disk
and reboot your computer with the FreeBSD disc in the CD drive. If you need
help with configuring your computer’s BIOS, check the manufacturer’s
documentation.
If your computer cannot boot from CD but can boot from floppy disk,
download floppy disk images from the Internet and boot from those. Many
older computers have CD drives that will not work as boot devices, but once
the system is running you can use them for installation media.
Some modern computers have neither a floppy disk nor a CD drive. This
is often the case with small rackmount servers, where space is expensive. With
such a system, you can either install a CD drive or use PXE installation, as
discussed in Chapter 20. (PXE installation requires bootstrapping from an
existing FreeBSD machine, however.)

3

The ancient Chinese curse “May you live in interesting times” certainly applies here.
I ns t al li ng F re eB SD

45

N O R EM O V AB LE - M ED I A D RI V ES ?
If your soon-to-be-FreeBSD machine lacks both a CD drive and a floppy, doesn’t
have the power cables or physical space to install a CD drive, cannot boot off a USB
device for whatever reason, and you don’t yet have the knowledge to set up a PXE
installer (this is a lot of “ifs,” but a whole slew of older small rackmount servers fit this
description), don’t despair. You can get a CD drive on your computer for the install.
The safest thing to do is remove your hard drive and install it on a system with a
removable-media drive. Unlike some other operating systems, FreeBSD will let you
install on one machine and run on another.
If that’s not an option, here’s a trick I’ve used more than once. (It might electrocute
the hardware or yourself, and will certainly invalidate your warranty. The author is
not responsible for barbecued hardware or system administrators!)
Find an old computer running any operating system with an IDE CD drive. Put the
old machine next to your FreeBSD box, unplug it, and open the case. Open the case
of your FreeBSD machine. On the old computer, detach the CD drive’s IDE cable at
the controller end. Leave the power attached to the CD drive. Attach the dangling IDE
cable from the old machine to an open port on your FreeBSD system’s IDE controller.
Turn on the old computer; the CD will power up even though it’s not attached to the
old computer’s IDE controller. Now turn on the new computer, and it will pick up the
CD as an attached device.
After the install, put everything back just the way you found it, and nobody will
ever know.

Choosing Installation Media
The two most common sources of installation media are CD and FTP.
CDs are great when you have many machines to install and these machines
have CD drives. They’re fast and easy, and work even if the network is down.
A variety of vendors produce FreeBSD CDs and DVDs. iX Systems, in particular, has supported FreeBSD for many years, and recently purchased
FreeBSD Mall, the original producer of FreeBSD CDs. The DVD sets have far
more content, including many files that can be downloaded separately from
the Internet, but the CD sets have everything you truly need. From now on
I’m going to only mention CDs, but everything that applies to a CD is also
true of a DVD. If you don’t want to purchase a CD, you can fetch an ISO
image from the FreeBSD FTP server and burn it to CD yourself.
Several dozen FTP servers carry FreeBSD ISO images, installation media,
and related materials. The FreeBSD installer can FTP the software directly
from these servers. To use the FTP installation method, however, you must
have a working Internet connection, and the installation speed will be largely
dependent upon the network between you and your chosen FTP server.
There is also a chance that an intruder has hacked into the FTP server and
uploaded a bad version of FreeBSD for the unsuspecting public, but the

46

C ha pt er 2

FreeBSD team watches carefully for such events and deals with them swiftly.
The FreeBSD release team also provides cryptographic checksums for every
release in the release announcement, which you can use to verify releases.

Preparing Boot Floppies
You will need several floppy disks (four as of this writing, but possibly more in
the future). Find the release directory for the architecture and version you
want to install. You’ll find a floppies subdirectory there. For example, for an
i386 system and FreeBSD release 7.0, look in ftp://ftp.freebsd.org/pub/freebsd/
releases/i386/7.0-RELEASE/floppies. (You’ll also find this directory in the root
directory of a FreeBSD CD.) You’ll find several files with the .flp extension,
one named boot.flp and several numbered kernX.flp files, such as kern1.flp and
kern2.flp. These files are floppy disk images. Download them all.
You need to put these images onto floppy disks. The catch is, you cannot
use basic file-level copying, such as drag-and-drop in Windows. An image file
must be copied onto the disk in a particular way.
If you’re already running a Unix-like system, the dd(1) command does
everything you need. You’ll need to know your floppy drive’s device name,
which is probably /dev/fd0, /dev/floppy, or /dev/rfd0. If the device name is
/dev/fd0, as it is on BSD systems, you’d enter
# dd if=kern1.flp of=/dev/fd0

to write the kern1.flp image to the floppy disk. Copy each disk image to a
separate floppy disk.
If you’re running Microsoft Windows, you’ll need a special utility to copy
disk images. Microsoft doesn’t provide one, but FreeBSD does, and you’ll
find it in the tools subdirectory of the main site. It’s called fdimage.exe.
This is a free Windows program to copy disk images, and it’s quite easy to
use. It takes only two arguments: the name of the image file and the name of
the drive the disk is in. For example, to copy the image boot.flp to the floppy
in your a: drive, open a DOS prompt and enter the following:
c:> fdimage boot.flp a:

Once the floppy drive finishes churning (which may take a while), repeat
the process for all other disk images you have downloaded.

Preparing Boot CDs
If you’ve purchased an official FreeBSD CD, your install media is ready. If
not, you need to choose an ISO image from the FTP site and burn it. The first
step is to find your image directory. Go to the FTP site and choose the ISO
image for your architecture. In that directory you’ll find a directory for each

I ns t al li ng F re eB SD

47

current release. For example, ISO images for FreeBSD 7.0 for i386 can be
found at ftp://ftp.freebsd.org/pub/freebsd/ISO-IMAGES-i386/7.0. You’ll find
multiple images there.
The name of an ISO image is composed of the release number, the label
RELEASE, the architecture, and a comment, all separated by hyphens. For
example, these are the names of the ISO images available for 7.0:
7.0-RELEASE-i386-bootonly.iso
7.0-RELEASE-i386-disc1.iso
7.0-RELEASE-i386-disc2.iso

The image labeled disc1 contains the entire FreeBSD distribution, the
X Window System, a few basic packages, and a live filesystem that can be used
to perform repairs when your server goes bad.
The image labeled disc2 contains the most popular pieces of software for
FreeBSD precompiled and ready for use with this release.
The bootonly image boots the FreeBSD installer so that you can do an FTP
install. Many people ask, “If you already have a CD drive, why would you want
to do an FTP install?” The standard FreeBSD ISO image contains a lot of stuff.
If you’re not installing the full distribution, you won’t need a lot of it. Not
everyone has unlimited, unmetered bandwidth on tap.4
Once you’ve chosen your image, burn it to CD. CD burning methods vary
widely among operating systems; even within the Unix-like world, different
operating systems have chosen different ways to burn CDs. On Windows,
many CD burning programs are available, such as Nero and Stomp. Here’s
how you would burn an image to disc on a FreeBSD system with a standard
IDE CD burner:
# burncd -f /dev/acd0 data imagename fixate

Check your operating system’s instructions on burning an image file to
physical media. Be sure to burn this file as an image, not as a regular file.
One clear hint that you’re doing it wrong is if your burning software complains
that the file won’t fit on a single CD. The image file will overflow a single CD
if you’re burning it as a regular file, but not if you burn it as an image.

FTP Media Setup
If you’re installing from CD, the install media is ready—it’s the same disk
you’re booting from. But to do an FTP install, you must choose an FTP server
and understand how to connect your machine to the local network.
Choosing an FTP server is half guesswork. Find the list of FTP mirror
sites and start pinging them. You’re looking for an FTP server with low ping
times—that’s a good sign that it’s fairly accessible from your location. Once
you have a couple of candidates, FTP to them from your desktop machine.

4

48

C ha pt er 2

And those of us who do have it must learn to refrain from taunting those of you who don’t.

See how responsive they are. Pick one that feels snappy, and make sure that
it has the release you want to install. Take note of the FTP server’s name for
use in the install process.
If your local network uses Dynamic Host Configuration Protocol (DHCP)
to assign IP addresses and other network information, you’re ready to go.
Otherwise, if your network administrators assign IP addresses by hand, get
the following information from them:
IP address for your FreeBSD system
Netmask for your FreeBSD system
IP addresses of nameservers for your network
IP address of your default gateway
Proxy server information (if necessary)
Without this information—and without DHCP—you will be unable to
connect to a network to perform an FTP install.

Actually Installing FreeBSD
Now that you’ve made all the decisions about how you’re going to install
FreeBSD, all that remains is the grunt work of walking through the installer.
Put your boot media in the drive and power up the computer. You’ll see a
series of startup screens and system debugging information, which we cover
in Chapter 3.
The first menu you see will offer you a chance to choose your keyboard
layout. This includes a list of all the keyboard maps supported by FreeBSD.
Note that this does not affect the language of the installer, merely the keyboard layout.
FreeBSD next presents you with the first installation screen (Figure 2-1).

Figure 2-1: The main sysinstall screen
I ns t al li ng F re eB SD

49

This is sysinstall(8), the notoriously ugly FreeBSD installer. While other
operating systems have pretty graphical installers with mouse-driven menus
and multicolor pie charts, FreeBSD’s looks like an old DOS program. While
replacements have been promised time and time again, as I write this it looks
like sysinstall will be with us for the foreseeable future.
Use the spacebar to select options from sysinstall menus, not the
ENTER key.
Use the arrow keys to go down to the Standard installation, and press
ENTER. You’ll see the fdisk warning with some simple instructions (Figure 2-2).

Figure 2-2: The fdisk instructions

Skim the instructions to be sure they haven’t changed since this was
printed, and then press ENTER.
If you have multiple hard drives, FreeBSD will let you choose which drive
you want to install on. Press the spacebar to select a drive (Figure 2-3).

Figure 2-3: Selecting an installation drive

50

C ha pt er 2

Some hard drives will flash up a scary-looking warning about disk
geometry at this point. It is not a concern on most modern hardware. We will
talk about disk geometry in Chapters 8 and 18; you can look there if you’re
interested. Just press ENTER to continue to the fdisk screen (Figure 2-4).

Figure 2-4: The fdisk menu

Here you determine how much of your hard drive you want to use for
FreeBSD. For a server, you want to use the entire hard drive. Press A to
allocate the whole hard drive to FreeBSD, and then press Q to finish. The
installer will drop you into the MBR selector, shown in Figure 2-5.

Figure 2-5: The MBR installer

I ns t al li ng F re eB SD

51

Arrow down to Standard, then TAB to highlight OK. This installs a
standard master boot record (MBR), which removes any existing boot
manager that your computer could use if it booted any other operating
system. (We’re building Internet servers and won’t be sharing the hard
drive with, say, Windows Vista.) Press ENTER to proceed.
If you have multiple hard drives, the installer will return you to the hard
drive selection screen. Choose your next hard drive, or use the TAB key to
take you down to the OK button and proceed to the next step of the install;
sysinstall then displays instructions for using the partitioning tool (Figure 2-6).

Figure 2-6: Partitioning instructions

Read the instructions to be sure they haven’t changed since this was
printed, then press ENTER to continue.
You should now have the partitioning menu. We talked about partitioning earlier in this chapter, and you should have already made your decisions
on how to partition your drive. This is where you implement your choices
(Figure 2-7).
To take FreeBSD’s default, generic partition recommendations, press A.
Otherwise, press C to create a partition. You’ll get a box asking for the size of
your partition. Enter the desired partition size, using M for megabytes and G
for gigabytes. The installer will then ask you if this is a filesystem or a swap
space. If you say it’s a partition, it will ask you for the partition mount point
(/, /usr, /var, and so on).
When you have created all your partitions, press Q to exit the partition
editor.

52

C ha pt er 2

Figure 2-7: The partition editor

Now you’ll be asked for an installation source (Figure 2-8).

Figure 2-8: Installation media choices

Arrow down to highlight your installation media, and press ENTER to
select it. FreeBSD will either spin up your CD to confirm it’s usable, ask you
to select your FTP server, or ask you to configure whatever other installation
media you’ve chosen. I recommend using either FTP or CD.

I ns t al li ng F re eB SD

53

The next menu asks how much of FreeBSD you would like to install
(Figure 2-9). While FreeBSD offers many stripped-down versions for limited
hard drives, these days hard drives are much, much larger than FreeBSD.
On a vaguely modern machine I recommend always installing everything,
especially if you’re just learning about FreeBSD. Arrow down to highlight All
and use ENTER to select it.

Figure 2-9: Choosing a distribution set

sysinstall then asks if you want to install the Ports Collection. You do,
even though you don’t know what it is yet. Select Yes.
You’ll be brought back to the distribution selection menu. Arrow up to
Exit this menu and press ENTER.
sysinstall offers you a last chance to change your mind before installing.
Once you say Yes, install, sysinstall will format your hard drive, your CD drive
will light up, and in a few minutes you’ll have a FreeBSD install.
The installer will then ask several questions to set up basic system services
for you.

Configuring the Network
The installer asks if you want to set up a network device. Say Yes.
You’ll get a whole choice of network interfaces to configure (Figure 2-10).
Yes, FreeBSD can run TCP/IP over FireWire! It can also run TCP/IP over a
parallel port. Neither is terribly common, but it can be done. Look for an
entry that looks like an Ethernet card and choose it. In Figure 2-10, we see
an Intel EtherExpress Pro/100B PCI Fast Ethernet card that looks about right.
Scroll down and press ENTER to configure it.

54

C ha pt er 2

Figure 2-10: The network interfaces menu

You’ll be asked if you want to try IPv6 configuration of this interface. You
probably don’t. You’ll then be asked if you want to try DHCP configuration.
As this is a server, you probably don’t. This will take you to the Network Configuration screen shown in Figure 2-11.

Figure 2-11: Network configuration

Here you fill in your hostname and domain name, as well as the network
information you got from your network administrator.
Even if you use DHCP configuration, you must still set a host and a
domain. Otherwise, your system will boot calling itself Amnesiac. (You can
use a DHCP server to set a hostname, but that’s an advanced topic most
environments aren’t equipped to provide.)
I ns t al li ng F re eB SD

55

Miscellaneous Network Services
The installer then asks you several questions related to the system function.
Unless you are an experienced system administrator, you don’t want most of
these functions to start. We will enable some of them as we proceed through
the book. Once you understand the systems described, you can enable them
for later installs.
For example, the installer asks if this is a network gateway, or if you want to
configure inetd. Answer No to both. When asked if you want to enable SSH
login, say Yes—that is a secure, safe service required on almost all systems.
Do not enable the anonymous FTP server, the NFS server, the NFS client,
or customize syscons at this time.

Time Zone
The installer prompts you to set your time zone. You’ll be asked if the system
clock is set to UTC: Answer No and walk through the screens presented.
You’ll be asked to choose a continent, a country, and then a time zone.

Linux Mode
Now the installer will ask you if you want to enable Linux mode. I suggest you
answer No at this point. If you need Linux mode, we’ll learn how to activate
it in Chapter 12.

PS/2 Mouse
USB mice work automatically, but PS/2 and older mice need special setup.
The installer will offer to set up a PS/2 mouse for you. If you have a standard
two- or three-button PS/2 mouse plugged in, answer Yes and choose Enable
from the menu. You should see a mouse pointer on your screen, and it
should wiggle when you move it.
sysinstall will ask if your mouse is working. If the mouse pointer wiggles
when you move the mouse, you can answer Yes. In all honesty, I haven’t had
a PS/2 mouse fail on me in the last 10 years. Older types of mice can be
difficult, but are increasingly uncommon.

Adding Packages
The installer asks if you want to install any additional software packages.
If you’re an experienced system administrator, you probably know what
software you want to install. You probably have a favorite shell, and it’s
probably not installed on FreeBSD by default.
FreeBSD divides software packages into categories. Find the category
that you think should include your desired software, and select the category
to bring up a list of all the software on your install media in that category.
Find the software you want, and press the spacebar to select it. For example,

56

C ha pt er 2

to install the popular Bash shell, scroll down to the Shells category, press
ENTER, scroll down to Bash, and press the spacebar. Then press ENTER to go
back to the Package Selection menu.
When you have chosen all the packages you want to install, return to
the main Package Selection menu. Press TAB to move the cursor from OK to
Install, then press ENTER. Your system will install the selected packages.

Adding Users
Whenever possible, you should do everything while signed on as a regular
user and only use the root account when you must change the system. That
will happen frequently at first, but will grow less common as time passes.
Before you can sign on as a regular user, however, you must create a regular
user account. The installer gives you a chance to create users during the
installation process. Say Yes when asked and you’ll see Figure 2-12.

Figure 2-12: Adding a user

Your first selection in this screen should be the Login ID, or username.
Your company might have a standard for usernames. I prefer the first and
middle initial and full last name (not using the middle initial creates a
surprising number of duplicates).
FreeBSD assigns the UID.
The FreeBSD default is to have the user in a group of the same name as
the username; for example, the user mwlucas is automatically in the group
mwlucas. Experienced system administrators can change this.
Full name is the user’s full name. Other system users can see this name
when they log in, so don’t set it arbitrarily. I’ve seen new system administrators
get in trouble when they give a customer a full name of, say, Pain in the
Tuckus.

I ns t al li ng F re eB SD

57

Member groups is just a list of other system groups this account is part of. If
you want this user to be able to use the root password and become root, add
the group wheel in the Member groups space. Only system administrators
need to be in the wheel group.
The Home directory is where the users’ files are kept. The default is
generally fine.
Finally, choose a shell for your new user. Older admins and greybeardsin-training frequently prefer /bin/sh. The examples in this book are written in
the BSD standard shell /bin/tcsh, which I find a very friendly shell. If you have
a preferred choice, use it.
Select OK when you’re done to create your user.

Root Password
Now the installer tells you to set your root password. If your machine doesn’t
have a root password, anyone can log in without using any password. As root
has absolute control over your hardware and software, this would be bad.
FreeBSD will ask you to enter your root password twice. Remember your root
password, as recovering it is a bit of an annoyance. We talk about the root
password and security in Chapter 7.

Post-Installation Setup
Finally, you’re asked if you want to do any post-installation setup of your
FreeBSD server. The FreeBSD Configuration Menu (Figure 2-13) provides
an easy way to do basic initial setup on your computer.

Figure 2-13: Post-installation configuration

58

C ha pt er 2

In this menu you’ll be able to enable or disable everything the installer
asked during earlier parts of the install, as well as set all sorts of interesting
network functions. If you have an NTP server on your network, for example,
you can use the menus here to enable it on your FreeBSD machine. We will
see how to enable all of these services later in this book, but if you already
know what you’re doing you can configure them here as well.

Restart!
Once you’ve finished your post-install configuration, go back to the main
sysinstall menu and select Exit. Your computer will then reboot into a fully
installed FreeBSD system, ready to perform all of the examples in this book.
If you want to use sysinstall(8) later to configure your system, you can
run it at any time. By the end of this book, you’ll learn how to do everything
that sysinstall can do quicker and more flexibly at the command line.
Now let’s see what actually happened at that reboot.

I ns t al li ng F re eB SD

59

3
START ME UP!
THE BOOT PROCESS

While FreeBSD will boot easily and automatically when you turn on the power,
understanding exactly what happens at each
stage will make you a better system administrator.
Intervention during the boot process is rarely necessary,
but one day you’ll be glad you know how to do it. And once you’re comfortable
with adjusting the boot process, you’ll find you can solve problems you’ve
previously accepted and endured.
We’ll start by discussing how the system loader starts, then look at some
interesting changes you can make and the information you can gather from
the boot loader’s command line, including booting alternate kernels and
starting in single-user mode. We’ll cover serial consoles, a standard system
management tool. The FreeBSD multi-user startup process is responsible for
starting all the various services that make your computer useful, and we’ll
give attention to that as well. In addition, we’ll cover the information FreeBSD
records about the boot process and how FreeBSD turns itself off without
corrupting data.

RE CU RS IO N W A RN IN G !
Some of the topics in this chapter reference material found in later chapters. Those
later chapters, in turn, require that you understand this chapter first. There’s no good
place to begin learning. If you don’t quite understand a part of this chapter, just skim
over it and continue reading; it really will coalesce in your mind as you proceed.

The boot process itself can be divided into three main parts: the loader,
single-user startup, and multi-user startup.

Power-On and the Loader
Every i386 computer has a Basic Input/Output System (BIOS) with just
enough brains to look for an operating system somewhere on a disk. (Other
hardware platforms have console firmware or bootroms that perform the
same function.) If the BIOS finds an operating system on a disk, it hands
control of the computer to that operating system. If the BIOS doesn’t find
an operating system, it complains and gives up. Most BIOSes are rather dumb
and can only recognize operating systems by very simple indicators. The boot
blocks are sections of the disk that are specifically designed to be recognized
as an operating system by the BIOS. On those boot blocks, FreeBSD installs
software that is only smart enough to load the main FreeBSD startup program,
loader(8). The loader presents you with a FreeBSD logo on the right and a
menu of seven options on the left. These are the options:
1.

Boot FreeBSD [default]

2.

Boot FreeBSD with ACPI disabled

3.

Boot FreeBSD in safe mode

4.

Boot FreeBSD in single-user mode

5.

Boot FreeBSD with verbose logging

6.

Escape to loader prompt

7.

Reboot

If you wait 10 seconds, the loader will automatically boot FreeBSD by
default. Several other options are only needed for debugging or troubleshooting. While you don’t have to memorize all these options, you should be
comfortable with using the basic ones when required.
Boot FreeBSD with ACPI disabled
ACPI is the Advanced Configuration and Power Interface, an Intel/
Toshiba/Microsoft standard for configuring hardware. It replaces the
legacy standards APM (Advanced Power Management1), PnPBIOS, the
MP table, the $PIR table, and a whole bunch of other standards even
1

62

C ha pt er 3

The lesson here is: Never name anything advanced. One day, it won’t be.

more obscure. We discuss ACPI in Chapter 5. ACPI provides many
benefits to modern hardware, but some hardware has troublesome
ACPI implementations. On the other hand, much new SMP hardware
absolutely requires ACPI.
If your newly installed system will not boot normally, try booting it
with ACPI disabled. If your system has worked well for some time, but
suddenly has trouble booting, disabling ACPI probably won’t help.
Boot FreeBSD in safe mode
FreeBSD’s safe mode turns on just about every conservative option in the
operating system. ATA hard disks run without DMA or write caching,
limiting their speed but increasing their reliability by working around
cabling issues and other physical problems. EISA slots are not probed,
and ACPI is disabled. On i386 systems, SMP is disabled. USB keyboards
will no longer work in single-user mode. This option is useful for disaster
recovery and debugging older or otherwise troublesome hardware.
Boot FreeBSD in single-user mode
Single-user mode is a minimal startup mode that is very useful on damaged
systems, even when the damage was self-inflicted. It’s the earliest point
where FreeBSD can provide a command prompt, and is important
enough to have its own section later in this chapter.
Boot FreeBSD with verbose logging
FreeBSD learns a lot about a computer as it boots. Much of this information is irrelevant to day-to-day use, but very helpful when debugging.
When you boot in verbose mode, FreeBSD prints all the details it can
about every system setting and attached device. (This information will
be available afterwards in /var/run/dmesg.boot, as discussed later in this
chapter.) You might try verbose mode once on each of your machines,
just to glimpse the complexity within your computers.
Escape to loader prompt
The loader includes a command-line interpreter, where you can issue
commands to tweak your system to boot exactly the way you need. We’ll
cover this in detail in “The Loader Prompt” on page 66.
Reboot
Once more, this time with feeling! Of these options, the most important
are single-user mode and the loader prompt.

Single-User Mode
FreeBSD can perform a minimal boot, called single-user mode, that loads the
kernel and finds devices but doesn’t automatically set up your filesystems,
start the network, enable security, or run any standard Unix services. Singleuser mode is the first point at which the system can possibly give you a
command prompt, however, and you can perform any or all of those nonautomated activities yourself.

S ta rt M e U p! Th e Boo t Proc es s

63

When you choose a single-user mode boot, you’ll see the regular system
startup messages flow past. Before any programs start, however, the kernel
offers you a chance to choose a shell. You can enter any shell on the root
partition; I usually just take the default of /bin/sh, but use /bin/tcsh if you
prefer.

Disks in Single-User Mode
In single-user mode, the root partition is mounted read-only and no other
disks are mounted. (We’ll discuss disks and filesystems in Chapter 8, but for
now just follow along.)
Many of the programs that you’ll want to use are on partitions other than
the root, so you’ll want them all mounted read-write and available. To make
sure that your filesystems are in a usable state, run the following commands:
# fsck -p
# mount -a

The fsck(8) program “cleans” the filesystems, confirms that they are
internally consistent and that all the files that a disk thinks it has are actually
present and accounted for. Make the filesystems accessible with mount(8).
The -a flag mounts every filesystem listed in /etc/fstab (see Chapter 8),
but if one of these filesystems is causing your problems, you can mount the
desired filesystems individually by specifying them on the command line
(for example, mount /usr). If you’re an advanced user with NFS filesystems
configured (see Chapter 8), you’ll see error messages for those filesystems at
this point because the network isn’t up yet.
If you have trouble mounting partitions by name, try using the device
name instead. The device name for the root partition is probably either
/dev/ad0s1a (for IDE disks) or /dev/da0s1a (for SCSI disks). You’ll also need to
specify a mount point for this partition. For example, to mount your first IDE
disk partition as root, enter the command:
# mount /dev/ad0s1a /

If you have network filesystems on your server but your network is not yet
up, you can mount all your local partitions by specifying the filesystem type.
Here, we mount all of the local filesystems of type UFS, FreeBSD’s default
filesystem type:
# mount -a -t ufs

Programs Available in Single-User Mode
The commands available for your use depend on which partitions are
mounted. Some basic commands are available in /bin and /sbin, on the root
partition, and are available even if root is mounted read-only. Others live in
64

C ha pt er 3

/usr and are inaccessible until you mount that partition. (Take a look at /bin
and /sbin on your system to get an idea of what you’ll have to work with
when things go bad.)
NOTE

If you’ve scrambled your shared library system (see Chapter 12), none of these programs
will work. If you’re that unlucky, FreeBSD provides statically-linked versions of many
core utilities in the /rescue directory.

The Network in Single-User Mode
If you want to have network connectivity in single-user mode, use the shell
script /etc/netstart. This script calls the appropriate scripts to start the network,
gives IP addresses to interfaces, and enables packet filtering and routing. If
you want some, but not all, of these services, you’ll need to read that shell
script and execute the appropriate commands manually.

Uses for Single-User Mode
In single-user mode, your access to the system is only limited by your
knowledge of FreeBSD and Unix.
For example, if you’ve forgotten your root password you can reset it
from single-user mode:
# passwd
Changing local password for root
New Password:
Retype New Password:
#

NOTE

Note that you weren’t asked for the old root password. In single-user mode, you’re automatically root, and passwd(8) doesn’t ask root for any password.
Or, if you find that there’s a typo in /etc/fstab that confuses the system
and makes it unbootable, you can mount the root partition with the device
name, then edit /etc/fstab to resolve the issue.
Or, if you have a program that panics the system on boot and you need
to stop that program from starting again, you can either edit /etc/rc.conf to
disable the program, or just set the permissions on the startup script so that
it cannot execute.
# chmod 444 /usr/local/etc/rc.d/program.sh

We’ll discuss third-party programs (ports and packages) in Chapter 11.
NOTE

There’s a reason all of these examples involve recovering from human errors. Hardware
failures are not common, and FreeBSD failures even less so. If it wasn’t for human
error, our computers would almost never let us down. As you learn more about FreeBSD,
you’ll be more and more capable in single-user mode.

S ta rt M e U p! Th e Boo t Proc es s

65

We’ll refer to single-user mode throughout this book, but for now, let’s
look at the loader prompt.

The Loader Prompt
The loader prompt is a small computing environment that allows you to
make basic changes to your computer’s boot environment and the variables
that must be configured early in the boot process. When you escape to a
loader prompt (option 6 in the boot menu), you’ll see the following:
OK

This is the loader prompt. While the word OK might be friendly and
reassuring, it’s one of the few friendly things about the loader environment.
This is not a full-featured operating system; it’s a tool for configuring a system
boot which is not intended for the ignorant nor the faint of heart. Any changes
you make at the loader prompt only affect the current boot. To undo changes,
reboot again. (We’ll see how to make loader changes permanent in the next
section.)
To see all the commands available to the loader, enter a question mark.
OK ?
Available commands:
heap
show heap usage
reboot
reboot the system
bcachestat
get disk block cache stats
...

The first three commands in the loader, listed above, are pretty much
useless to anyone except a developer. Instead, we’ll focus on the commands
useful to a system administrator.
To view the disks that the loader knows about, use lsdev.
OK lsdev
cd devices:
disk devices:
disk0:
BIOS drive C:
disk0s1a: FFS
disk0s1b: swap
disk0s1d: FFS
disk0s1e: FFS
disk0s1f: FFS
disk1:
BIOS drive D:
disk1s1a: FFS
disk1s1b: swap
pxe devices:







The loader checks for CD drives and doesn’t find any. (The loader
will only find CD drives if you boot from a CD, so don’t be alarmed at this.)
It finds two hard drives, known to the BIOS as drives C and D . It then



66

C ha pt er 3



describes the partitions it finds on these hard drives. As we’ll see in Chapter 8,
the root partition generally ends in a . This means that the only root partition
here is shown as disk0s1a . On an unfamiliar system that’s having trouble
booting, you might find this knowledge useful.
The loader has variables set within the kernel and by a configuration file.
View these variables and their settings with the show command.
OK show
LINES=24
acpi_load=YES
autoboot_delay=NO
...

The spacebar advances to the next page. These values include IRQ and
memory addresses for old ISA cards, low-level kernel tunables, and information gleaned from the BIOS. We’ll see a partial list of loader variables in
“Loader Configuration” on page 69, and additional values will be brought
up throughout the book in the appropriate sections.
You can change these values for a single boot with the set command. For
example, to change the value console to comconsole, you would enter:
OK set console=comconsole

By the time the loader gives you a command prompt, it has already
loaded the system kernel into memory. The kernel is the heart of FreeBSD
and is detailed in Chapter 5. If you’ve never worked with a kernel before, just
file these tidbits away until you get to that chapter. Use the lsmod command to
view the kernel and kernel modules currently in memory.
OK lsmod
0x400000: /boot/kernel/kernel (elf kernel, 0x6a978c)
modules: elink.1 io.1 splash.1 agp.1 nfsserver.1 nfslock.1 nfs.1 nfs4.1
wlan.1 if_gif.1 if_faith.1 ether.1 sysvshm.1 sysvsem.1 sysvmsg.1 cd9660.1
isa.1 pseudofs.1 procfs.1 msdosfs.1 usb.1 cdce.0 random.1 ppbus.1 pci.1
pccard.1 null.1 mpt_raid.1 mpt.1 mpt_cam.1 mpt_core.1 miibus.1 mem.1 isp.1
sbp.1 fwe.1 firewire.1 exca.1 cardbus.1 ast.1 afd.1 acd.1 ataraid.1 atapci.1
ad.1 ata.1 ahc.1 ahd.1 ahd_pci.1 ahc_pci.1 ahc_isa.1 ahc_eisa.1 scsi_low.1
cam.1
0xaaa000: /boot/kernel/snd_via8233.ko (elf module, 0x6228)
modules: snd_via8233.1
0xab1000: /boot/kernel/sound.ko (elf module, 0x23898)
modules: sound.1
0xad5000: /boot/kernel/atapicam.ko (elf module, 0x4bac)
modules: atapicam.1











While some of this information is of value only to developers, a system
administrator can still learn a lot. Perhaps the most obviously useful information is the path to the loaded kernel . This should always be /boot/kernel/kernel
unless you configured the loader to look elsewhere.



S ta rt M e U p! Th e Boo t Proc es s

67

You’ll also get a list of the modules included in each loaded kernel file .
The example lists modules from the main kernel itself, ranging from elink to
cam . The loader has also pulled in the files snd_via8233 , sound , and
atapicam , with their respective modules.
To completely erase the loaded kernel and all modules from memory,
use the unload command.











OK unload

You won’t get any confirmation, but a subsequent lsmod will show that the
loader no longer remembers any kernel files.
To load a different kernel, use load.
OK load boot/kernel.good/kernel
boot/kernel.good/kernel text=0x4a6324 data=0x84020+0x9908c
syms=[0x4+0x67220+0x4+0x7e178]

The loader will respond with the name of the file and some low-level
information about it.
While I touch on loading alternate kernels here, before doing this you
really need to understand why you would want to and how to do it safely.
Go read the discussion on “Booting an Alternate Kernel” on page 137.
Once your system boots just the way you need it to, you’ll probably want
to make those settings permanent. FreeBSD lets you do this through the
loader configuration file, /boot/loader.conf. Before you can make changes,
however, you must understand FreeBSD’s default configuration filesystem.

Default Files
FreeBSD separates configuration files into default files and customization
files. The default files contain variable assignments and are not intended to
be edited; instead, they’re designed to be overridden by another file of the
same name. Default configurations are kept in a directory called default.
For example, the boot loader configuration file is /boot/loader.conf, and
the default configuration file is /boot/defaults/loader.conf. If you want to see a
comprehensive list of loader variables, check the default configuration file.
During upgrades, the installer replaces the default configuration files but
does not touch your local configuration files. This separation ensures that
your local changes remain intact while still allowing new values to be added to
the system. FreeBSD adds features with every release, and its developers go to
great lengths to ensure that changes to these files are backward compatible.
This means that you won’t have to go through the upgraded configuration
and manually merge in your changes; at most you’ll have to check out the
new defaults file for nifty configuration opportunities and new system
features.

68

C ha pt er 3

The loader configuration file is a good example of these files. The /boot/
defaults/loader.conf file contains dozens of entries much like this:
verbose_loading="NO"

# Set to YES for verbose loader output

The variable verbose_loading defaults to NO. To change this setting, do not
edit /boot/defaults/loader.conf—instead, add the line to /boot/loader.conf and
change it there. Your /boot/loader.conf entries override the default setting,
and your local configuration contains only your local changes. A sysadmin
can easily see what changes have been made and how this system differs from
the out-of-the-box configuration.

D O N ’ T C OP Y T H E DE F AU L T C O N F I G !
One common mistake is to copy the default configuration to the override file and
then make changes there directly. Such copying will cause major problems in certain
parts of the system. You might get away with it in one or two places, but eventually
it will bite you. Copying /etc/defaults/rc.conf to /etc/rc.conf, for example, will
prevent your system from booting. You have been warned.

The default configuration mechanism appears throughout FreeBSD,
especially in the core system configuration.

Loader Configuration
To make loader setting changes permanent, use the configuration file /boot/
loader.conf. Settings in this file are fed directly into the boot loader at system
startup. (Of course, if you enjoy being at your console every time the system
boots, then you don’t have to bother with this!)
If you look at the default loader configuration, you’ll see many options
that resemble variables listed in the loader. For example, here we can set the
name of the console device:
console="vidconsole"

Throughout the FreeBSD documentation, you’ll see references to boottime tunables and loader settings. All of these are set in loader.conf. This includes
many sysctl values that are read-only once the system is up and kicking.
(For more on this, see Chapter 5. I present a list of popular kernel sysctls
in Appendix A.) Here, we set the kernel variable kern.maxusers to 32 .
kern.maxusers="32"

S ta rt M e U p! Th e Boo t Proc es s

69

Some of these variables do not have a specific value set in loader.conf;
instead, they appear as empty quotes. This means that the loader normally
lets the kernel set this value, but if you want to override the kernel you can.
kern.nbuf=""

The kernel has an idea of what the value of kern.nbuf should be, but you
can have the loader dictate a different value if you must.
We’ll discuss system tuning via the boot loader in the appropriate
section—for example, kernel values will be discussed in Chapter 5, where
they will make something resembling sense—but here are some commonly
used loader values that affect the appearance and operation of the loader
itself and basic boot functionality. As FreeBSD matures, the developers introduce new loader values and alter the functionality of old ones, so be sure to
check /boot/defaults/loader.conf on your installation for the current list.
boot_verbose="NO"

This toggles the verbose boot mode that you can reach through the boot
menu. In a standard boot, the kernel prints out a few basic notes about
each device as it identifies system hardware. When you boot in verbose
mode, the kernel tells each device driver to print out any and all information it can about each device as well as display assorted kernel-related
setup details. This is useful for debugging and development, but not
generally for day-to-day use.
autoboot_delay="10"

This is the number of seconds between the display of the boot menu and
the automatic boot. I frequently turn this down to 2 or 3 seconds, as I
want my machines to come up as quickly as possible.
beastie_disable="NO"

This controls the appearance of the boot menu (originally, an ASCII art
image of the BSD “Beastie” mascot decorated the boot menu). If set to
YES, the boot menu will not appear.
loader_logo="fbsdbw"

You can choose which logo appears to the right of the boot menu.
The default “FreeBSD” in ASCII art is the fbsdbw option. Other options
include beastiebw (the original logo), beastie (the logo in color), and
none (to have the menu appear without any logo).

Serial Consoles
All this console stuff is nice, but it can be a problem when your FreeBSD
system is in a co-location facility on the other side of the country or on
another continent. A keyboard and monitor are nice, too, but in many data
centers you won’t have room for them. And how do you reset the machine
remotely when it won’t respond to the network? A serial console solves all
these problems and more.
70

C ha pt er 3

A serial console simply redirects the computer’s keyboard input and
video to the serial port instead of the keyboard and monitor. Serial consoles
appear on all sorts of network equipment, from Cisco routers and Ethernet
switches to network-based KVM switches. Many physical security systems,
such as keypad-based door locks, also have serial consoles. By hooking up a
standard null modem cable to the serial port and attaching the other end to
another computer’s serial port, you can access the first system’s boot messages
from the second computer. This is especially useful if the machines are at a
remote location. Your system must have a serial port to have a serial console.
An increasing number of systems are arriving “legacy-free,” meaning that they
lack such basic features as serial ports or PS/2 keyboard and mouse ports.
Serial consoles can occur in both hardware and software.

Hardware Serial Consoles
Real Unix hardware (such as Sparc64 systems) has hardware serial console
capability. On these boxes, you can attach a serial cable to the serial console
port and have unfettered access to the hardware configuration, boot messages,
and startup messages. Most x86 hardware does not allow this; you must be
at the keyboard looking at the monitor to control the BIOS, and you must
press the spacebar to interrupt the loader. A few x86 and amd64 motherboards do have this functionality, and more and more vendors such as Dell
and HP are offering serial port consoles as a feature on their higher-end
machines—but this is a special feature you must search for. (The HP RILOE
serial console support even lets you control power over the serial console,
which is very nice.)
If your machine doesn’t have a serial console, nothing any operating
system can do will give you access to the PC-style BIOS messages across the
serial port. Boot messages all appear before the operating system starts and
even before the hard drive is accessed. Fortunately, hardware exists to work
around this. The best I’ve seen is the PC Weasel (http://www.realweasel.com).
It’s a video card with a serial port instead of a video port. The PC Weasel lets
you access the BIOS, interrupt the boot to come up in single-user mode, and
in general do whatever you like with the system as if you were at the console.
Hardware serial consoles do not require any operating system support.

Software Serial Consoles
If you don’t need early access to the BIOS messages but only to the boot
loader, FreeBSD’s software serial console will suffice. As FreeBSD boots,
the loader decides where to print console messages and from where to
accept input. While this defaults to the monitor and keyboard, with a few
tweaks you can redirect the console to a serial port. You cannot access the
BIOS, but this serial console gives you the ability to tweak your boot in
almost any way. FreeBSD lets you set the console in two different places.
For production systems, it’s best to set the console in the file /boot/config.
This gives you access to the first stage of the boot process. You have three
choices: Use the standard keyboard/video/mouse as a console, use a serial
S ta rt M e U p! Th e Boo t Proc es s

71

port as a console, or use a dual console. The standard console is the default,
so choosing this setup requires no action. To force FreeBSD to use a serial
console, enter -h all by itself in /boot/config.
Dual consoles let you use either the standard or the serial console as
needed. You must pick one console to be the primary console, however.
There are certain low-level tasks, such as booting from an alternate loader or
breaking into the debugger, which you can only perform from the primary
console, but otherwise the consoles are functionally identical. Enter -D in
/boot/config to enable a dual console with the standard console as primary.
Enter -Dh in /boot/config to enable a dual console with the serial console as
primary. I recommend using a dual console.
You can also control consoles from /boot/loader.conf. These entries take
effect slightly later in the boot process, during the final stage of the kernel
bootstrapping process. To use the serial console exclusively, add this entry
to /boot/loader.conf:
console="comconsole"

To switch back to the default video console, remove this line or comment
it out. You can also set the keyboard and video console in /boot/loader.conf
explicitly with this line:
console="vidconsole"

You can specify a dual console configuration by listing both comconsole
and vidconsole, with the preferred console first. Here, we prefer the serial
console:
console="comconsole vidconsole"

If you’re in a server-room situation, you might want to switch back and
forth between a standard console and a serial console. I generally manage
large arrays of FreeBSD systems via the serial console.

K EY B O A R D A U T O D ET E C T I O N
In some FreeBSD documentation found on the Web, you’ll see references to using
keyboard autodetection to choose a console. The idea is that you want to use the
serial console unless you have a keyboard plugged in. This worked just fine in the
days of AT and PS/2 keyboards, but autodetection of USB keyboards is prone to
failure. You’re better off choosing a dual-console configuration rather than relying
on keyboard autodetection.

72

C ha pt er 3

Serial Console Physical Setup
No matter what sort of serial console you have, you’ll need to plug into it
correctly to make it work. You’ll need a null modem cable, available at any
computer store or from online vendors. While the gold-plated serial cables
are not worth the money, don’t buy the cheapest model you can find either;
if you have an emergency and need the serial console, you’re probably not in
the mood to deal with line noise!
Plug one end of the null modem cable into the serial console port
on your FreeBSD server—by default the first serial port (COM1 or sio0,
depending on what operating system you’re used to). You can change this
with a kernel recompile, but it’s generally simplest to just use the default
on a server.
Plug the other end of your null modem cable into an open serial port on
another system. I recommend either another FreeBSD (or other Unix) system
or a terminal server, but you can use a Windows box if that’s all you have.
If you have two FreeBSD machines at a remote location, make sure that
they each have two serial ports. Get two null modem cables and plug the first
serial port on each box into the second serial port of the other machine.
That way, you can use each machine as the console client for the other.
If you have three machines, daisy-chain them into a loop. By combining
twos and threes, you can get serial consoles on any number of systems.
I’ve worked data centers with 30 or 40 FreeBSD machines, where installing
monitors was simply not practical, and we used serial consoles to great effect.
Once you have a rack or two of servers, however, investing in a terminal
server is a really good idea. You can find them cheaply on eBay.
Another option is to use two DB9-to-RJ45 converters, one standard
and one crossover. These allow you to run your console connections over
a standard CAT5 cable. If you have a lights-out data center where human
beings are not allowed, you can have your serial consoles come out near
your desk, in your warm room, or anywhere else your standard Ethernetstyle patch panels reach. Most modern data facilities are better equipped
to handle Ethernet than serial cables.

Serial Console Use
Now that you’re all set up, configure your client to access the serial console.
The key to using a serial console is to remember the following settings:
9600 baud
8 bits
no parity
1 stop bit

S ta rt M e U p! Th e Boo t Proc es s

73

Enter these values into any terminal emulator on a client computer,
and the serial console will “just work.” You can find terminal emulators for
Microsoft platforms (HyperTerm being the most famous), Macintosh, and
almost any other operating system. A few years ago, I frequently used a Palm
handheld with a serial cable to access serial consoles.
FreeBSD accesses serial lines with tip(1), a program that allows you to
connect to remote systems in a manner similar to telnet. To run tip, do this
as root:
# tip portname

A port name is shorthand for specifying the serial port number and
speed to be used on a serial port. The file /etc/remote contains a list of port
names. Most of the entries in this file are relics of the eon when UUCP was
the major data-transfer protocol and serial lines were the norm instead of the
exception.2 At the end of this file, you’ll see a few entries like:
# Finger friendly shortcuts
sio0|com1:dv=/dev/cuad0:br#9600:pa=none:
sio1|com2:dv=/dev/cuad1:br#9600:pa=none:
sio2|com3:dv=/dev/cuad2:br#9600:pa=none:
sio3|com4:dv=/dev/cuad3:br#9600:pa=none:
sio4|com5:dv=/dev/cuad4:br#9600:pa=none:
sio5|com6:dv=/dev/cuad5:br#9600:pa=none:
sio6|com7:dv=/dev/cuad6:br#9600:pa=none:
sio7|com8:dv=/dev/cuad7:br#9600:pa=none:

The sio entries are the standard Unix-type device names, while the
com names were added for the convenience of people who grew up on x 86
hardware. Assume that you have two FreeBSD boxes wired back-to-back, with
each one’s serial port 1 null-modemed into serial port 2. Both machines are
configured to use a serial console. You’ll want to connect to your local serial
port 2 to talk to the other system’s serial console:
# tip sio1
connected

You won’t see anything else, no matter what you type.
If you log into the other system and reboot it, you’ll abruptly see action
in your tip window:
Shutting down daemon processes:.
Stopping cron.
Shutting down local daemons:.
Writing entropy file:.
Terminated
.
Waiting (max 60 seconds) for system process 'vnlru' to stop...done
Waiting (max 60 seconds) for system process 'bufdaemon' to stop...done

2

74

C ha pt er 3

This might not predate dinosaurs, but it was before spam. Imagine that.

Waiting (max 60 seconds) for system process 'syncer' to stop...
Syncing disks, vnodes remaining...1 0 0 done
All buffers synced.
Uptime: 1m1s
Shutting down ACPI
Rebooting...

There will be a long pause while the system runs its BIOS routines and
hands control over to the serial console. Eventually you’ll see something
like this:
/boot/kernel/kernel text=0x4a6324 data=0x84020+0x9908c
syms=[0x4+0x67220+0x4+0x7e178]
/boot/kernel/snd_via8233.ko text=0x3a14 data=0x328 syms=[0x4+0xa10+0x4+0xac5]
loading required module 'sound'
/boot/kernel/sound.ko text=0x17974 data=0x37a8+0x10d8
syms=[0x4+0x3290+0x4+0x3d7d]
/boot/kernel/atapicam.ko text=0x2a30 data=0x1d8+0x4 syms=[0x4+0x7b0+0x4+0x7d6]

This indicates that the loader initially found and read the kernel files
before showing the loader menu. Congratulations! You’re using a serial
console. Press the spacebar to interrupt the boot just as if you were at the
keyboard. It doesn’t matter how far away the system is; you can change
your booting kernel, get a verbose boot, bring it up in single-user mode,
or manually fsck the hard drive—whatever. A software serial console might
not show you the BIOS, but chances are that’s set up correctly already. Once
you’ve used a serial console for a while, it won’t matter if the machine is on
the other side of the world or the other side of the room; getting out of your
chair just to access the console will feel like too much work.
If you allow the boot to continue, however, you’ll get to a point where
the boot messages stop and the serial console freezes. This is because it’s a
console; it’s not a logon device. (Being able to log onto a machine via the
serial console is quite useful on occasion; see Chapter 20 for details.)
If a system in a remote location entirely locks up, you can connect to your
serial console and have the “remote hands” at the colocation facility powercycle the system. It might not be good for your computer, but it’s also not
good for it to be locked up. With the serial console, you can boot into singleuser mode and fix the problem by digging through the logs and whatever
other troubleshooting you feel capable of. We’ll discuss troubleshooting this
sort of problems in Chapter 21.

Serial Console Disconnection
The tip(1) program uses the tilde (~) as a control character. To disconnect
the serial console, press ENTER and then type the disconnect sequence
“tilde-dot” at any time:
~.

You’ll be gracefully disconnected.
S ta rt M e U p! Th e Boo t Proc es s

75

Startup Messages
A booting FreeBSD system displays messages indicating the hardware attached
to the system, the operating system version, and the status of various programs
and services as they start. These messages are important when you first install
your system or when you do troubleshooting. The boot messages always start
off the same way, with a statement listing the copyrights for the FreeBSD
Project and the Regents of the University of California:
Copyright (c) 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 7.0-CURRENT-SNAP010 #0: Tue Dec 13 11:25:44 UTC 2005
root@harlow.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC

We also get a notice of the version of FreeBSD that’s booting, along with
the date and time it was compiled. You can also see who compiled this kernel,
what machine it was built on, and even where in the filesystem this kernel was
built. If you build a lot of kernels, this information can be invaluable when
trying to identify exactly what system features are available.
WARNING: WITNESS option enabled, expect reduced performance.

The kernel will print out diagnostic messages throughout the boot
process. The message shown above means that I have debugging and faultidentifying code enabled in this particular kernel, and my performance will
suffer as a result. In this case I don’t care about the performance impact, for
reasons which will become clear momentarily.
Timecounter "i8254" frequency 1193182 Hz quality 0

This message identifies a particular piece of hardware. The timecounter,
or hardware clock, is a special piece of hardware, and while your computer
needs one, it’s such a low-level device that the end user really can’t do much
with it directly. Now and then, you’ll see messages like this for hardware that
isn’t directly visible to the user but is vital to the system. FreeBSD errs on the
side of printing too much information, rather than obscuring details that
might be critical. For example, it’ll also show all the information it can about
the CPU in the system:



CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ ( 2200.10-MHz 686-class CPU)
Origin = "AuthenticAMD" Id = 0x20fb1 Stepping = 1

 Features=0x178bfbff
Features2=0x1
AMD Features=0xe2500800
AMD Features2=0x3
Cores per package: 2

You probably didn’t know that a simple CPU could have so many details
and features, did you? Here’s why I’m not worried about the performance hit
caused by the WITNESS option shown earlier: This box has a dual-core processor , each core is pretty darn fast
and supports a whole bunch of
features important to modern CPUs as well as a few AMD-specific features .
I have CPU power to spare3 and a fair amount of memory as well.



real memory





= 1072693248 (1023 MB)
992 MB)

avail memory = 1040453632 (

The real memory is the amount of RAM physically installed in the
computer, while the avail memory is the amount of memory left over after
the kernel is loaded. I have 992 MB of RAM available for real work, which
more than suffices for the load on this system.





FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs

The kernel also prints messages about the hardware it finds and how it’s
going to handle that hardware. For example, in the snippet above, the kernel
announces that it’s found both cores of the CPU and is ready to manage them.

ioapic0  irqs 0-23 on motherboard
ioapic1  irqs 24-47 on motherboard

Here’s a fairly typical device driver entry. This device is known as ioapic,
and the kernel has found that this hardware is version 0.3 and has extra
information associated with it . What’s more, we’ve found two devices of
that type, numbered 0 and 1 . (All device drivers are numbered starting
with zero.) You can find out more about the device handled by a given driver
by reading the manual page for the driver. Almost all—but not all—device
drivers have manual pages.




npx0: [FAST]
npx0: 
Nothing to see here.
This is not the
site you're looking for.


Connection closed by foreign host.

If you’ve ever looked at any HTML, the output should look familiar. If
not, try the View Source option in your web browser next time you view a web
page. You’ll see that this is the actual HTML that generates the pretty picture
in your browser. If you can’t get this much from your web server, it probably
isn’t working.

The Apache Web Server
If you look under /usr/ports/www, you’ll see several different web servers.
You’ll find dhttpd, thttpd, bozohttpd, XS-HTTPD, and more, each with its
own special features. For example, tclhttpd is a web server implemented
entirely in Tcl. If you need a special-purpose web server, chances are it’s
available. We’ll concentrate on the most popular web server, Apache.
FreeBSD includes ports of several versions of Apache, but most of these
are legacy versions. If you have an application that specifically requires
Apache version 1.3 with particular modules, FreeBSD will support you. For
most new installs, however, you want the latest version. As of this writing,
that’s Apache 2.2. While Apache supports many different features, such as
LDAP and SQL backends, for the moment we’re just going to concentrate
on basic Apache configuration. Apache 2.2 installs fine either from the
port (/usr/ports/www/apache22) or from the package.

500

C h ap te r 1 7

Apache Configuration Files
You’ll find Apache 2.2’s configuration files in /usr/local/etc/apache22. While
Apache configuration has changed a lot over the years, the current configuration layout strikes a good compromise between complexity and modularity.
The key files are httpd.conf, mime.types, and magic.
mime.types
The mime.types file contains a list of all standard file types and their identifying
characteristics. When a web server is transmitting a file to a client, it must
identify the type of file so the client can handle it appropriately. You don’t
want your web browser to interpret a video stream as HTML! The mappings
in mime.types provide Apache with the necessary information to correctly
specify these types. The Apache’s mime.types file is largely comprehensive.
Even if a program’s documentation tells you to add some information to
mime.types, confirm that that information isn’t already there before making
any changes. You should almost never edit this file.
magic
The mime.types file cannot deal with every file type in the world. Apache’s
built-in mime_magic module uses the information in the magic file to try to
identify the otherwise unknown file types. Throughout my many years of
system administration, whenever I thought that I needed to change anything
in the magic file, I was wrong.
httpd.conf
The httpd.conf file is where the interesting things happen. The file is well
commented and is several hundred lines long, so I won’t exhaustively analyze
it. If you really want to know everything about Apache, you can find several
big books on the topic. We’ll cover those particular settings that you must
configure, as well as some popular options. Whatever you do, don’t change
anything you don’t understand.
The subdirectories Includes, extra, and envvars.d all add particular functions
to Apache, separating the core server functionality from frequently changed
add-ons. For example, each of the functions in the extra directory can be
enabled by uncommenting a single line in httpd.conf. We’ll see how this works
and how you can use this design to enforce your security policy.

Core Apache Configuration
While Apache supports many different features in its complicated configuration, getting a basic web server up and running is quite simple. First, set
apache_22_enable="YES" in /etc/rc.conf. After that, setting up your first website is
just a matter of making a few minor changes to httpd.conf.

We b a n d F T P Ser vi ces

501

Server Root Path
The ServerRoot setting specifies the directory containing all the website files
and server programs. When you reference another file in httpd.conf, Apache
prepends the ServerRoot to it unless you begin the filename with a slash (/).
ServerRoot "/usr/local"

With the default ServerRoot, a configuration entry of libexec/apache22 indicates the real directory of /usr/local/libexec/apache22. An entry of /var/log/httpd
.log remains unchanged, however. I don’t recommend changing this, simply
because so much of the default Apache configuration relies on ServerRoot
being /usr/local.
Listen
The Listen option controls which TCP ports or IP addresses Apache binds to.
The default is port 80 (the standard port for HTTP) on every IP address on
the local machine:
Listen 80

You can also specify individual IP addresses by listing them on a line:
Listen 192.168.8.44

By combining these, you can listen on a single IP address on an
unusual port:
Listen 192.168.8.44:8080

Use multiple Listen statements to make Apache available on any number
of ports or IP addresses on your system.
User and Group
These options specify an unprivileged user under which Apache will run.
FreeBSD ships with the user and group www specifically for web server use.
While occasionally you’ll hear someone suggest running Apache as root,
don’t do it no matter what; if an intruder breaks into an insecure application
on your web server, he’ll get root access as a side benefit!
User www
Group www

Administrator’s Email Address
The web server needs to know the email address of the system administrator.
Apache automatically inserts this information into various output and error
pages.

502

C h ap te r 1 7

ServerAdmin webmaster@blackhelicopters.org

Address harvesters can get this address pretty easily, so be sure to have
basic antispam protections on this address.
Server Name
This is the name of your website. It must be a real hostname with a DNS entry,
or Apache won’t start. For testing purposes, however, you can use an entry in
/etc/hosts instead of an actual DNS entry. You can also list an IP address.
ServerName www.absolutefreebsd.com

Document Root Path
The document root directory is where you place the HTML files that make
up the actual website. Apache’s default site contains a simple message
indicating the web server works, which is nice but won’t draw many visitors.
If you place your own HTML documents here, the Ports Collection will notice
the change and complain when you try to upgrade. It’s best to choose your
own directory for your own documents. I prefer to place my websites in
/var/www, as I try to keep the contents of /usr as static and unchanging as
possible.
DocumentRoot "/var/www/mywebsitename"

Apache Logs
Apache has fairly sophisticated logging facilities and allows you to choose
your desired level of detail in your logs. You can also design your own log
formats, but I highly recommend using one of the default log formats. Many
third-party tools can process and analyze Apache logs, but if you redesign
your logs, these tools will fail.
Basic Logs
The easiest log is the error log. If you have multiple websites running on a
single server, I recommend renaming the default error log to include the
site name:
ErrorLog /var/log/httpd-mysite-error.log

The log that interests most web administrators is the list of requests for
files. To log site access, define a log format and then declare where to place
the logs in that format. Apache 2.2 includes three log formats:
The common log format includes the IP address of the client, the time of
the request, and the file requested.

We b a n d F T P Ser vi ces

503

The combined log format includes everything from the common log format, as well as the site that referred the client to this site and the user
agent used by the client.
The combinedio log format includes everything from the combined log
format, as well as the actual number of bytes transferred over the
network. This data size information includes the headers and other
metainformation transferred with the actual documents.
Once you choose or define a log format, implement it with a CustomLog
statement:
CustomLog /var/log/httpd-mysite-access.log combinedio

You can have several CustomLog statements for each site and use each log for
a different purpose. For example, you may be using an Apache module that
provides special functionality that you want to log, but you don’t want to break
the web log analysis software used by your clients. Use multiple CustomLog
statements to write multiple logs for the same site.
Rotating Logs
A good rule of thumb is that each 10,000 requests generate about 1MB of log
files. This might not seem like much on a small site, but even a small website
can get that many requests over months. My (extraordinarily lame) personal
website gets approximately 60,000 hits a year.1 A busy site can generate this
many hits in minutes. If you don’t rotate your logs, finding information in
them becomes nearly impossible.
Apache doesn’t handle logging as gracefully as most other server programs, mainly because Apache is designed to handle thousands and thousands
of simultaneous users. If you simply use newsyslog(8) to rotate your logs as you
do for other programs, Apache will corrupt its own logs. Apache supports logging to programs, however. I recommend handling your logs via rotatelogs(8),
included with Apache. Use a piped call to rotatelogs(8) instead of a filename
in your log statements. Here’s an example:
ErrorLog "|/usr/local/sbin/rotatelogs /var/log/httpd-mysite-error-log 86400"

Instead of a filename, we have a call to rotatelogs(8) and then a base
filename. The trailing number is the number of seconds between log file
rotations, 86,400 seconds being equal to one day.
Similarly, use rotatelogs(8) for the access log by specifying the log type at
the end of the entry. You can also make use of the ServerRoot path to avoid
typing the entire path to rotatelogs(8).
CustomLog "|sbin/rotatelogs /var/log/httpd-mysite-access-log 86400" combined

1
This is not a plea for more web traffic to my home page. Really. Should you feel the urge to visit
my website, I suggest you get some fresh air and exercise instead.

504

C h ap te r 1 7

The following examples use ErrorLog only because those entries are
slightly shorter and therefore easier for your lazy author to type. They all
apply equally well to access logs.
You can rotate logs based on their size, which will be completely irrelevant
to time but might make your life easier. Here, we split the error log every
5 megabytes:
ErrorLog "|sbin/rotatelogs /var/log/httpd-mysite-error-log 5M"

By default, rotatelogs names each log file by the time the log starts, in
seconds since the epoch. While you can translate epochal time to humanfriendly time easily enough with date(1), you can also make rotatelogs use a
human-friendly date for the filename. Here, the error log’s filename includes
the time the file was created:
ErrorLog "|sbin/rotatelogs /var/log/httpd-mysite-error-log.%Y-%m-%d-%H_%M_%S 86400"

If you have a special logging situation that rotatelogs(8) cannot
solve, investigate cronolog (/usr/ports/sysutils/cronolog) or httplog
(/usr/ports/sysutils/httplog) instead. I’ve never been in such a situation,
however.

Apache Modules
Apache is a modular program, much like the FreeBSD kernel. Apache can
handle such diverse things as Microsoft Front Page extensions, scripting
languages (including PHP), and embedded Perl. Apache 2.2’s core functions
are contained in modules. You could choose to disable some of these, but
doing so would change the standard web server behavior and cause you grief.
You can find Apache modules to compress pages before transmission, vastly
decreasing bandwidth. FreeBSD has many modules in the Ports Collection
under /usr/ports/www. Module port names begin with mod_, such as mod_gzip.

A P AC H E M OD U L ES V S . K E RN E L M OD U L ES
While kernel modules are not Apache modules, FreeBSD includes a kernel module
just to optimize web servers. If a client sends a long HTTP request, that request might
take a long time (in computer terms) to completely arrive. The HTTP accept filter buffers
incoming HTTP traffic in the kernel until a complete request arrives. With this buffering,
the web server doesn’t waste time waiting for a request to arrive; it receives the
whole request at once and can act on it. Enable the HTTP accept filter by setting
apache22_http_accept_enable="YES" in /etc/rc.conf.

We b a n d F T P Ser vi ces

505

Load and unload Apache modules via the configuration file. The entries
look like this:
LoadModule authn_file_module libexec/apache22/mod_authn_file.so
LoadModule authn_dbm_module libexec/apache22/mod_authn_dbm.so
LoadModule authn_anon_module libexec/apache22/mod_authn_anon.so
...

Each entry consists of a LoadModule statement, the name of the module, and
the file where that module can be found. The filenames respect ServerRoot,
so these files are are actually in /usr/local/libexec/apache22.
Here are some of the popular Apache modules available in the Ports Collection. Other people’s definition of popular will probably differ from mine,
but that’s their prerogative.2 Most of these can be found in /usr/ports/www.
mod_bandwidth Allows controlling the amount of bandwidth used by
a site. Very useful for virtual servers where clients have a certain amount
of bandwidth per month.
mod_dtcl Embeds a Tcl interpreter in Apache, for faster Tcl-based
applications.
Accelerates CGI scripts.

mod_fastcgi

mod_gzip Accelerates websites and decreases bandwidth usage by
compressing content. Highly recommended.
mod_mp3

Turns Apache into a streaming MP3 server.

mod_perl2 Embeds a Perl interpreter in Apache, for faster Perl
applications.
mod_python
applications.

Embeds a Python interpreter in Apache, for faster Python

mod_ruby Embeds a Ruby interpreter in Apache, for faster Ruby
applications.
mod_webapp-apache2
tion servers.

Connects Apache to Tomcat for Java applica-

php5 Provides the popular PHP web scripting language. (This module
is in /usr/ports/lang/php5.) When you install PHP, it gives you the choice
to pull in MySQL, another popular tool.
You’ll find many other Apache modules scattered throughout the Ports
Collection. When you have an Apache problem, I suggest searching the Web;
chances are the software you need is already there.
Many of these ports configure themselves in httpd.conf upon installation.
If you’re using revision control on httpd.conf (as you should!), this means that
you’ll need to check out httpd.conf before you install the port. If a port doesn’t
configure itself upon installation, read the module documentation; chances
are there’s a good reason why. You might have to make some decisions
about how you want the software to behave before configuring Apache.
2

506

C h ap te r 1 7

Yes, I do allow people to disagree with me. Even though they’re wrong.

Directories and Permissions
Apache has many interesting features, but it’s not a good idea to enable
everything everywhere—a bit of sloppy programming can result in you broadcasting extra information from your website or even allowing an intruder to
break in. Apache permissions are set on a directory-by-directory basis. The
configuration looks a little like XML: You have a Directory label in angle
brackets, a list of permissions and settings, and then a closing Directory with a
backslash. Any options or settings between the opening and closing Directory
statements affect that directory:

...options and settings...


By default, Apache uses very restrictive permissions and settings. For
example, you’ll see the following entry right at the top of the directory listings
in httpd.conf:

AllowOverride None
Order deny,allow
Deny from all


Apache has features to allow users to override the default server configuration and change server options, password protection, MIME types, and so
on. The AllowOverride None line means that users cannot use these features
unless we explicitly permit them. The Order and Deny statements mean that no
directory on the system may be accessed from the Web. This is a default deny
security stance. Unless you specify otherwise, every directory on your system
has these permissions.
Now that you’ve disallowed everything, you can explicitly enable the
features you want. Here are some settings you might find of interest.

Controlling Access by IP Address
The Allow and Deny options list the IP addresses and hostnames that Apache
permits to access content in a directory. Apache compares client IP addresses
against the Allow and Deny lists in the order given in the Order statement, and
then permits or rejects access requests based on the results. When Order is
deny,allow, Apache permits access unless prohibited by a Deny statement. When
Order is allow,deny, Apache denies access unless explicitly permitted by an Allow
statement. The last matching rule applies.
Order allow,deny
Allow from all

We b a n d F T P Ser vi ces

507

This example, the default for the DocumentRoot directory, specifically sets
a default deny policy and then allows all hosts to access the site with the Allow
statement. As with TCP wrappers, you can use the special all host to indicate
every host. You can block some sites by adding a Deny statement to this default:
Order allow,deny
Deny from 192.168.0.0/16
Allow from all

Here, you’ve blocked certain IP addresses from reaching your site. You
could also use hostnames, if you’re willing to rely on reverse DNS for your
web server security:
Order allow,deny
Deny from *.absolutefreebsd.com
Allow from all

In this case, all I must do to access your site is walk over to a machine that
has reverse DNS not in the absolutefreebsd.com domain. Changing my reverse
DNS is easier than changing my IP address.
Inversely, you can easily restrict access to your internal website to your
company’s IP addresses only by doing something like this:
Order deny,allow
Allow from 192.168.0.0/16
Deny from all

You’ll see similar examples throughout the Apache configuration files.

Directory Options
Options are general server features enabled and disabled on a directory-bydirectory basis. They allow a web server to do all sorts of nifty things, such as
execute CGI scripts, password protect directories, and change language
handling. These options give web developers a lot of power but can also
generate a lot of support calls. Enabling only the necessary options reduces
the amount of time you spend troubleshooting problems later.
Inside a directory entry, specify options with the Options keyword. For
example, to enable the ExecCGI and MultiViews options in the directory
/var/www/mysite/cgi-bin, use the following configuration:

Options ExecCGI, MultiViews


Now let’s see what options Apache supports.
None

The None option disables all options. The sample httpd.conf ships with
Options set to None. If you’re building your httpd.conf on top of the default
configuration, you must explicitly enable any options you want to use.
508

C h ap te r 1 7

All

The All option is Apache’s built-in default. If you don’t specify any
options, almost any Apache option works on the website. If the user
uploads a password-protection script to control directory access, it will
work. If a user uploads a CGI script that exploits a system flaw to start a
root shell on a high-numbered port, granting everyone in the world a
backdoor onto your system, that will work too. The All option allows
every Apache option except MultiViews (as shown on the next page).
ExecCGI

Apache can run any CGI scripts in this directory.
FollowSymLinks

You can use symlinks—symbolic links, or file aliases, as discussed in ln(1)—to
point to other files or directories on the server. A user could symlink to
just about any file on the server, and that file will be visible if the file permissions allow.
SymLinksIfOwnerMatch

The server follows symlinks if the owner of the symlink is the owner of
the file that the symlink points to. In English, this means that a user can
employ symlinks only to point to her own documents.
Includes

Server-side includes (shell commands inside HTML files, also known as SSI)
and CGI scripts work in this directory. SSI can be a serious security risk
unless securely programmed. After all, you’re allowing anyone who can
see your website to run the command you use in your HTML page. Conniving visitors can make a command do things the web designer never
intended. Search the Web for discussions on the security of SSI, and you’ll
find enough to keep you busy for a long, long time. If you don’t know
how to use SSI safely, don’t enable this!
IncludesNOEXEC

This allows server-side includes, but disables the #exec feature and the
include function of server-side includes. Without the #exec feature, SSI
commands must be written within carefully restricted parameters. Basically, this permits simple server-side includes and CGI scripts but eliminates many common security holes. Just because the most common
security holes are eliminated doesn’t mean that this is safe; you’re just
making the intruder’s work a little harder.
Indexes

If a directory doesn’t contain an index document (index.html), the Indexes
option lets the server return a prettily formatted list of the directory
contents. You might consider this a security problem, depending on
the contents of your directory. For example, if someone browses the
directory of my personal web page I don’t really care, but if they browse
a directory containing my private code, I care a great deal. (In my case
it’s because my private code is embarrassing, but other people have code
that’s actually worth money.)
We b a n d F T P Ser vi ces

509

MultiViews

This option permits the server to handle HTML documents written in
multiple languages. For example, a web developer can write a single page
that contains text in English, Chinese, and Spanish. MultiViews allows the
server to negotiate the desired language with the client’s web browser.

Configuration by Users
One of the interesting things about Apache is that users may upload their own
configuration files, and the server will read and use them. The AllowOverride
keyword lets the Apache administrator dictate what configuration settings
users may or may not adjust in a given directory. This allows web developers
to handle much routine configuration themselves, as well as to install
insecure CGI scripts in random locations.
Users place configuration overrides in a file called .htaccess in the affected
directory. If you’re running a corporate web server and your web developer
basically gets what he wants, there’s no reason not to allow whatever override
he desires. If you’re running an ISP web server and you don’t allow a certain
group of clients to use CGI scripts, don’t enable the ExecCGI option and disable the override that permits CGI use.
AllowOverride appears on a single line within a Directory statement,
followed by the allowed overrides. Here is a reasonable set of defaults for
most websites:
AllowOverride FileInfo AuthConfig Limit Indexes

These are some valid AllowOverride statements that you might allow users
to adjust via a .htaccess file.
AuthConfig
AuthConfig allows the user to password protect directories. This is a safe
option; it is generally expected on server farms where any idiot with a
credit card can get an account.
FileInfo
FileInfo permits users to insert their own MIME information for files in

a directory. While it’s generally better to add MIME information to the
server’s mime.types file, some users think they need this.
Indexes

The Indexes override lets the user control how directory indexing is handled, including setting a new default document, controlling how icons
appear in server-generated indexes, and so on.
Limit

Users who can override Limit configuration can use the Allow, Deny, and
Order keywords to build their own hostname and IP address controls on
their directories. This option is also quite safe.

510

C h ap te r 1 7

None

The None override means that the user may not override any server configuration. This is a good default, but it is a little too restrictive for most
environments.
Options

Finally, you could allow your users to set their own directory options as
discussed in the previous section. This is useful if you trust your web
developers or if you don’t care that someone might upload an insecure
program and get your server compromised.

Other Directory Settings
While Apache lets you tweak all kinds of settings in each directory, we’re only
going to cover the most essential settings here. You can find many, many more
settings in the Apache documentation.
Index Documents
The DirectoryIndex statement defines the names of default documents in
directories. When a client requests a directory rather than a filename,
Apache checks for files with these names, in order. Change this if you’re
using a tool such as PHP (whose filenames end in .php) or those Windows
web page editors that use .htm or, worse, insist on naming the index page
default.htm.
DirectoryIndex index.php index.htm index.html

Aliases
Use the Alias statement to provide shortcuts to directories on your website,
much like a symlink. You can use an Alias statement to join disparate directories into a single coherent site without using FollowSymlinks like a maniac.
This is especially useful with third-party programs and web applications.
Alias /icons/ "/usr/local/www/icons"

This example means that if someone calls up http://www.absolutefreebsd
.com/icons, they would actually be viewing the directory /usr/local/www/icons,
even though the DocumentRoot of my site is nowhere near that directory.
You probably still need a Directory statement to grant permissions to the
aliased directory.
Custom Error Pages
In addition to serving web pages, web servers can also display errors to the
client. While Apache includes standard error pages, you can create your own
custom error pages and direct clients to them. Specify the ErrorDocument
keyword, the error number, and the name of the file to be served:
ErrorDocument 404

/missing.html
We b a n d F T P Ser vi ces

511

404 is the code for the classic “Page not found” error. When a user
requests a page that doesn’t exist, he gets the file missing.html instead.

Password Protection and Apache
Restricting a website to users with a username and password is a common
requirement. Apache can authenticate via all sorts of username and password
schemes, from integration with LDAP and Kerberos domains to databases to
plaintext files. We’re going to examine two standard methods of authentication: Apache password files and Radius authentication.
Password Files
Password files are very common in environments where you have only a few
users accessing a password-protected directory. They’re also popular in
virtual host environments, where users want to control access to their own
sites. Do not put the password file in a directory in the website itself, or users
could download it and try to crack the passwords. Put the password file in a
directory completely outside any website. If you have user accounts on your
system, and a user manages the website, you can put the password file in the
user’s home directory.
Create an empty password file with touch(1), then use htpasswd(1) to
add usernames and change user passwords. The syntax is very simple:
# htpasswd passwordfile username

For example, to add a user named mwlucas to the password file
webpasswords, I would run htpasswd webpasswords mwlucas. To change that
user’s password, I would use the exact same command. The password contains a single line for each user, listing the username and a password hash:
mwlucas:iJf2e7KIgS5i6

To remove a user from the database, just remove the corresponding entry
from the password file. You can also use htpasswd(1)’s -D flag, but I find that
removing users with vi(1) is easier than trying to remember this.
Now tell Apache to apply password protection to the directory. While
you can configure password protection directly in httpd.conf, it’s most common
to configure authentication in a .htaccess file. That way, changes to your
authentication system don’t necessitate reloading the entire web server. End
users commonly want to reconfigure their own authentication systems, and
using password files is an easy way to accomplish this. Define AllowOverride
AuthConfig for the password-protected directory (or the entire site), then
create a .htaccess file in the protected directory that looks something like this:
AuthName "Employees Only"
AuthType basic
AuthUserFile /home/mwlucas/sitepasswords
require valid-user

512

C h ap te r 1 7

AuthName is the text that appears in the password prompt box. You can
place any text you like between the quotes, and it will be shown by the web
browser.
AuthType tells Apache how to configure authentication with the client.
For standard usernames and passwords, use an AuthType of basic.
The AuthUserFile directive tells Apache where the user database is kept and
what kind of database it is. In this case, we’re pointing Apache to a htpasswd(1)
user file.
The require valid-user statement tells Apache that it should prompt for
a username and password, and only grant access to users who have valid
credentials.

Radius User Authentication
Radius is a decent way to authenticate against third-party directory services,
such as Active Directory and LDAP. You can find a whole variety of Radius
servers, from freely available OpenRADIUS to the Internet Authentication
Service bundled with Windows servers. The nice thing about Radius integration is that you don’t need to muck about with LDAP or Kerberos, because
enterprise directory management teams can be reluctant to allow nonstandard
systems to talk to the directory. In most cases, you can just build a Radius server
and let it authenticate against your enterprise directory without involving
anyone else. While an Internet search shows several Radius authenticators
for Apache, I prefer mod_auth_xradius (/usr/ports/www/mod_auth_xradius).
It works well with modern Apache and is fairly easy to configure.
First, configure your Radius server. If you’re in an enterprise or service
provider setting, chances are you already have a Radius server. Many different
vendors provide Radius servers, each with its own pluses and minuses. FreeBSD
includes several in the Ports Collection. The good news is that Apache doesn’t
need any fancy Radius features; any Radius server suffices. If you are in a
Microsoft Active Directory environment, check out Internet Authentication
Services. It’s very small, and chances are you can get your AD administrator
to install it for you. Your fellow employees will be happy that your web app
provides a logon integrated with their desktop accounts, when so many don’t.
Now tell Apache to load the module. Go to the end of the LoadModule list
and add it:
LoadModule auth_xradius_module libexec/apache22/mod_auth_xradius.so

Then, configure the Radius cache. Apache uses the cache to store the list
of users who have successfully authenticated. Without a cache, Apache makes
a Radius request for every object on every web page. This might be dozens of
requests for a single click of the mouse, which is obviously undesirable.
AuthXRadiusCacheTimeout 300
AuthXRadiusCache dbm "/tmp/auth_xradius_cache"

We b a n d F T P Ser vi ces

513

The AuthXRadiusCacheTimeout tells Apache how long to cache objects, in
seconds. The AuthXRadiusCache line tells Apache where to store the cache. In
this case, I’m using a dbm (hash database) file in /tmp. If you have untrusted
users logging into your web server on the command line, place your cache in
a directory that users cannot access, but on a dedicated server /tmp is adequately secure.
Now, tell your protected directory to require Radius authentication. You
could do this in a .htaccess file just as you do for usernames and passwords, but I
recommend placing the configuration directly into httpd.conf. Sites using
Radius authentication are generally managed by a system administrator, not
by users. Place your configuration directly into a Directory statement. For
example, I run Nagios on FreeBSD. Here’s a configuration for passwordprotecting the nagios directory with Radius:

AllowOverride None
...
AuthName "Nagios"
AuthBasicProvider "xradius"
AuthType basic
AuthXRadiusAddServer "radius.absolutefreebsd.com" "RadiusSecret"
AuthXRadiusTimeout 2
AuthXRadiusRetries 2
require valid-user


While this looks confusing at first glance, it’s not that bad if you go
through it slowly. The AllowOverride statement specifically disallows configuration of the directory via a .htaccess file. As we’re configuring the directory
within the web server itself, that’s perfectly fine.
The AuthName entry gives the text displayed in the user-visible password
prompt.
AuthBasicProvider tells Apache where to get its source of authentication
information. While the password file has been built into Apache for many
years, so Apache knows about it by default, when you use add-on authentication systems you must tell Apache.
The AuthType of basic indicates that you’re using HTTP basic
authentication.
To tell Apache where to find your Radius server and the shared secret,
use the AuthXRadiusAddServer value. Put both the server name and the shared
secret in quotes.
A Radius server should respond in less than a second. To tell Apache how
many seconds to wait for an answer before trying again, use AuthXRadiusTimeout.
To tell Apache how many times to retry, use AuthXRadiusRetries.
Finally, tell Apache that the user must provide a valid username and
password to get access.

514

C h ap te r 1 7

While this example is specific to Radius, in particular the XRadius module,
you can apply the same principles to any other authentication module. Read
the module documentation and configure Apache as required. While it won’t
look exactly like the Radius setup, chances are it will be very similar.
Groups and .htaccess
One useful feature of usernames and passwords is the ability to restrict access
by group. Perhaps you have a large website, where authenticated users get
access to the majority of the site content but only a few of those authenticated
users should have access to the administrative area. You can use groups to
accomplish this. Apache group files look much like /etc/group: a group name,
followed by a colon, followed by a comma-delimited list of users:
administrators: mwlucas, gedonner

Then, tell Apache about the group file and add a require-group statement
to the .htaccess file or directly to httpd.conf :
authgroupfile /usr/local/etc/apache22/users/webgroup
require-group administrators

This way, your administrators have a single password for both the semipublic and administrative areas of your site, but only the administrators can
access the administrative area. You can use this function to divide your website
in any way by creating more groups.

Including Other Configuration Files
One feature that makes Apache configuration easier to manage is the ability
to include other configuration files. Older httpd.conf files were over a thousand
lines long and included large amounts of text irrelevant to most users. These
features are now segmented off into separate configuration files. Near the end
of httpd.conf you’ll see entries like this:
...
# Fancy directory listings
#Include etc/apache22/extra/httpd-autoindex.conf
# Language settings
#Include etc/apache22/extra/httpd-languages.conf
# User home directories
#Include etc/apache22/extra/httpd-userdir.conf
...

By uncommenting the configuration entry, you enable the functionality
in the included file. For example, to enable fancy directory listings you would
uncomment the line pointing to that configuration file . Of course, you must
review the file to confirm that the functionality is configured as you wish.

We b a n d F T P Ser vi ces

515

The default Apache configuration file has two directories set aside for
included configurations: extra and Includes. The Includes directory is for your
use. Any file in the Includes directory with a name ending in .conf is sucked
into the global Apache configuration. We’ll use this feature when we create
virtual hosts later in this chapter. The files in extra come with Apache and
cover special Apache functions that are not required in every setting, but are
sufficiently popular to be integrated with Apache itself. These functions are:
MPM The multiprocessing module tells Apache how to handle worker
processes. All the settings in this file are defaults.
Multi-Language Error Messages Apache defaults to providing error
messages in English. If you need to support multiple languages, enable
this module. The browser and the server will negotiate a language for
the error messages.
Fancy Directory Listings The default Apache autoindex function looks
rather drab. The fancy directory module tells Apache to generate prettier directory indexes.
Language Settings You can give Apache a different native language and
tell it about different character sets.
User Home Directories Traditionally, a user’s home directory on a web
server was available as a web page via http:///~.
This module enables this functionality. You can see this in play at my
home page, http://www.blackhelicopters.org/~mwlucas.
Status Apache can generate a web page that tells you about Apache’s
status, configuration, and other real-time information.
Virtual hosts You can run multiple websites on a single web server
through virtual hosts. We’ll cover virtual hosts later in this chapter.
Manual Apache ships with the manual for the version you installed.
Enabling this makes the manual available on the website for easy
reference.
WebDAV DAV lets you create a shared file area where users can upload
and download documents. You must consider the security of your site and
read the WebDAV documentation before enabling this.
Defaults Apache includes many default settings set in the code itself.
If you must override these settings, load this default configuration and
make your changes here.
SSL SSL is important enough that we dedicate a section later in this
chapter specifically to HTTPS sites.
Each of these functions has a configuration file in extra. Take a look
there for more details.

516

C h ap te r 1 7

Virtual Hosting
Virtual hosting occurs when one web server handles multiple websites. The
server is configured to handle web requests for each of the hosted domains.
Many companies need a very small website, with just a few pages of information and perhaps a CGI script or two. This is an excellent application
for virtual hosting. I’ve run thousands of virtual domains on one FreeBSD
system without putting the system load up over 0.2. When each site pays
$19.95 a month to handle a couple dozen hits a day, you’re quickly looking
at real money on inexpensive hardware.
One common stumbling block to understanding virtual hosts is the belief
that the www in a URL is a magic incantation that points to a website. This is
incorrect. When you type a URL such as http://www.freebsd.org, you’re telling
your browser to go to a machine named www.freebsd.org and check its website.
The www started off as system administrators’ shorthand for “the server with
our world wide website on it” and has spread into popular usage. If you pay
attention, you’ll see websites on machines with many different names. With
virtual hosts, many hostnames point to one machine. The server needs to
differentiate between the requests for different domains and then answer
each request with the appropriate file. To enable virtual hosts, uncomment
the Include line for virtual hosts in httpd.conf and restart Apache.

Configuring Virtual Hosts
As one Apache server can handle thousands of virtual hosts, I suggest placing
the configuration for each server in its own file in the Includes directory. If you
end the filename in .conf, Apache will automatically include the configuration
on the next reload. Further, I recommend naming the file after the domain
it serves. You’ll end up with files like www.customer1.com.conf, www.customer2
.com.conf, and so on. Compared to sorting through a single monolithic configuration file, this makes site troubleshooting much easier.
Apache supports two different styles of virtual hosts: name-based and IPbased. Name-based virtual hosts assume that the client asks for the name of the
website. All browsers since Netscape 3 and IE 4 support this behavior. Namebased virtual hosts are almost always the proper choice. An IP-based virtual host
is attached to a single individual IP address; this is only necessary when a virtual
host needs a SSL certificate. You can use both on a single server.
Here’s a name-based virtual server configuration for absolutefreebsd.com.
It’s installed as /usr/local/etc/apache22/Includes/absolutefreebsd.com.conf.







ServerAdmin webmaster@absolutefreebsd.com
DocumentRoot /var/www/absolutefreebsd.com
ServerName absolutefreebsd.com
ServerAlias www.absolutefreebsd.com
ErrorLog /var/log/http/absolutefreebsd.com-error_log
CustomLog /var/log/http/absolutefreebsd.com-access_log combined


We b a n d F T P Ser vi ces

517

The first thing to note is the  and  tags.
Everything between these tags defines a single virtual host. The *:80 entry
tells Apache that a request for this website can come to port 80 on any IP
address on this host.
Just like our main server, a virtual host needs a ServerAdmin and
DocumentRoot , where Apache will find the documents that make up the
website. The web server also needs ErrorLog and CustomLog directives.
(You can have all of your virtual websites log to a single file, but then you’ll
have to sort them out later.)
Perhaps the most interesting thing here is the ServerName and ServerAlias
directives. The server name is absolutefreebsd.com. The ServerAlias directive
gives this site a second name, www.absolutefreebsd.com. Apache serves up the
same site for either name. If someone doesn’t type the www part of the site
name, you still want them to reach the page.
The IP-based virtual host is almost identical to the name-based host, with
the exception of the content of the  tags. Instead of using *:80 to
indicate that you’re listening for requests for this host on all addresses on the
machine, you use a single IP address:










ServerAdmin webmaster@AbsoluteFreeBSD.com
...

This virtual host attaches to the IP address 192.168.1.5. This is only useful
for a site using HTTPS, which must have its own unique IP address. See
“HTTPS Websites” on page 520.
This virtual host entry should contain everything needed for this website.
For example, the Nagios server for which I configured Radius authentication
earlier in this chapter is actually a virtual host. The Radius configuration
should go into the virtual host configuration file rather than httpd.conf itself.

T H E D E F AU L T H O S T
If you’re using virtual hosts, the first host you set up will be the default website. Make
sure that the DocumentRoot and error logs are the same as you set for the default site
in httpd.conf. I also recommend removing the sample name-based virtual host from
the virtual host configuration in extra/httpd-vhosts.conf.

Tuning Virtual Hosts
Once you have your minimal virtual host working, you can add some more
touches. Here we’ll discuss various options that work with both IP-based and
name-based virtual hosts.

518

C h ap te r 1 7

Port Numbers
You can serve different sites on different TCP/IP ports. You’ve probably seen
this before—a hostname in a URL may end in a colon followed by a number.
If Apache is listening on ports 80 and 8080, for example, you could have a
different virtual host on each port, as long as you add the port number after
the IP address in the VirtualHost directive.
Here’s an example configuration that creates two different sites on two
different ports. If I was to use these in production, I’d fill them out with
logging statements and administrator information, but this example is intentionally simple:

DocumentRoot /var/www/www.absolutefreebsd.com
ServerName www.absolutefreebsd.com


DocumentRoot /var/www/data.absolutefreebsd.com
ServerName data.absolutefreebsd.com


Of course, you could point both of these sites to the same directory to
serve the same content, or you could have them both listening on port 80. It
doesn’t really matter, so long as it solves your problem.
Options and AllowOverride
By default, virtual hosts inherit the Options and AllowOverride settings of the
Apache root directory. The default permissions are very restrictive, allowing
no access from anywhere. You can use the Options and AllowOverride statements
with a virtual host. Any option that is valid in httpd.conf is valid on a virtual host.
This lets you set server configuration on a site-by-site basis; for example, you
could allow server-side includes on one virtual host but not on others. The
following virtual host has Options settings that override the server’s defaults
and special privileges for the DocumentRoot directory:

DocumentRoot /var/www/absolutefreebsd.com
ServerName absolutefreebsd.com
Options IncludesNOEXEC


AllowOverride AuthConfig


This makes your virtual host almost as flexible as a dedicated server—
until your customers want their own custom Apache modules installed, of
course. That’s when you sell them a jailed server and let them install their
own modules.

We b a n d F T P Ser vi ces

519

HTTPS Websites
Many online shopping malls and password-protected areas use what they
call secure websites. What they mean by this is that they use SSL to encrypt traffic
between the server and the client. While these sites aren’t as secure as the
name implies, SSL provides a vital layer of protection. SSL functionality is
integrated with Apache via the extra/httpd-ssl.conf file. Uncomment the entry
for this file in httpd.conf to enable SSL.
All SSL web servers need a secure certificate. We discussed generating a
certificate request and creating a self-signed certificate in Chapter 9. For your
private use, a self-signed certificate is sufficient, but anything that faces the
public really needs a certificate from an accredited CA. If you attempt to use
a self-signed certificate on a customer-facing application, the client’s web
browser will spew scary looking warnings about your lack of security.
The completed certificate has two parts: a certificate file (hostname.crt)
and a host key (hostname.key). Place these files in a directory outside of the
web content, so that nobody can download them from the web server itself.
Be sure to make these files readable only by the web server unprivileged
user and not by regular users:
#
#
#
#

chmod
chmod
chown
chown

600 hostname.crt
600 hostname.key
www:www hostname.crt
www:www hostname.key

Now that you have the certificate on the system, tell Apache about it.
In the past, SSL configuration used to bring tears to the eyes of experienced
system administrators; today, it only requires four lines within your virtual
host configuration:

ServerName secure.absolutefreebsd.com
SSLEngine on
SSLCertificateFile etc/apache22/ssl.crt/hostname.crt
SSLCertificateKeyFile etc/apache22/ssl.key/hostname.key
...

First, note that the we’re using an IP-based virtual host that’s listening on
port 443. Standard HTTPS sites run on TCP port 443.
The ServerName is extremely important for HTTPS websites. The ServerName
should exactly match the reverse DNS of this IP address and the name on the
SSL certificate. If the three names do not exactly match, the user might see
security warnings. ServerAlias is not useful for HTTPS websites.
We then turn the SSL engine on for the site and list the full path to the
certificate and the host key. Use the full hostname for the files containing
the certificate and key.

520

C h ap te r 1 7

S E C U RE W EB S ER V ER S AN D S S L
I recommend avoiding the term “secure website” when you mean SSL. Encrypting
the network traffic between the client and the server only defends against one
particular type of network attack. Intruders can still penetrate either the server or the
client. A secure web server requires regular maintenance, good web design, and an
educated system administrator who is left alone long enough to do his job. While
that last one looks like a significant barrier, it pales compared to one of the
requirements for a secure web client: an educated user!

Controlling Apache
Apache is a complicated program that you can manage in several different
ways. While apachectl(8) works quite well, I recommend using FreeBSD’s
integrated Apache startup script. This script runs apachectl(8) with any
special settings needed for your environment and ensures that the next time
your system boots, your Apache server works as it did before shutting down.
Apache is a little different from most other programs, however, so you have
several different arguments you can use other than the plain old start and
stop. The common options are start, stop, restart , graceful , gracefulstop,
and configtest.
/usr/local/etc/rc.d/apache22 start activates Apache with all modules as
configured. There is no longer any special command for starting SSL web
servers. If the server configuration is invalid, it prints out the problem with
the config and does not start the service.
/usr/local/etc/rc.d/apache22 stop shuts off Apache immediately, terminating all open connections without completing the requests.
/usr/local/etc/rc.d/apache22 restart checks the server configuration. If it
finds a configuration problem, the script prints out the problem and does
nothing. If the configuration is valid, it stops and immediately starts Apache.
/usr/local/etc/rc.d/apache22 graceful performs a graceful restart. Open
connections are allowed to complete before being shut down. While this might
seem unnecessary, it’s quite important when you serve large files or have a
group of web servers behind a load balancer. Like restart, this command
checks the configuration before shutting down the service and takes no action
if there is a configuration problem.
/usr/local/etc/rc.d/apache22 gracefulstop shuts Apache down without
abruptly terminating open connections. Open connections are permitted to
remain open and complete their requests before the process shuts down.
/usr/local/etc/rc.d/apache22 configtest checks the Apache configuration
and prints out any problems it sees. This is the function used by the restart and
graceful commands to validate the configuration before shutting down the
current process.
Now that you can manage Apache, let’s see how you can get files to and
from the server itself.

We b a n d F T P Ser vi ces

521

File Transfer
A web server isn’t any good without web content. While I design my websites
on FreeBSD, my websites are hideous.3 Web design professionals usually
design their sites on a workstation and upload them to the server. The
standard methods for file transfer are FTP and sftp/scp.
FTP, the File Transfer Protocol, is the classic method of moving files from
one computer to another over the Internet. Most of your users will prefer to
use FTP. Like many other protocols, FTP has not aged well. FTP has problems
in many modern environments. Over the years, fixes for these issues have
been bolted onto the specification, creating a Frankensteinian horror of a
protocol that lurches after network administrators when they turn away even
for a moment. While FreeBSD makes handling FTP as easy as possible, you’ll
still need to do some work to keep it chugging along.

FTP Security
FTP transmits passwords and usernames in clear text, which means anyone
with a packet sniffer can capture this information. Nobody except the network
administrator should have a packet sniffer on your local network, but if your
users are on remote networks or behind cable modems, their passwords are
vulnerable.
As the system administrator, however, you should never, never, never
transmit your password over the network in clear text! Instead, use scp(1)
and sftp(1) to move files between machines.

The FTP Client
FTP is a fairly complex protocol which, unlike POP3 or SMTP, cannot be
easily tested via telnet(1). You must use an FTP client. To connect to a host,
just type ftp and the hostname:
# ftp sardines
Connected to sardines.blackhelicopters.org.
220 sardines.blackhelicopters.org FTP server (Version 6.00LS) ready.
Name (sardines:mwlucas):

The client sends your local username as a default, but you can enter a
different username if necessary. You’ll then be asked for your password:
331 Password required for mwlucas.
Password:
230 User mwlucas logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp>

3
Mind you, my websites are not hideous because I design them on FreeBSD. They are hideous
because my personal strengths of cynicism, sarcasm, and bitterness are completely inapplicable
to making web pages look nice.

522

C h ap te r 1 7

If everything goes as planned, you’re now logged into the remote server
with a shell almost like a command shell. You cannot execute commands, but
you can move around and view files using standard Unix commands such as
ls and cd.
To download a file with FTP, use the get command and the filename:
ftp> get .cshrc
local: .cshrc remote: .cshrc
229 Entering Extended Passive Mode (|||50451|)
150 Opening BINARY mode data connection for '.cshrc' (614 bytes).
100% |*************************************| 614
272.17 KB/s
226 Transfer complete.
614 bytes received in 00:00 (193.48 KB/s)
ftp>

00:00 ETA

As you watch, your FTP client opens a connection to transfer the file.
A line of asterisks crawls across the screen as the file moves, and an ETA line
updates with the length of time remaining in the download. When the file
transfer finishes, you’ll see a notification, the size of the file you transferred,
and an FTP prompt.
Similarly, use the put command to copy files from your local system to
the FTP server. The output looks almost the same as that for get, so I won’t
repeat it here.
Use the mget and mput commands to move multiple files at once. For
example, if you want to download all the files that end in .html, enter mget
*.html. Annoyingly, the server will prompt you to confirm every single file.
You can toggle the verification on and off with the prompt command.
Finally, you can view text files over FTP. The less command displays the
contents of a remote file, one page at a time, exactly like less(1). While it’s
rude to view a file and then download it, using less to view README and index
documents is fine.

Binary and ASCII Transfers
The difference between binary and ASCII transfers is a big source of user
confusion, caused by the different handling of the return and newline
characters. DOS and Unix systems have long disagreed on how to mark the
end of a line, as you might have noticed when moving files between the two
systems. An Internet search will uncover many documents describing the
issue in painful detail and many more articles denouncing one side or the
other for being Just Plain Wrong. All you need to know is how to live with the
problem and get on with your life.
While you can transfer both binary and text files in binary mode, you
cannot transfer binary files in ASCII mode. Unix-like systems default to using
binary transfers, while Windows-based systems default to using ASCII. You
can tell an FTP server to use binary transfers with the bin command and
ASCII with the a command. Binary works for everything, so use binary mode
for all FTP transfers and you’ll never have this problem.

We b a n d F T P Ser vi ces

523

The FTP Server
Now that you know how to use the FreeBSD FTP client, let’s look at how to
provide FTP services. Your first task is to decide if you’re going to run your
FTP daemon, ftpd(8), from inetd(8) or in standalone mode.
FreeBSD defaults to running the FTP daemon from inetd(8). Most
systems don’t get that many FTP requests, and inetd can easily handle the
few requests that do arrive. If you won’t be supporting more than a couple
hundred of simultaneous FTP sessions, inetd(8) works well. Just uncomment
the FTP line in /etc/inetd.conf (see Chapter 15) and restart inetd. Any changes
to the ftpd(8) command line can be made in inetd.conf.
However, if your FTP server handles hundreds or thousands of simultaneous connections, running FTP from inetd just adds to the system overhead.
You’ll want to set up FTP in standalone mode, where it is permanently
listening to the network and handling requests itself. Confirm that ftpd(8) is
not running out of inetd and then set ftpd_enable="YES" in /etc/rc.conf. You
can either start ftpd(8) with /etc/rc.d/ftpd or reboot.
You can adjust ftpd(8) in many different ways through command-line
flags. Add these flags to the ftpd(8) line in /etc/inetd.conf or to ftpd_flags in
/etc/rc.conf. Now, let’s look at the features that FreeBSD’s ftpd(8) offers.
Logging ftpd(8) Usage
ftpd(8) has two levels of logging. If you specify -l once (this is the default in
/etc/inetd.conf), ftpd(8) will log all successful and unsuccessful login attempts.
If you specify it twice, ftpd(8) logs all FTP activities: downloads, uploads,
directory creation and destruction, and so on.
Read-Only Mode
You might want to prevent FTP users from uploading files, replacing files, or
changing the server’s filesystem in any way. This is good for a server that only
provides downloads, such as a mirror site. Use the -r flag to set this behavior.
Write-Only Mode
Perhaps you want a server where users can only upload files, not download
them. In this case, use the -o flag.
Timeout
By default, if a user leaves an FTP session idle for 15 minutes (900 seconds),
it is disconnected. You can set a new idle timeout with the -t flag, specifying
the number of seconds of permitted idle time.

FTP User Control
Two common concerns with running an FTP server are that it can allow users
to download arbitrary system files and that FTP passwords can be sniffed. You
don’t want your server compromised because some dingbat user used his

524

C h ap te r 1 7

password at the local cybercafe, his login was stolen, and the intruder downloaded key system files! The best way to control users is by choosing who may
log in and which directories those legitimate users may access.
Chrooting Users
You can lock FTP users into their own home directories with chroot(8). To
the user, their home directory will look like the top of the filesystem. They
won’t be able to leave their home directory or access arbitrary system files.
This is much like a small jail. Chrooting is useful for web servers that have
multiple clients on one machine—that is, web servers with many virtual hosts.
After all, customers only need to see their own directories, not anyone else’s.
To chroot a user, add her username to the file /etc/ftpchroot. Put each
user on a new line. Every time a user logs in via FTP, the user’s account is
checked against /etc/ftpchroot. If the username appears there, the user is
locked into her home directory. A chrooted user has complete control of her
home directory (unless you set the permissions otherwise) and can create as
many subdirectories and store as many files as the disk space allows; the user
just can’t leave her home directory and explore the system.
You can also list groups in /etc/ftpchroot, chrooting all users who are in
that group. Preface group names with an @ sign (for example, @customers).
Suppose we have a system with two web designers, Gordon and Chris.
These admins should only upload files into their home directories. Similarly,
we have a group of users who maintain their own websites. These clients are
all in the group webclients. To chroot all of these users, set up /etc/ftpchroot
like this:
gordon
chris
@webclients

All of these users are now restricted to their home directories.
Disallowing Users’ FTP Access
The file /etc/ftpusers is deceptively named. Instead of a list of users permitted to
use FTP, it contains a list of users who are not allowed to use FTP. FreeBSD’s
default /etc/ftpusers contains a variety of system accounts, such as root and
nobody. No system administrator should log in as root!
You can list groups in this file by prefacing them with the @ symbol.
I habitually ban members of the wheel group from using FTP. People who
have access to the root password should never transmit their own password in
clear text. Anyone whom I would trust with root access knows how to use
SSH, scp(1), and sftp(1)!

FTP Server Messages
The FTP server provides two different connection messages: /etc/ftpwelcome
and /etc/ftpmotd.

We b a n d F T P Ser vi ces

525

When a client first makes an FTP connection, ftpd(8) displays the contents
of /etc/ftpwelcome. You can put here the terms of use, legal warnings, capacity
statements, obscenities and threats, or whatever else you like. Users see this
message before they even get a login prompt. This is an excellent place to
put an “Unauthorized use not permitted” message. While not as much fun as
an “Unauthorized users will be prosecuted to the full extent of an unfriendly
Rottwieler and my 12-gauge shotgun” message, the former is actually admissible in court if an intruder hijacks your server.
Once the user has logged in, ftpd(8) displays the contents of /etc/ftpmotd.
This is a good place to remind your users of the system terms of service.

Setting Up Anonymous FTP Servers
Anonymous FTP sites are a popular way to provide files and documents to the
Internet at large. Anonymous FTP sites are frequently broken into, however.
While FreeBSD’s ftpd(8) is quite robust and secure, you should still take
basic precautions and configure your server properly to avoid problems.
Here are a few recommendations:
Do not share anonymous and non-anonymous FTP services on a single
server.
Set ftpd to run read-only by starting it with the -r flag.
Use ftpd’s -S flag to log all anonymous FTP activity to the file /var/log/ftpd.
This file must exist before ftpd(8) starts using it, so run touch /var/log/ftpd
first.
Create a nonprivileged user ftp for ftpd(8). This user’s home directory
will be the root of the anonymous FTP directory, and all files you want
the world to see must be placed in this directory.
Create the directory /home/ftp/pub for the traditional pub folder in an
FTP server and change the permissions on the ftp user’s home directory
to prevent that user from changing the home directory. This will force
all uploaded files (if permitted) to go into pub.
Anonymous FTP Pitfalls
Allowing just anyone to upload to your server might seem friendly. You
might have bandwidth and disk to spare, and you might desire to provide a
public service. In an ideal world (such as the Internet of the ’80s and early
’90s), this would be lovely.
If you allow anyone to store data on your system, however, people can
use your FTP server to store illegal software, child porn, or terrorist data. To
make it harder for you to find, they can create hidden directories or disguise
the data. Even if you go looking through all the crud people upload, you’ll
probably ignore files that look innocuous. It’s not your fault that the file
labeled CookieRecipies.txt is actually a MPEG of the Gerbil Liberation Front
training for its secret mission to cram the President of the United States into
a giant wheel and make him run for his life. But you’ll have a hard time
explaining that to Homeland Security.
526

C h ap te r 1 7

Chrooting sftp(1) and scp(1)
Finally, a quick word on sftp(1) and scp(1). These are file transfer services
that run over SSH (see Chapter 15), providing authentication, integrity, and
nonrepudiation for data transfers. sftp is a secure replacement for FTP, while
scp replaces rcp(1). Users who transfer files to a server can use these protocols
instead of FTP.
One problem that system administrators face is that of chrooting users
of these services. While it’s nice that you can provide a secure encrypted
connection for users to transfer their data, you still don’t want them to
download random files from all over your server! For these users, I recommend scponly (/usr/ports/shells/scponly). A user whose shell is set to
/usr/local/bin/scponly will only have sftp and scp access to your server.
Now that you can serve websites and FTP data, let’s spend some time
with FreeBSD’s disk layer. Say hello to GEOM!

We b a n d F T P Ser vi ces

527

18
DISK TRICKS WITH GEOM

FreeBSD has an incredibly flexible disk
management system called GEOM. GEOM
is an infrastructure system that allows kernel
developers to easily program modules, called
GEOM classes, for different types of disk functionality.
FreeBSD uses GEOM to support disk encryption,
journaling, several different types of software RAID, and exporting disk
devices across the network. The GEOM tools provide flexibility, redundancy,
and ease of implementation for both developers and system administrators.
We’ll take you through FreeBSD’s disk partitioning tools and then cover
several of the popular GEOM classes.
Throughout this chapter, we’ll experiment with disk devices. Any time
you touch a disk’s layout or format, you risk the data on that disk. While it’s
entirely possible to manipulate unused disk space without affecting data on
the rest of the disk, learning how to do that is still risky. Back up your data.
Better still, find an unused disk and learn on it! (On the other hand, learning
on the disk that contains your most precious data will nicely focus your mind.)

We specifically discuss the disk formats used with i386 and amd64
systems. Other platforms have slightly different disk management systems.
Sparc uses a disklabel at the front of the disk, but no slice table. The ia64
platform uses slices based on GPT, while partitioning an ARM disk is by far
the easiest part of making FreeBSD run on that hardware.

GEOM Essentials
GEOM is a generic framework that allows almost arbitrary layering of storage
devices. Different GEOM modules transform storage media in various ways.
Additionally, these modules are stackable; the output of one can be used as
the input for another. You want your hard drives mirrored? Sure. How about
mirroring across the network? No problem. What if you want the drive
encrypted, as it’s mirrored, across the network, in stripes? Wake me up when
you want something difficult, buddy. Mind you, FreeBSD doesn’t promise
massive performance through too many GEOM layers, but it can be done.
GEOM separates storage devices into consumers and providers. A consumer
lies beneath the module, while a provider offers services to the next layer up.
For example, suppose you’re using a GEOM class to mirror the hard drives
/dev/da0 and /dev/da1 into one virtual disk, /dev/mirror/mirror0. The disk
devices are consumed by the mirrored disk, while the combined device node
/dev/mirror/mirror0 is a provider that offers services to the next layer.
FreeBSD sees all providers identically. A physical hard disk is just another
provider. Maybe you’re putting a filesystem right on the provider, or perhaps
/dev/mirror/mirror0 is a consumer of the GEOM disk encryption module.
In any case, to use any of FreeBSD’s advanced disk features, you must know
a little more about hard disk management.

Disk Drives 102
Historically, data on disk could be mapped to a location on the hard drive.
This location could be expressed in terms of cylinders, tracks, and sectors.
Remember, each hard disk is made of a stack of disk platters. Each disk
platter has a series of circular rings, or tracks, arranged much like the growth
rings in a tree. These tracks hold data as a string of zeroes and ones. A head
moves over a particular track at a certain distance from the center of the disk
and reads this data as the platter spins beneath it. When you request data
from a particular track, the head shifts its position and lets that track rotate
past beneath it, catching the data much as a parent might snatch a child off
a spinning merry-go-round.
If you stack the tracks on top of one another from all the platters, you
have a cylinder. The innermost track of each platter forms cylinder 0. The
next-innermost is cylinder 1. The 1,603rd track of each platter is cylinder
1,602. Many operating systems expect to find that disk slices encompass
complete cylinders and get quite irate if they don’t.

530

C h ap te r 1 8

Each track is broken up into segments called sectors, or blocks, which can
each hold a certain amount of data (512 bytes for many years now). Each
sector within a track has a unique number, starting with 1.
So, sectors combine into tracks, which are stacked into cylinders, which
combine to describe the disk’s geometry. This all seems straightforward enough,
and would be if you could rely on it.
Over the years, both hard drive manufacturers and operating systems
have set and broken limits. This applies to all aspects of machine design—
some of you probably remember when 640KB was the maximum RAM that
could be put in a PC. A few of you even remember when 640KB was an
unimaginably vast amount of RAM and you had no idea what to do with it.1
Hard drive manufacturers avoided these limits by tricking the system BIOS
and/or the operating system. If you’re a hard drive manufacturer making a
hard drive with 126 sectors per track, and the most popular operating system
can only accept 63 sectors per track, you have a problem. The easy solution is
to teach your hard drive to lie. If you claim you have half as many sectors per
track but twice as many platters, the numbers still add up, and all operating
systems can still identify unique blocks. A lie makes the problem go away.
You’ll encounter several different types of these translations if you read
about disk management history. A leading FreeBSD developer summed
this all up nicely with, “It’s all just lame x86 BIOS crap, though.”
By the time hard drive information reaches the user, it has quite possibly
been through one or more such translations. If you have hardware RAID,
either on a local RAID card or from a SAN, geometry information is obviously
bogus. Today, the important thing to understand is how this fits together in a
logical sense, not necessarily in a physical sense. Accept the lies your hard drive
tells you, even when they’re obvious.
Many operating systems try to optimize performance based on cylinder
information. Geometry translations make these optimization schemes not
work as intended.

Slicing Disks
Remember from Chapter 8 that FreeBSD divides disks in two distinct ways:
slices and partitions. A slice is a BIOS partition, the only sort of partition
recognized by older Microsoft Windows systems. A FreeBSD partition is a
subdivision of a slice. FreeBSD lets you configure slices and partitions separately. A single disk (on i386 and amd64) can have up to four slices, and
each slice can have up to eight partitions. FreeBSD filesystems are on
partitions inside slices.
Before the first slice, you’ll find the Master Boot Record (MBR) at cylinder 0,
head 0, sector 1. The MBR contains the slice table as well as the system’s
initial bootstrapping code. The MBR is strictly limited to 512 bytes, which
seemed like a lot of space at the time it was defined. A booting computer

1
And yes, I know, I know, some of you weren’t even born then! Please review the footnote on
page 4.

D is k Tr ick s wit h GEO M

531

reads the MBR of the boot drive to identify disk slices, finds the slice with the
operating system on it, and activates the bootstrap code on that slice. You can
use any portion of the disk after the MBR for FreeBSD.

Viewing the Slice Table with fdisk(8)
To read and edit the slice table in the MBR, use fdisk(8). fdisk always needs
one argument, the name of the disk you want to examine. Here I run fdisk
on one of my test hard drives:
# fdisk /dev/da2
******* Working on device /dev/da2 *******
parameters extracted from in-core disklabel are:
cylinders=1116 heads=255 sectors/track=63 (16065 blks/cyl)
Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=1116 heads=255 sectors/track=63 (16065 blks/cyl)

 Media sector size is 512





Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
start 63, size 17928477 (8754 Meg), flag 80 (active)
beg: cyl 0/ head 1/ sector 1;
end: cyl 1023/ head 254/ sector 63
The data for partition 2 is:

The data for partition 3 is:

The data for partition 4 is:


This particular SCSI disk claims to have 1,116 cylinders, 255 heads, and
63 sectors per track . Remember, a head is a mechanical device that sits
above the platter. 255 heads would be an awful lot to cram within a single hard
drive! While I prefer that systems that lie to me pay me the courtesy of making
the lies believable, we have no choice but to accept the hard drive’s answers
and treat them as fact. No matter how many heads, cylinders, and sectors
this hard drive really has, I’m confident it has 17,928,540 sectors (1,116 ×
255 × 63 = 17,928,540).
Back when hard drives first implemented translations, you might gather
conflicting drive geometry information from different sources. fdisk(8)
provides all the information it can gather about the drive in the hope
that the system administrator can sort out enough truth to use the drive.
This is rarely necessary on modern drives; these days, the information in
these first two sections should always be identical.
While all hard drives should have a sector size of 512 bytes , fdisk(8) is
kind enough to verify this.



532

C h ap te r 1 8

We then see the information for each of the four possible slices on the
disk. Our first partition is labeled as a FreeBSD partition. Each operating
system has a unique slice identifier, and the 165 marks this as a FreeBSD slice.
We also see which of the millions of sectors are assigned to this slice.
Slice 1 begins at sector 63—remember, the MBR takes up the first head on
the first cylinder, or sectors 1 through 62. Slice 1 fills 17,928,477 sectors, so it
ends at sector number 17,928,539 (17,928,477 + 62 = 17,928,539). The disk
has 17,928,540 sectors, but don’t forget the MBR. This disk is full.
One annoying thing is that the MBR only has enough bits in its slice
table to indicate up to 1,024 cylinders per disk. (This limitation created the
504MB limit on hard drives.) This disk has far more cylinders than that!
That means you cannot use the cylinder counts in each individual partition
to measure the disk. The only meaningful number is the number of sectors.
Finally, slices 2, 3, and 4 are unused .









Backing Up the Slice Table
Before changing the slice table on a disk, back up the old slice table. This lets
you restore the disk to its previous configuration, should you make an
error. To back up the slice table, use fdisk(8)’s -p flag:
# fdisk -p /dev/da0 > da0.slice.backup

To restore a disk’s slice table from the backup, use the -f flag and the
name of the backup file:
# fdisk -f da0.slice.backup /dev/da0

If you haven’t touched the hard drive since reslicing it, you might even
get your data back. Most of the time, however, restoring the slice table won’t
restore the data in those slices, merely the disk configuration. You must restore
the data from backup.

Changing the Slice Table
fdisk(8) not only lets you view the table, you can edit it both interactively and
via scripts as well. The simplest way to initialize a new FreeBSD disk is to tell
fdisk(8) to mark it with a single large slice dedicated to FreeBSD. It’s common
enough that fdisk’s -I flag is dedicated to doing exactly that. The -B flag tells
FreeBSD to completely reinitialize the MBR, which is always a good idea when
starting over. Here, we reinitialize the SCSI disk /dev/da17 to have a single
FreeBSD partition:
# fdisk -BI /dev/da17

You could also write your custom slice table in the same format that
fdisk(8) uses to back up the slice table, and write that slice table to the disk
just like restoring a backup. This is generally inadvisable for most users.
D is k Tr ick s wit h GEO M

533

If you use a backup file, fdisk(8) assumes that you know what you’re doing—
even when you clearly don’t. fdisk(8) includes an interactive mode that
automatically detects a large number of subtle error conditions and offers
alternatives that won’t mess up your hard drive. I suggest always using interactive mode to edit slice tables. The -u flag enters fdisk(8)’s interactive mode.
Remember, we covered using sysinstall in Chapter 8 to reslice, repartition,
and reformat hard drives. Use fdisk(8) if sysinstall won’t work for some reason
or if you want more detailed control over the process.

DISKS AND MATH
When working with fdisk(8) and disklabel(8), do as much math as possible
beforehand on a piece of scratch paper. If you have to do more math partway
through the process, do that on scratch paper as well. Disk management uses large
numbers, and you should spend your valuable brain power on ensuring you entered
them correctly rather than on manipulating them. While simple addition, subtraction,
and division might seem easy, most disk configuration errors are caused by errors in
third grade math. By writing out all the numbers beforehand, you will save yourself
the embarrassment of learning that you can’t do third-grade math.

Let’s use fdisk(8) to split our sample hard drive into two roughly equal
slices. We know that this hard drive has 17,928,477 sectors. This means we
want each slice to be about 8,964,238 sectors.
# fdisk -u /dev/da17
******* Working on device /dev/da17 *******

fdisk prints the drive geometry information, then asks if you want to
change the disk geometry. Changing the geometry is almost always a bad
idea unless you have a thorough, almost intimate understanding of your
particular disk, so leave it alone:
Do you want to change our idea of what BIOS thinks ? [n] n

You’ll then get a printout of the first slice’s configuration and will be
asked if you want to change it:
The data for partition 1 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
start 63, size 17928477 (8754 Meg), flag 80 (active)
beg: cyl 0/ head 1/ sector 1;
end: cyl 1023/ head 254/ sector 63
Do you want to change it? [n] y

534

C h ap te r 1 8

Today, slice 1 uses up all of the sectors on the disk. We want to reduce
the number of sectors in slice 1 and assign the remainder to slice 2. Slice 1
begins in sector 63 and uses about 8,964,238 sectors. We just have to tell
fdisk(8) that. Hit ENTER to take the defaults:





Supply a decimal value for "sysid (165=FreeBSD)" [165]
Supply a decimal value for "start" [63]
Supply a decimal value for "size" [17928477] 8964301
fdisk: WARNING: partition does not end on a cylinder boundary
fdisk: WARNING: this may confuse the BIOS or some operating systems
Correct this automatically? [n] y
fdisk: WARNING: adjusting size of partition to 8964207
Explicitly specify beg/end address ? [n]
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
start 63, size 8964207 (4377 Meg), flag 80 (active)
beg: cyl 0/ head 1/ sector 1;
end: cyl 557/ head 254/ sector 63
Are we happy with this entry? [n] y

We let the slice begin right where FreeBSD thinks it should, but change
the number of sectors in the slice . fdisk(8) does a bit of calculation and
points out that this slice doesn’t end evenly on a cylinder . The odds that
your calculated slice size would end on a cylinder boundary are close to
negligible. Slices ending mid-cylinder can be a problem, especially if you have
a multiboot machine. Always let fdisk(8) correct this problem. fdisk(8) shows
the size in sectors of your new slice and displays the new slice entry
before asking you to confirm the change and proceeding to the next slice:







The data for partition 2 is:

Do you want to change it? [n] y
Supply a decimal value for "sysid (165=FreeBSD)" [0] 165
Supply a decimal value for "start" [0] 8964270
Supply a decimal value for "size" [0] 8964207

This slice has never been used before, so it has no defaults. We
must enter the slice type, as well as the starting sector. Slice 1 used
8,964,207 sectors, and the MBR used 62, so we’ve used up 8,964,269 sectors
(8,964,207 + 62 = 8,964,269). Our slice can start at sector 8,964,270. The
disk has 17,928,477 sectors in total, leaving us 8,964,207 sectors for this slice
(17,928,477 – 8,964,269 = 8,964,207). Let’s see what fdisk(8) thinks of these
numbers:
fdisk: WARNING: partition does not start on a head boundary
fdisk: WARNING: partition does not end on a cylinder boundary
fdisk: WARNING: this may confuse the BIOS or some operating systems
Correct this automatically? [n] y

fdisk(8) doesn’t think much of these numbers at all, but it knows what
to do. Let fdisk(8) make its corrections.

D is k Tr ick s wit h GEO M

535

fdisk: WARNING: adjusting start offset of partition to 8964333
Explicitly specify beg/end address ? [n]
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
start 8964333, size 8964207 (4377 Meg), flag 0
beg: cyl 558/ head 1/ sector 1;
end: cyl 91/ head 254/ sector 63
Are we happy with this entry? [n] y

Again, don’t explicitly specify the beginning or end address. fdisk(8) has
already done the hard work for you, so let it stand.
You’ll then be prompted to change the other two slices, if you desire. You
have no space left on the disk, so you really can’t use those slices right now:
Partition 1 is marked active
Do you want to change the active partition? [n]

Remember, the active partition is the slice with the root filesystem on it.
The boot loader hands control of the booting system to the slice you indicate.
If this is not a boot drive, the active partition is irrelevant.
We haven't changed the partition table yet. This is your last chance.
...
Information from DOS bootblock is:
1: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
start 63, size 8964207 (4377 Meg), flag 80 (active)
beg: cyl 0/ head 1/ sector 1;
end: cyl 557/ head 254/ sector 63
2: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
start 8964333, size 8964207 (4377 Meg), flag 0
beg: cyl 558/ head 1/ sector 1;
end: cyl 91/ head 254/ sector 63
3: 
4: 
Should we write new partition table? [n] y

This really is your last chance to salvage any data on that disk. Review the
slice table and, if the numbers look sensible, enter Y to write your new slice
table to the hard drive.
That’s everything you need to do to get a slice table on a disk. Now let’s
make some partitions in those slices.

Partitioning Slices
The i386-style slice table is not sufficient to meet FreeBSD’s partitioning
needs. A slice table only supports four disk divisions, while FreeBSD needs a
root partition, swap space, /usr, /var, and /tmp. The system administrator might
need his own collection of partitions as well. A disklabel is a special data block
at the beginning of a slice that indicates partitions’ positions within a slice.
Different platforms have different disklabel formats and requirements—i386
and amd64 use bsdlabels, sparc64 uses sunlabels, and so on. On all platforms,
536

C h ap te r 1 8

disklabel(8) provides native disklabel management. If you’re working on an
unfamiliar hardware architecture, be absolutely certain to read disklabel(8)
before trying to edit the disk!

Reading Disklabels
To view a slice’s disklabel, run disklabel(8) with the device name of a slice as
an argument. For example, slice 1 on disk ad0 is /dev/ad0s1:
# disklabel /dev/ad0s1
# /dev/ad0s1:
8 partitions:
#
size offset
a: 1048576
0
b: 2097152 1048576
c: 39179889
0
d: 10485760 3145728
e: 2097152 13631488
f: 23451249 15728640





fstype [fsize bsize bps/cpg]
4.2BSD
swap
unused
4.2BSD
4.2BSD
4.2BSD

2048

16384

8

0
2048
2048
2048

0
16384
16384
16384

28552
28552
28552

# "raw" part, don't edit



The disklabel shows that the disk supports up to eight partitions ,
although only six are configured. Partitions are named by letters a through h.
While the disk supports “up to eight” partitions, that doesn’t mean that
you’ll actually get eight usable partitions from your new disk. Partition a
is traditionally the root partition. While you could use this for normal data,
I prefer not to risk confusing myself. The b partition is traditionally swap
space. While FreeBSD only swaps on partitions labeled as swap in /etc/fstab,
you can bet having a b partition used for data would confuse me when I’m
troubleshooting at 3 AM. Finally, the c partition is the label for the whole
slice. While FreeBSD no longer needs the c partition, that doesn’t mean that
all of the add-on tools I might use would understand a c partition with data
on it! It’s been around so long that I would not dream of using it for data.
This means that you can have up to six data partitions on your root disk (d to
h, plus possibly a).
Each partition has a size in sectors and an offset from the beginning
of the slice. The offset is how far into the slice the partition begins, in sectors.
Remember, sector counts start at 0. Partition c covers the entire disk, with a
size of 39,179,889 sectors and an offset of 0. In this example, partition a has
a size of 1,048,576 and an offset of 0. The root partition is the first on the slice.
Partition b begins 1,048,576 sectors into the slice, or right after the end of
slice a, and has a size of 2,097,152. If partition a takes up 1,048,576 sectors,
and partition b takes up 2,097,152 sectors, then the next partition must begin
at sector 3,145,728 (1,048,576 + 2,097,152 = 3,145,728). Check out the offset
for partition d. This is all just basic addition.
Each partition needs a filesystem type . The only types of filesystems
normally used within a FreeBSD disklabel are 4.2BSD (any FreeBSD filesystem
such as UFS or UFS2), swap (for swap space), or unused (for the c partition or
empty space). FreeBSD supports other filesystem types, but it’s very rare to





D is k Tr ick s wit h GEO M

537

plop a FAT or a CD filesystem in the middle of your FreeBSD hard disk, and
odds are you’ll never see a disk with a Sixth Edition Unix partition in the
middle of it.
The next two columns give the size of fragments and blocks for UFS
and UFS2 filesystems. FreeBSD defaults to using 16KB blocks and 2KB fragments. We talked about UFS2 design back in Chapter 8. The defaults are a
good average on most modern systems. If you have a special-purpose filesystem
that will only hold files of an unusual size, you might want to change the
block size. For example, if your application uses millions of files that are only
6KB each, it makes sense to change the block size to 8KB. The minimum size
is 4KB. If your application will use only a few files, but they’re monstrously
huge, you might want to use 32KB or even 64KB blocks. One word of warning,
however: FreeBSD expects the fragment size to be exactly one-eighth of the
block size. Using another ratio for block:fragment size reduces performance.
The final column gives the number of cylinders in a cylinder group in
this filesystem. Always, always let FreeBSD calculate this. Given the number of
lies—er, sorry, I mean “translations”—that hard drives and operating systems
perform on hard disk geometry, and the fact that you can’t do anything useful
by changing cylinder group information, leave it alone.
Now that you understand disklabels, let’s make our own.







Backing Up and Restoring Disklabels
Before doing anything, back up the current disklabel to a file:
# disklabel /dev/da0s1 > da0s1.label.backup

With this file, you can restore the disklabel to its previous state with
disklabel’s -R flag:
# disklabel -R /dev/da0s1 da0s1.label.backup

Abracadabra! Your old disklabel has been restored!

Editing Disklabels
To create or rearrange partitions on a slice, change the disklabel. Changing
the disklabel destroys the partitions on the disk. This means that any data on
that disk is lost. (It really isn’t “destroyed until you write new data on the
drive”; it’s just lost.) I recommend editing the disklabels only on new disks
without any data, or on disks with excellent backups. You cannot edit the
disklabel on slices with mounted partitions.
For example, here’s the disklabel on one of my test disks:
# disklabel /dev/da0s1
# /dev/da0s1:
8 partitions:
#
size offset
c: 17767827
0
d: 17767827
0

538

C h ap te r 1 8

fstype
unused
4.2BSD

[fsize bsize bps/cpg]
0
0
# "raw" part, don't edit
2048 16384 28552

Decide how you want to partition this disk. According to fdisk(8), this
slice has 8,675MB of free space. I want 1GB of swap space, two 2GB partitions,
and a fourth partition with all the remaining space (3.5GB, more or less). All
of these partitions use the standard block size and fragment size. Now, edit
the disklabel using the -e flag, and you’ll get a copy of the disklabel in a text
editor. To create the four desired partitions, I edit the label to look like this:
# /dev/da0s1:
8 partitions:
#
size
b: 1G
c: 17767827
d: 2G
e: 2G
f: *

offset
*
0
*
*
*

fstype
swap
unused
4.2BSD
4.2BSD
4.2BSD

[fsize bsize bps/cpg]
0
0
2048 16384 28552

# "raw" part, don't edit

Where are all the sector and offset counts? disklabel(8) recognizes the
abbreviations K (kilobytes), M (megabytes), and G (gigabytes), and can do the
multiplication necessary to get a proper number of sectors in a partition of
that size. We just have to specify the size of each partition in the desired units.
Note the size of partition f, however. Here, the asterisk (*) tells disklabel to use
all of the remaining space. You can also use percentages in a disklabel to say,
for example, “Give this partition 50 percent of the remaining free space.”
disklabel(8) can also calculate the proper offset for each partition. The
asterisk (*) tells disklabel(8) to calculate the offset for you.2
Finally, note the fragment and block sizes. The fragment and block sizes of
partition d are explicitly stated, but only because the original label had a partition d that listed these values and I didn’t bother to erase them. disklabel(8)
assumes the default unless you specifically tell it otherwise.
Save your work and exit the text editor, then look at your new disklabel:
#
b:
c:
d:
e:
f:

size offset
2097152
16
17767827
0
4194304 2097168
4194304 6291472
7282051 10485776

fstype
swap
unused
4.2BSD
4.2BSD
4.2BSD

[fsize bsize bps/cpg]
0
0
2048 16384 28552
0
0
0
0
0
0

# "raw" part, don't edit

You can see that the sectors and offsets have been calculated, but what
about the block and fragment sizes on partitions e and f? When you create
filesystems on these partitions, newfs(8) will fill these in with the default values.

Replicating Drive Slicing and Partitioning
So, you can both back up and restore slice tables and disklabels from files.
Shouldn’t you be able to use these to duplicate slicing and partitioning on
identical disks? Why yes, you can! First, confirm that your disks are truly
2
Remember, BSD grew features because the program authors needed them. Apparently the
authors of disklabel(8) knew that they couldn’t do third grade math either, so don’t feel bad.

D is k Tr ick s wit h GEO M

539

identical. Not all 80 gig disks have the same number of sectors. If your disks
are slightly different, you can replicate the smallest disk on all of the other
disks.
First, edit your first disk’s slice and partition tables exactly the way you
want all of your disks to be subdivided. Back up that disk’s disklabels and slice
table to files, then “restore” those configurations to the other disks. For
example, here the disk /dev/da0 has a single slice. We copy the slice table of
that disk and the disklabel of that slice, and restore them to the drive /dev/da2:
#
#
#
#

fdisk -p /dev/da0 > da0.slice.table
disklabel /dev/da0s1 > da0s1.disklabel
fdisk -f da0.slice.table /dev/da2
disklabel -R /dev/da2s1 da0s1.disklabel

Use fdisk(8) and disklabel(8) to verify the new configuration of the disk
/dev/da2. You can use this to mass-produce identically partitioned disks when
you want to, say, use GEOM classes to build a storage array.

Missing Disklabels
Occasionally you might come across a slice with no disklabel at all. I find
this most commonly on test disks, when I have been repartitioning and
reslicing the disk to test some goofball idea. If a slice has no disklabel,
create one with -w:
# disklabel -w /dev/da0

You new disklabel now has a single a partition covering the entire slice.
Editing this to fit your needs is much easier than writing a disklabel from
scratch.

Building Filesystems
Now that you have slices and partitions, you’re almost ready to use the disk. All
that remains is creating a filesystem. Use newfs(8) to create a UFS filesystem on
a newly allocated partition. Use the -U flag to enable soft updates. We’ll talk
about some other newfs(8) options as we proceed through this chapter, but
for the moment we just want to create standard UFS2 filesystems:
# newfs -U /dev/da2s1d
/dev/da2s1d: 2048.0MB (4194304 sectors) block size 16384, fragment size 2048
using 12 cylinder groups of 183.77MB, 11761 blks, 23552 inodes.
with soft updates
super-block backups (for fsck -b #) at:
160, 376512, 752864, 1129216, 1505568, 1881920, 2258272, 2634624, 3010976,
3387328, 3763680, 4140032

540

C h ap te r 1 8

newfs(8) starts by printing out basic information about the new filesystem,
such as its size, the number of sectors and blocks, and the size of blocks and
fragments. You’ll then see the number of inodes. newfs(8) prints out the
location of each superblock as it proceeds, just so you know it’s doing something. (This used to be much more important when computers were much
slower than they are today; newfs(8) could take minutes to run on a large
filesystem, and it was nice to see that it was working and didn’t just lock up.)
You can specify the block and fragment sizes with the -b and -f flags,
respectively. If you do not specify the block and fragment sizes on the
command line, newfs(8) checks the disklabel for this information. If you do
not specify these, newfs(8) uses the defaults. I prefer to specify these in the
disklabel, simply because that gives me a better opportunity to double-check
my work. In any case, newfs(8) updates the disklabel with the block and
fragment sizes for later reference.
Now that you can slice, partition, and format disks at will, let’s look at
some more advanced topics in disk management. One such topic is RAID.

RAID
One of the big requirements in any serious data storage system is RAID, or a
Redundant Array of Independent Disks. The I in the definition once meant
Inexpensive, but that’s relative. A one-petabyte RAID array costs far less than a
single one-petabyte disk, but it is still rather expensive. A RAID system splits
data between the drives to improve performance or reliability. RAID can
work in either hardware or software.

Hardware vs. Software RAID
FreeBSD supports both hardware and software RAID. Hardware RAID is managed by the SCSI controller, and host adapters that provide RAID are called
RAID controllers. When you run RAID in hardware, the controller handles all
the computations of how to arrange data on the hard drives. Most hardware
RAID systems are very stable, and a hardware controller is usually the most
efficient way to handle RAID.
One problem with hardware RAID is that different controllers store data
on the disks in different formats. If your RAID controller goes bad, you cannot
replace it with a RAID controller of a different brand; it probably won’t
recognize the data on your disks. If you store critical data on a hardware RAID
array, you must have either a support agreement that provides you with an
identical controller in case of failure, a spare controller of the same make
and model, or excellent backups.
Software RAID is managed by the operating system, and the OS figures
out how to arrange the data on the disks. Compared to hardware RAID, this
method slightly increases system load but uses less expensive equipment. You
also don’t run the risk of losing your data when you cannot find a replacement
RAID controller of the exact same type.

D is k Tr ick s wit h GEO M

541

Using hardware RAID is typically much simpler than software because all
you have to do is follow the manual. RAID controllers usually have a simple
menu-driven BIOS that lets you set virtual disk sizes, restore damaged disks,
and configure your virtual disks. Software RAID, on the other hand, demands
that the system administrator actually knows what he’s doing. If you want to
know about hardware RAID, read your RAID controller’s manual—we’re
discussing software RAID. Many of the types of RAID supported by FreeBSD
also appear in hardware.
Many RAID manufacturers provide software tools to manage their
controllers from the operating system. You can manage and configure
amr(4) arrays with megarc (/usr/ports/sysutils/megarc), mfi(4) arrays with
MegaCli (/usr/ports/sysutils/linux-megacli), and aac(4) arrays with aaccli
(/usr/ports/sysutils/aaccli). 3Ware provides management software for twe(4)
and twa(4) controllers, and Areca provides a management program for
arcmsr(4) controllers.

GEOM RAID and Disk Size
For any RAID system, all of your hard drives must be the same size—or, at
least, you must use the same amount of space on each hard drive. If you have
a set of 10 500GB drives in a RAID array, but one of the drives fails and you
must replace it with a 700GB drive, you can only use 500GB of that drive. You
cannot use the remaining 200GB on that oversized drive without hampering
the performance of the entire array.
We’re going to assume that the disks or slices you use for software RAID
are all the same size. If you have several drives, check fdisk(8) for each to
confirm they are all the same size. If some of the drives are of slightly different
size, even by just a couple hundred sectors, GEOM uses the smallest drive size
as a limit.
Technically, you don’t have to have a slice table on data storage volumes.
You can run a mirror or a stripe on top of the raw disk—it’s all the same to
GEOM. Our examples do not use slice tables on the disks.

Parity and Stripe Size
Many RAID types use parity for error detection. Parity is just a very, very simple
checksum to ensure that data is copied correctly. A RAID system uses parity
for error detection, so that it can determine the proper state of a section of
disk. This gives the disk array the ability to provide redundancy. RAID systems
that use parity generally need to dedicate some disk space to parity, but most
of these systems configure a parity area transparently.
Similarly, RAID levels that distribute data amongst multiple drives can
use different stripe sizes. The stripe size is the size of the chunk of file written
on a single drive. For example, if you have a 512KB file and a stripe size of
128KB, this file is written in four stripes scattered amongst your disks. You’ll
see papers and references claiming that such-and-such stripe size is better for
this or that reason. These arguments might even be true—for particular

542

C h ap te r 1 8

workloads. If you really want to experiment with stripe sizes, do so with your
own real operations on your own hardware. You might get a performance
boost, but chances are it won’t be very much. In our examples, we’ll default
to using 128KB stripes unless specified otherwise. 128KB is a nice middle-ofthe-road value that works well for most workloads.

RAID Types
RAID comes in several varieties. Most hardware supports RAID-0, RAID-1,
and RAID-5. There are other types, however. FreeBSD notably supports
RAID-3, which is useful for certain applications. You’ll also see RAID 0+1
and RAID 10, with occasional appearances from RAID 50 and RAID 6.
RAID-0, or striping, is not really redundant and technically isn’t RAID at
all. It requires at least two disks, and data is shared between the disks in a way
that increases throughput and disk size. You can use RAID-0 to combine
several 500GB hard drives into a multiterabyte virtual disk, but a hard drive
failure on any one drive destroys all of the data on all drives. You must restore
from backup to access any data on that drive. RAID-0 is useful if you need a
single really big filesystem, but it provides no reliability benefits and is actually
more vulnerable than a single disk. The size of the RAID-0 array is the size of
all the hard drives combined.
RAID-1, or mirroring, is where the content of one disk is duplicated on
another. This is a great way to achieve low-cost reliability. I use mirroring
on all of my servers. Mirroring can give even a cheap desktop-chassis server
some measure of data protection. The size of the RAID-1 array is the size of
the smallest hard drive in the array.
In RAID-3, or striping with a dedicated parity disk, a spare hard drive is used
to maintain parity and integrity data for the data striped across the remaining
hard drives. This means that a failure in any one hard drive won’t cause loss
of data. You must have an odd number of disks—at least three—to use RAID-3.
For certain applications, the dedicated parity drive provides higher throughput. RAID-3 can only satisfy one I/O request at a time, however. If you’re
loading large files one at a time, you might consider RAID-3. The size of the
RAID-3 array is the size of all but one of your hard drives combined.
RAID-5, or striping with parity shared across all drives, is the current industry
standard for redundancy. Much like RAID-3, it uses parity to provide data
redundancy; loss of a single drive doesn’t destroy any data. Unlike RAID-3, the
parity space is divided amongst all of the drives. Throughput is unexceptional,
but a RAID-5 array can serve multiple I/O requests simultaneously. The size of
your RAID-5 array is the combined size of all but one of your hard drives.
Unfortunately, FreeBSD 7 does not have a stable software implementation
of RAID-5. You’ll find references to gvinum(8), a GEOM-based port of the
Vinum volume manager that supports RAID-5, but this software is not yet
fully reliable. Other implementations are in development, but not available
at this time.

D is k Tr ick s wit h GEO M

543

RAID 0+1 is a mirror of striped disks. You require disks in multiples of 2,
with a minimum of 4. Set up an array of disks in RAID 0, so that you have a
single large disk area. Then, mirror the entire array onto an identical set
of hard drives. The size of the RAID 0+1 array is the total size of half of your
hard drives, much like RAID-1. RAID 0+1 is not as fault-tolerant as RAID 10,
so it is rarely used. We will not discuss it.
RAID-10 is a stripe of mirrored disks. You need disks in multiples of 2, with a
minimum of 4, and (for full redundancy) two separate disk controllers. Disks
are paired into mirrors, and data is then striped across the mirrors. As the
system does not have to calculate parity, this is the fastest-performing highavailability disk system you can find. The size of the array is the total of half
of your hard drives.
GEOM can provide all of these RAID types, and more, in software.

Generic GEOM Commands
Most GEOM-related commands (such as gstripe(8), graid3(8), and friends)
support a set of common subcommands to assess and control various GEOM
operations. While you can use these commands through geom(8), I find it
less confusing to use the command for a specific module such as gstripe(8).
For example, the status subcommand shows the condition of a particular
GEOM subsystem. I can ask GEOM about the status of all of my RAID-3
devices by running geom raid3 status. I find it easier to use graid3(8) and
type graid3 status. We’ll use the latter form throughout the examples in this
chapter. Almost every GEOM module has the four commands load, unload,
list, and status.
The load command activates the kernel module for that class. For
example, gstripe load loads the geom_stripe.ko module into the kernel.
Similarly, the unload command removes that module from the kernel.
The status command prints the condition of the devices of that type. For
example, here we see the condition of the GEOM-mirrored disk set on my
mail server:3
# gmirror status
Name
Status
mirror/gm0 COMPLETE

Components
ad1
ad2

The format of this output varies between GEOM modules, as appropriate
for the device.
Finally, the list command lists all the module’s consumers, or the devices
underlying the module. The list command is frequently a synonym for “other
interesting stuff about this module that doesn’t quite fit in the other three

3
“What, after all this griping about using SCSI disks in servers you’re using IDE?” Yes, but it’s my
personal mail server. I get more email than I want, so having my hard disks catch on fire really
wouldn’t upset me. I doubt your boss would say the same thing.

544

C h ap te r 1 8

commands,” such as debugging output. For example, here’s a snippet of
the list output from a graid3 device:
# graid3 list

 Geom name: MyRaid3





State: COMPLETE
Components: 3
Flags: ROUND-ROBIN
GenID: 0
...
Providers:
1. Name: raid3/MyRaid3
Mediasize: 18210036736 (17G)
Sectorsize: 1024
Mode: r1w1e1
Consumers:
1. Name: da0
...



This particular server has a graid3 device called MyRaid3 . We see some
generic information applicable to most GEOM-based devices: The device is
complete (meaning that all the disks in the device are present, working, and
up to date) and has three consumers. We then get into graid3-specific debugging information .
Under the Providers heading , we see the list of devices provided by this
GEOM class. This server offers a RAID-3 device under/dev/raid3/MyRaid3 .
We can see the size of the device, block size, and other useful disk information.
We also have the Consumers section , which lists all of the underlying
disks in the GEOM device. Here you’ll find information not only about
the disks themselves, but also about the interaction of those disks with the
GEOM module.
These commands constitute the interface of the various GEOM modules
providing software-based disk virtualization.







Striping Disks
FreeBSD uses gstripe(8) to configure and manage disk striping. This lets you
create a single virtual disk out of two or more smaller disks. Striping has the
advantage of sharing transactions between the hard drives, theoretically
multiplying throughput by the number of drives in the array. (Reality rarely
mimics theory, of course!)
For this example, I’m using three hard drives, /dev/da0, /dev/da1, and
/dev/da2. Each has a single slice, with only one partition (d) on that slice. While
you can use a manual method to configure the striped drive each time you
need it, I assume that you want the striped drive to be automatically available
at boot.
First, load the geom_stripe kernel module with gstripe load. You can also
specify geom_stripe_load="YES" in /boot/loader.conf to load striping at boot.
D is k Tr ick s wit h GEO M

545

Creating a Striped Provider
Now tell the drives that they’re in a striped provider. Below, we create a stripe
named MyStripe. This stripe includes the three drives. Use the -s flag to give
the stripe size of 131,072 bytes, or 128KB. The -v flag tells gstripe(8) to be
more verbose:
# gstripe label -v -s 131072 MyStripe /dev/da0 /dev/da1 /dev/da2
Metadata value stored on /dev/da0.
Metadata value stored on /dev/da1.
warning: /dev/da2: only 9105018368 bytes from 9186602496 bytes used.
Metadata value stored on /dev/da2.
Done.

Here we see that gstripe(8) stores information about the striped provider
on the disks. The man page tells us that stripe information is stored in the
last sector of the disk. We can also see that the disk /dev/da2 is slightly larger
than the others, but gstripe(8) is smart enough to not use the excess space
on that disk.
Now that you’ve created your striped provider, look in /dev/stripe:
# ls /dev/stripe/
MyStripe

We have the disk device /dev/stripe/MyStripe. This virtual disk doesn’t
require a disklabel or a slice table, although you can use them if you like.
Mind you, aggregating several disks into a single large partition only to divide
them up again might seem odd. You just need a filesystem and a mount point
for the new drive. Don’t forget newfs(8)’s -U for soft updates.
# newfs -U /dev/stripe/MyStripe
/dev/stripe/MyStripe: 26042.8MB (53335720 sectors) block size 16384, fragment size 2048
using 142 cylinder groups of 183.77MB, 11761 blks, 23552 inodes.
with soft updates
super-block backups (for fsck -b #) at:
160, 376512, 752864, 1129216, 1505568, 1881920, 2258272, 2634624, 3010976, 3387328,
...

Take a close look at the output of newfs(8) above. Somehow, newfs(8)
found cylinder groups in this virtual disk. fdisk(8) reports finding cylinders,
heads, and sectors. This should really drive home that cylinder, head, and
sector information doesn’t mean much; sectors are the only thing that matters.
Now you can mount and use your striped disk, listing devices under
/dev/stripe in /etc/fstab just as you would any other disk device.

gstripe Destruction
If you want to stop striping (say, because you want to upgrade to something
with redundancy), be sure to tell your disks that they’re no longer part of a
stripe. Unmount the striped device and run gstripe clear /dev/da0 /dev/da1
/dev/da2 to erase the stripe configuration from the disks. You can then
unload the kernel module.
546

C h ap te r 1 8

Daily Status Check
If you’d like FreeBSD to perform a status check on your gstripe(8) devices
every day and include the results in your daily periodic(8) message, just add
daily_status_gstripe_enable="YES" to /etc/periodic.conf.

Mirroring Disks
FreeBSD configures and manages mirrored disks with gmirror(8). While
mirrors are traditionally pairs of disks, you can have a mirror with any number
of disks. Mirrored disks are one of the simplest forms of fault tolerance; if a
single drive fails, the data remains on the mirror. When choosing drives to
mirror, it’s best to make them physically separate: on different controllers,
different cables, even different servers if possible. For this example, I’m using
two SCSI hard drives, /dev/da0 and /dev/da1.
Mirroring a disk writes metadata at the very end of each partition, overwriting any data already there. I recommend only mirroring drives and
partitions that are newly created, as any data on these partitions tends to be
written at the front end of the partition. Older partitions frequently have
files scattered throughout the entire space.
Load the geom_mirror kernel module, or run gmirror load. To load
mirroring support at boot, add geom_mirror_load="YES" to /boot/loader.conf.

Creating a Mirror
gmirror(8) works much like gstripe(8): Identify the disks that you want to
mirror and use gmirror(8) to label the disks as participants. Unlike gstripe(8),
however, you usually create a mirror of a disk that already has data on it.
In this example, I’m already using disk /dev/da7 for /usr/ports. (Technically,
I’m using /dev/da7s1d as the partition, but that’s the only partition on the
disk.) You can only configure mirroring on unmounted disks, so I have to
unmount this partition before attempting the mirror. Labeling the disk as
part of a mirror is very fast, however.
# umount /usr/ports/
# gmirror label -v MyMirror /dev/da7
Metadata value stored on /dev/da7.
Done.
# mount /dev/da7s1d /usr/ports/
mount: /dev/da7s1d : No such file or directory

Ack! What happened to our partition? Don’t panic. Your disk is now part
of a mirrored device called MyMirror in /dev/mirror :
# ls /dev/mirror/
MyMirror
MyMirrors1
MyMirrors1c
# mount /dev/mirror/MyMirrors1d /usr/ports/

MyMirrors1d

A check of /usr/ports shows that your data is once again available.
D is k Tr ick s wit h GEO M

547

View the status of disks in your mirror with gmirror’s status command:
# gmirror status
Name
Status
mirror/MyMirror COMPLETE

Components
da7

All of the disks in your mirror are completely up to date. As there’s only
one disk, that’s what you would expect! Add disks to the mirror set with the
insert command. You must provide two arguments, the name of the mirror
and the drive you want to use:
# gmirror insert MyMirror /dev/da2

Once you have added the disk, check the mirror’s status:
# gmirror status
Name
Status
mirror/MyMirror DEGRADED

Components
da7
da2 (11%)

The mirror set is not fully up to date yet—it takes a few minutes to copy
data from the “good” disk to the new mirroring disk. Depending on the disk
size and the speed of your disk system, this might take from a few minutes to
several hours. You can always check mirror status to see how your update is
progressing.

Repairing Mirrors
The main purpose of a mirror is redundancy; when one hard drive dies, you
want to be able to recover data and/or maintain operations. Your hardware
dictates your ability to continue operations and/or recover. If you have a
system with hot-swappable SCSI or SATA drives, you can repair mirrors on
the fly, but if you’re using an ATA system you’ll probably have to shut down
to replace the bad hard drive.
First, remove the broken drive from the mirror set with gstripe(8)’s
forget command. Despite the scary-sounding name, this instructs gmirror(8)
to remove all drives that are not currently available from the mirror configuration. The forget command does not erase the whole mirror! Then just
use the same insert command you used to add a drive to the configuration
in the first place:
# gmirror forget MyMirror
# gmirror insert MyMirror /dev/da8

548

C h ap te r 1 8

Mirrored Boot Disks
Mirroring your boot disk isn’t as hard as you might think. While it’s even
possible to mirror your drives during installation using the emergency shell,
we’ll take a slightly easier route and do it after installation.
First, make sure that your /boot/loader.conf loads the geom_mirror kernel
module. Without this, FreeBSD won’t be able to find the boot disk; it’ll be on
a disk device that the kernel cannot recognize. I suggest rebooting once just
to be absolutely certain that geom_mirror loads at boot.
Your next problem is helping FreeBSD find the boot drive. In a default
FreeBSD install, your root partition is on a disk device such as /dev/ad0s1a.
When you have your root drive on a mirrored disk, however, your root partition will be on a device like /dev/mirror/MyBootDrives1a. Your other partitions
and slices will be on similar devices. You must have the correct devices in
/etc/fstab for FreeBSD to boot correctly. People suggest all sorts of tricks to
change /etc/fstab in single-user mode, without even vi(1) available. You can
use sed(1) or ed(1), or you can mount /usr to get vi(1). But why not make
the change in multi-user mode, saved in a separate file?
Copy /etc/fstab to /etc/fstab-mirror. Edit that file to give the new device
names for your partitions. For example, suppose you intend to create a drive
RootMirror0 for your boot drive. Here’s a small /etc/fstab:
/dev/ad0s1a
/dev/ad0s1e
/dev/ad0s1d

/
/usr
/var

ufs
ufs
ufs

rw
rw
rw

1
2
2

1
2
2

Once you have your mirror created, FreeBSD will expect an /etc/fstab like
this instead:
/dev/mirror/RootMirror0s1a
/dev/mirror/RootMirror0s1e
/dev/mirror/RootMirror0s1d

/
/usr
/var

ufs
ufs
ufs

rw
rw
rw

1
2
2

1
2
2

As you know what the necessary /etc/fstab looks like, create it ahead of
time while you have your friendly text editor handy. Triple-check your work,
however; it’s much easier to make certain your filesystem table is correct than
muck around with the boot loader or the fixit CD!
Now reboot into single-user mode. In single-user mode, FreeBSD mounts
only the root partition in read-only mode. While you can’t normally configure
any GEOM module on a mounted disk, setting the sysctl kern.geom.debugflags
to 16 lets you do exactly that. This won’t help you if you want to get your root
partition on almost any other sort of RAID, but in the case of mirrored disks
it works nicely. In single-user mode, put your new /etc/fstab in place, enable
GEOM debugging, and then create your root mirror:
#
#
#
#

cp /etc/fstab /etc/fstab-nomirror
cp /etc/fstab-mirror /etc/fstab
sysctl kern.geom.debugflags=16
gmirror label -v RootMirror0 /dev/ad0

D is k Tr ick s wit h GEO M

549

After you reboot, FreeBSD should find your root partition on the new
mirrored GEOM disk. Your mirror has only one disk in it, mind you, which
makes it a pretty wimpy mirror. Once you’re in multi-user mode again,
however, just run the appropriate insert command to add the backup disk:
# gmirror insert RootMirror0 /dev/ad6

Congratulations! You have boot disk redundancy.4 If you lose your main
hard drive, you’ll need to tell the BIOS to boot the other hard drive—but
that’s fairly minor compared to losing all your data.

M U LT I - WA Y M I RR OR S
While mirrors are traditionally two disks, using three-way mirrors can dramatically
decrease the time necessary for database backups without affecting redundancy. Put
your database data on a three-way mirror. When you want to run a cold backup,
shut down the database, remove a hard drive from the mirror, and start the database
again. Back up the hard drive removed from the mirror at your leisure; when the
backup is done, throw the third drive back into the mirror and let it recover normally.

Destroying Mirrored Disks
If you no longer want to mirror your disks, use the gmirror stop command on
the mirror name. You can then run gmirror clear to erase gmirror’s metadata
from the disks previously involved in the mirror:
# gmirror stop MyMirror
# gmirror clear /dev/da7 /dev/da2

This frees these disks for use in other GEOM modules.

Daily Status Check
FreeBSD can include a status check of your mirrored disks in its daily
periodic(8) run. Just add the line daily_status_gmirror_enable="YES" to
/etc/periodic.conf.

RAID-3
While RAID-3 doesn’t have a great deal of mindshare in the IT community, it
is very fast in applications such as CAD systems, which tend to work with
single large files. To use RAID-3, you need at least 3 disks, or an odd number

4
One question people ask is, “Why call this drive RootMirror0 and not just RootMirror?” When
you stick the slice name on the end of the device name, it comes out as RootMirrors1, and that
confuses me more than RootMirror0s1. Name the devices anything you like, so long as you don’t
confuse yourself or your co-workers.

550

C h ap te r 1 8

of disks above 3, such as 5, 7, 9, 11, and so on. Manage RAID-3 arrays with
graid3(8). As with the other RAID schemes, it’s best to make these disks as
physically separate as possible.
First, load the RAID-3 kernel module with graid3 load. You can also use
kldload(8) or put geom_raid3_load="YES" into /boot/loader.conf.

Creating a RAID-3
You should start to recognize a trend here: Identify the disks you want to
include in the RAID-3 and use graid3(8) to label these disks as participants.
Unlike gmirror(8), creating a RAID-3 destroys any data on these disks. In this
example, I’ll use /dev/da0, /dev/da1, and /dev/da2 to create a RAID-3 device
called MyRaid3. The last disk device you specify will be your parity device.
# graid3
Metadata
Metadata
warning:
Metadata
Done.

label -v -r MyRaid3 /dev/da0 /dev/da1 /dev/da2
value stored on /dev/da0.
value stored on /dev/da1.
/dev/da2: only 9105018368 bytes from 9186602496 bytes used.
value stored on /dev/da2.

This looks much like creating a mirror or stripe, with the exception of
the -r flag. Using -r with labeling makes graid3(8) use the parity disk when
reading, which accelerates random I/O requests but slows down sequential
I/O. As most disk activity is random, I’ve chosen to use it.
The disk devices are in /dev/raid3. Create a filesystem with newfs -U /dev
/raid3/MyRaid3, mount it as you like, and start loading it with data!

Repairing a RAID-3
RAID-3 offers a certain level of redundancy and fault tolerance. For example,
if one of our drives fails, you will see it in the status report:
# graid3 status
Name
Status
raid3/MyRaid3 DEGRADED

Components
da2
da0

That doesn’t look good. What happened to /dev/da1? We’ll figure that
out later,5 but for now the important thing is to restore service. I have several
warm spares in the SCSI array, so let’s put one in place to restore our
redundancy.

5
An investigation would show that someone ripped the drive out of the array to generate a
failure so he’d have something to write about. Fortunately, my boss doesn’t read my books.

D is k Tr ick s wit h GEO M

551

First, let’s remove any reference to the old drive from our RAID-3. List
the devices and get the number of each drive:




# graid3 list
...
1. Name: da2
Mediasize: 9186603008 (8.6G)
...
Number: 2
Type: PARITY
2. Name: da0
Mediasize: 9105018880 (8.5G)
...
Number: 0
Type: DATA
...

We’re looking for the type and number of each disk that’s left in the
RAID-3. Disk /dev/da2 is disk number 2 in the array; it is the parity disk.
Disk /dev/da0 is disk number 0 and contains data . So, we’ve lost one of
the data disks, but it’s backed up courtesy of the parity disk. Had we lost the
parity disk, our RAID-3 would be working roughly as a striped disk until we
replaced the parity disk.
We have disks 0 and 2 of our three disks, so the disk that failed was
number 1. Start by removing drive number 1 from the RAID-3 device,
making room for a replacement device. (Remember, a RAID-3 device can
only have an odd number of members, so you can’t just add another drive.)
Then add in a new drive number 1, device /dev/da3:





# graid3 remove -n 1 MyRaid3
# graid3 insert -n 1 MyRaid3 da3
graid3: warning: da3: only 9105018368 bytes from 9186602496 bytes used.

Well, that seemed painless, although we wasted a little bit of disk space
because our new disk was larger than the old one. But did it work? Check the
status.
# graid3 status
Name
Status
raid3/MyRaid3 DEGRADED

Components
da2
da0
da3 (1%)

It certainly seemed to work. Wait a little to see if the array rebuilds
successfully.

552

C h ap te r 1 8

Destroying a RAID-3
If you no longer want to use the RAID-3 device, unmount the partition and
use the graid3 stop command on the mirror. You can then erase graid3’s
metadata from the disks involved:
# graid3 stop MyRaid3
# graid3 clear /dev/da3 /dev/da2 /dev/da0

You can then use these disks elsewhere—say, in a RAID-10.

RAID-10
The stripe of mirrored disks, or RAID-10, is perhaps the fastest way to get
large amounts of high-performance storage. Disks are grouped into mirrors,
and then data is striped across the pairs. There is no parity disk; instead,
mirroring provides redundancy, and striping provides performance. While it
uses a whole lot of disks, RAID-10 is the only real choice for high-throughput
performance.
Technically, RAID-10 is a nested RAID. It’s a RAID 0 running on top of
RAID 1. Start by creating your mirrors (RAID 1) and then stripe (RAID 0)
the data across those mirrors. GEOM lets you build layers like this easily.
RAID-10 is normally used in high-performance, high-availability environments. It’s too expensive to use for customer web pages when a regular
mirror would suffice, but, oddly, the finance department seems willing to
spend that kind of money when their own data is at risk. It is very important
for the mirrored disks, if at all possible, to be on different controllers and in
different SCSI shelves in different racks.

RAID-10 Setup
Write down your disk arrangement before you begin, so you don’t get
confused. We’ll use four hard drives: /dev/da0, /dev/da2, /dev/da3, and
/dev/da4.6 Drives da0 and da2 are paired as mirror1, da3 and da4 make up
our mirror2. If possible, da0 and da3 should not be on the same SCSI controller as da2 and da4.
Configure your mirrors with the gmirror(8) command:
# gmirror label -v mirror0 /dev/da0 /dev/da2
# gmirror label -v mirror1 /dev/da3 /dev/da4

Check gmirror status to verify that all the mirrors were created as specified
and that each mirror is complete.

6

Not only did I unceremoniously rip /dev/da1 out of the SCSI cage, but I’m far, far too lazy to
get out of my chair to plug it in back in.
D is k Tr ick s wit h GEO M

553

Now create a gstripe(8) volume that includes both mirror devices. We’ll
use the standard stripe size and call the device raid10:
# gstripe label -v -s 131072 raid10 /dev/mirror/mirror0 /dev/mirror/mirror1
Metadata value stored on /dev/mirror/mirror0.
warning: /dev/mirror/mirror1: only 9105017856 bytes from 9186601984 bytes
used.
Metadata value stored on /dev/mirror/mirror1.
Done.

You’re done.
No, really. That’s everything. There’s a raid10 device in /dev/stripe. Run
newfs -U /dev/stripe/raid10 to put a filesystem on the device, mount it, and
you’re ready to go!

RAID-10 Status
To verify the condition of a RAID-10 array, you must check the status of both
the mirrors and the stripe device:
# gmirror status
Name
Status
mirror/mirror0 COMPLETE
mirror/mirror1

COMPLETE

# gstripe status
Name Status
stripe/raid10
UP

Components
da0
da2
da3
da4

Components
mirror/mirror0
mirror/mirror1

The important things you learn here is that the mirrors are working. If
one of the mirrors is degraded, replace the drive and rebuild the mirror. The
stripe device needs no maintenance; as with a regular gstripe(8) partition,
entirely losing one of the mirrors would destroy data. Remember to separate
your mirrored disks as widely as possible.

Destroying a RAID-10
To stop using the RAID-10, you must destroy the stripe and then destroy the
mirrored disks just as you would for any other RAID-0 and RAID-1 array.

Journaling Filesystems with gjournal(8)
Traditional wisdom says that the best way to ensure filesystem integrity is by
using a journaling filesystem. A journaling filesystem writes all disk changes to
a small file, or journal, before writing them to the actual filesystem. This means
that changes are safely and quickly cached on disk, so that in case of a power
outage there is less chance of data loss. The system then goes back and reads
554

C h ap te r 1 8

O T H E R N ES T E D R A I D T Y PE S
You’ve probably already thought, “Hey, I could make a RAID-13 by putting a RAID-3
on top of mirrored disks!” Yes, GEOM supports that. You can invent your own RAID
levels. You could build a RAID-33 out of three RAID-3 devices, a RAID-130 by striping
across two RAID-3 on top of mirrored disks, a RAID-303, and so on, and so on.
These setups are just wasteful of disks without any real gain, and many of them
actually pose more risk to your data than simpler RAID-10 setups. Nested RAID
involves a variety of subtle error conditions, and people have probably already
tested whatever bright idea you have. Be sure to read the literature on any type
of nested RAID before risking production data on it!

the journal, writing the changes to the filesystem proper in a leisurely fashion.
On boot, the system checks the journal file for any changes that were not yet
written to disk and makes those changes, ensuring up-to-date data.
For years, FreeBSD relied on soft updates to provide many journaling-like
features without implementing a full journaling filesystem. As disks grew
larger and larger, however, they increasingly challenged soft updates. UFS
and UFS2 are well known as stable and reliable filesystems. Implementing
journaling requires digging around in the guts of UFS2—which, even if done
by a programmer with a rare combination of sunny optimism and rampaging
paranoia, would still risk FreeBSD’s hard-earned reputation.
This remained a tough nut to crack until the introduction of GEOM’s
stackable filesystem modules. If adding journaling to UFS2 is a difficult
problem, why not create a disk layer beneath UFS2 to do the journaling for
us? That’s where gjournal(8) fits in. gjournal(8) only needs minor hooks in
the filesystem layer to teach the filesystem where to write its changes, and
then gjournal actually writes the changes to the disk. This eliminates fsck(8)
at boot after a crashed system.
gjournal(8) is a new feature in FreeBSD 7 and is actually a rather radical
approach to filesystem journaling. While many people use gjournal(8) with
great success, it should still be considered an experimental feature as of this
time. I use it on a multiterabyte filesystem with millions of Netflow files.
Compared to the mature UFS, however, gjournal(8) is an untested child.
While bugs get fixed when found, you might be the lucky one to find a new
bug that puts data at risk. Consider yourself warned.
The journal file uses one gigabyte of disk by default. This makes it
unsuitable for small filesystems. I usually use soft updates on filesystems of
20GB or smaller, and configure journaling on anything larger. Journaling
and soft updates are mutually exclusive; you may use both on the same
server, but not on the same filesystem.
Implementing gjournal(8) on a filesystem destroys any data on that filesystem. (In rare circumstances this can be avoided, but this is a safe assumption
most of the time.) If you want to convert an existing filesystem to use gjournal,
be sure to back it up first and be prepared to restore it from backup. Given the
size of modern disks, I find it fairly simple to back up the 400MB or so of an
D is k Tr ick s wit h GEO M

555

out-of-the-box FreeBSD /usr partition to another partition on the disk,
configure journaling on /usr in single-user mode, and restore /usr. Other
partitions start with much less data and are hence easier to convert. Once
gjournal(8) loses its experimental status, I expect support for it will be
integrated into the installer.

Configuring gjournal(8)
gjournal(8) requires a journal file. This can be a standalone disk device, or it
can actually be included in the partition containing the journaled filesystem.
If both the journal and the filesystem are on the same disk device, there’s no
performance reason to separate the two. If you can place your journal on a
separate disk device, however, you might see better performance. (Again,
two old-fashioned ATA drives on the same controller don’t count!) For our
initial examples, we’re going to assume that you have the journal file on the
same disk partition as the filesystem to be journaled.
Load the kernel module with gjournal load or manually load the
geom_journal.ko kernel module. You can also load the module at boot time
in /boot/loader.conf.
Now label the partition as a gjournal(8) user. Here we convert
/dev/da0s1d, the single partition on this disk, to use journaling:
# gjournal label /dev/da0s1d
gjournal: File system exists on /dev/da0s1d and this operation would destroy it.
Use -f if you really want to do it.

All right, convert isn’t the word I want. We eradicate the existing filesystem
and plop a journal file at the end of the partition.
# gjournal label -f /dev/da0s1d

gjournal prepares a journaling device for this partition. Instead of creating
a separate directory in /dev for journal devices, it appends .journal to the end
of the partition’s device name. All further filesystem operations are done on
the journaling device.
Jounalling and newfs(8)
Now create a filesystem on the journal device. Instead of the -U that enables
soft updates, use -J to enable journaling.
# newfs -J /dev/da0s1d.journal
/dev/da0s1d.journal: 7651.7MB (15670656 sectors) block size 16384, fragment size 2048
...

556

C h ap te r 1 8

One thing to note here is the size of the journaling device. This is an
8.5GB disk, but the journaling device is 7,651MB. Where’s my remaining
gigabyte? The journal file ate it. This is normal and expected.
Mounting Journaled Filesystems
Most UFS2 partitions are mounted noasync, but any journaled filesystems
must be mounted async. As we discussed in Chapter 8, async mounts just
dump data right to the disk and don’t wait to confirm that the data is safely
written to the disk. Journaling handles verification and integrity checking.
Specify async as a mount option in /etc/fstab :
/dev/da0s1d.journal

/usr

ufs

rw,async

2

2

Using a Separate Journal Device
The main advantage of using a separate journal device is that you might—
might—be able to preserve the existing filesystem. gjournal(8) stores its
metadata in the last sector of the partition. If that sector is used by the
filesystem, then the filesystem cannot be converted and must be newfs’ed.
Use the journal device as the second argument when you label the device.
Here we label /dev/da0s1d as a journaling filesystem using /dev/da1s1e as
the journaling device. If you are using the last sector of the to-be-journaled
filesystem, gjournal(8) will warn you.
# gjournal label /dev/da0s1d da1s1e

Instead of using newfs(8) to build a journaled filesystem on /dev/da0s1d,
use tunefs(8) to enable journaling and disable soft updates on an existing
filesystem:
# tunefs -J enable -n disable /dev/ad0s1d

Now mount the new journaled filesystem async using the .journal
device node.

De-Journaling Partitions
Just as with the RAID-based GEOM modules, use gjournal stop to shut down
the journal and gjournal clear on the device node to remove gjournal
metadata from the partition:
# gjournal stop da0s1d.journal
# gjournal clear /dev/da0s1d

Then unload the kernel module to completely shut down journaling
on your system.

D is k Tr ick s wit h GEO M

557

Filesystem Encryption
FreeBSD supports two different disk encryption methods, GBDE and GELI.
Both tools work very differently, support different cryptographic algorithms,
and are designed for different threat models. People talk about encrypting
disks all the time, but you rarely hear discussions of what disk encryption is
supposed to protect the disk from.
GBDE, or Geom-Based Disk Encryption, has specific features for high-security
environments where protecting the user is just as important as concealing
the data. In addition to a cryptographic key provided by the user, GBDE uses
keys stored in particular sectors on the hard drive. If either key is unavailable,
the partition cannot be decrypted. Why is this important? If a secure data
center (say, in an embassy) comes under attack, the operator might have a
moment or two to destroy the keys on the hard drive and render the data
unrecoverable. If the bad guys have a gun to my head and tell me to “Enter
the passphrase or else,” I want the disk system to say The passphrase is
correct, but the keys have been destroyed. I don’t want a generic error saying
Cannot decrypt disk. In the first situation, I still have value as a blubbering
hostage; in the latter, either I’m dead or the attackers get unpleasantly
creative.7
GELI is much more flexible, but won’t protect me from bodily harm the
way GBDE might. If someone might steal my laptop for the confidential
documents on it, or if an untrusted system user might snoop my swap space
to steal secrets, GELI suffices. GELI doesn’t try to protect my person, just my
data. As I won’t take any job that poses a higher than average risk of exposure
to firearms (keeping in mind that I live in Detroit), that’s perfectly fine with
me. GELI also uses FreeBSD’s cryptographic device driver, which means that
if your server has a hardware cryptographic accelerator, GELI takes advantage
of it transparently.
I should mention that in most cases, filesystem encryption is not only
pretentious but actively increases the risk of data loss. More data is lost to
encryption misconfiguration or lost keys than to theft. When I hear someone
say, “I’ve encrypted my whole hard drive!” I have a nearly psychic vision of
the future where that same person is saying, “I’ve lost access to everything on
my hard drive!” More often than not I’m correct. Consider carefully if you
really, truly need disk encryption.
In this example, we’ll use GELI to encrypt a disk partition on /dev/da0,
storing the cryptographic keys on the USB storage device mounted on /media.
You might find it more sensible to use a filesystem in a file (see Chapter 8) as
an encrypted partition. Very few people actually need to encrypt their entire
hard drive, and in certain circumstances, doing so might raise suspicions. I
have enough trouble explaining to airport security why my computer “looks
so weird.” In their minds, a boot prompt that says Insert cryptographic key
and enter cryptographic passphrase is only one step away from This man is a

7
Just for the record: If you have a sharp stick and the proper attitude, you can have my
passphrases.

558

C h ap te r 1 8

dangerous lunatic who requires a very thorough body cavity search. If you

really do need to encrypt certain documents, chances are they only total a
few megabytes. That’s a perfect application for a filesystem in a file.

Kernel Configuration
GELI requires the kernel module geom_eli.ko, which in turn requires crypto.ko.
You can load these at boot in /boot/loader.conf or add the appropriate devices
to your kernel configuration:
options GEOM_ELI
device crypto

Generating and Using a Cryptographic Key
GELI lets you use a key file and/or a passphrase as cryptographic keys for an
encrypted device. We’ll use both. To generate your cryptographic key file,
use dd(1) to grab a suitable amount of data from /dev/random and write it to
a file. Remember, /media is where our USB device is mounted. If you really
want to protect your data, create your key directly on the USB device and
don’t leave it on your filesystem where a hypothetical intruder could recover
it. (Even deleting the file still leaves remnants that a skilled attacker could
conceivably use.)
# dd if=/dev/random of=/media/da0.key bs=64 count=1
1+0 records in
1+0 records out
64 bytes transferred in 0.000149 secs (429497 bytes/sec)

The 64 bytes of data constitute a 512-bit key. You can increase the size of
the key if you like, at the cost of extra processor overhead when accessing the
encrypted filesystem. Don’t forget that your passphrase also increases key
complexity.
To assign a passphrase to the key, use geli init. The -s flag tells geli(8)
the desired sector size on the encrypted filesystem; 4,096, or 4KB, is usually a
decent sector size for this application. The -K indicates the key file. You must
also specify the device to be encrypted.
# geli init -s 4096 -K /media/da0.key /dev/da0
Enter new passphrase:
Reenter new passphrase:

A passphrase is much like a password except that it can contain spaces
and be of any length. If you really want to protect your data, I recommend
using a passphrase that is several words long, contains non-alphanumeric
characters, and is not a phrase in your native language.

D is k Tr ick s wit h GEO M

559

Now that you have a key, attach it to the device to be encrypted.
# geli attach -k /media/da0.key /dev/da0
Enter passphrase:

geli(8) now knows that /dev/da0 is an encrypted disk and that the file
/media/da0.key contains the key file. Once you enter the passphrase, you can
access the decrypted contents of the encrypted disk at the new device node
/dev/da0.eli. Of course, you need a filesystem to put any data on that disk.

Filesystems on Encrypted Devices
Before you build a filesystem on your encrypted device, purge the disk of any
lingering data. newfs(8) doesn’t actually overwrite most of the bits in a new
partition; it simply adds superblocks that indicate the location of inodes. If
you’ve used this disk before, an intruder would be able to see chunks of old
files on the disk. Worse, he’d see chunks of encrypted data placed there by
GELI. Before you put a filesystem on the disk, it’s best to cover the disk with a
deceptive film of randomness to make it much more difficult for an intruder
to identify which blocks contain data and which do not. Use dd(1) again:
# dd if=/dev/random of=/dev/da0.eli bs=1m
8684+0 records in
8683+1 records out
9105014784 bytes transferred in 1575.226434 secs (5780131 bytes/sec)

As you can see, this elderly system needed over 25 minutes to cover the
whole 8.5GB disk with high-quality randomness. The amount of time necessary
varies with system speed and the amount of randomness available. A cryptographic accelerator increases the amount of available randomness nicely.
Now that your disk is full of garbage, put a filesystem on it and attach it
to your system. Do not use soft updates on your encrypted disk. Encrypting a
disk device decreases system performance anyway.
# newfs /dev/da0.eli
# mount /dev/da0.eli /mnt

Your encrypted disk device is now available on /mnt. Store your confidential
files there.

Deactivating Encrypted Disks
When you have finished with your encrypted partition, shut down GELI and
return the partition to its inaccessible, encrypted state. First unmount the
partition, then use geli detach to disconnect the encrypted disk from the key.
While other GEOM modules usually place metadata in the last sector of
a partition, GELI does no such thing. You don’t have to clear any metadata
from a GELI partition before using it elsewhere. You must remember which

560

C h ap te r 1 8

partitions are encrypted and which are just empty, especially if you have many
free disks in your system. A completely encrypted disk doesn’t even have a
disklabel on it, so it’s easy to confuse an encrypted disk with a blank one!

Encrypting Swap Space with geli(8)
If you allow untrusted users to log onto your system, you might find it sensible
to encrypt your swap space. I find this useful on servers where multiple users
log in to use a particular application. Instead of generating a key that requires
protection, FreeBSD randomly generates a passphrase-free key on every boot
and encrypts the swap space with it. This scrambles the data in the swap
partition sufficiently so that no intruder can capture sensitive information
from swap space. While both GBDE and GELI support swap encryption in
this manner, we’re going to stick with GELI.
In addition to loading the geom_eli kernel module at boot, you have to
make a minor change to /etc/fstab. By adding the .eli extension to your swap
device name, you tell FreeBSD to automatically use a GELI-encrypted swap
partition. So, for example, the swap space on my machine would change from:
/dev/ad0s1b

none

swap

sw

0

0

0

0

to:
/dev/ad0s1b .eli

none

swap

sw

This minor change is all you need to encrypt your swap. Personally, I
find it most useful to simply add more memory to a secure server so it does
not have to swap to disk, but that might not be an option for you.

Disk Device Network Exports
You need a CD burner in machine A, but it’s installed in machine B? Or
maybe it’s a hard drive partition you need to access, or even a filesystem on
disk? Not a problem! Just export the device across the network through
geom_gate and mount it on the other machine.
Why bother to export the device, when you could just log into the other
machine? Perhaps you want to use a piece of software that expects to access
a device rather than a file. Maybe you want to take an image of an unmounted
partition, and the machine with that partition doesn’t have enough space to
hold that image. Perhaps you’d like to use gmirror(8) across the network, so
if one machine dies you have an up-to-date copy of the disk. Or maybe just
because it’s kind of nifty and will make other system administrators look at
you cross-eyed. geom_gate is a client-server application; the system with
the device to be exported is the server and the system that mounts the remote
device is the client. Both client and server require the geom_gate.ko kernel
module loaded.

D is k Tr ick s wit h GEO M

561

geom_gate Security
The geom_gate client-server connection is not encrypted, so anyone who
could intercept the traffic can see the transactions. Mind you, they’ll probably
have a great deal of difficulty recognizing the contents of that traffic! If you’re
using geom_gate across an untrusted network, a VPN or at least an SSH tunnel
is advisable. geom_gate traffic uses TCP port 3080 by default. (If you want to
have some real fun, you can stack geom_gate on top of GELI!)
The server has access control in the form of an exports file, much like
NFS. The server only exports the devices specified in the exports file, and only
to the identified clients.

geom_gate Server Setup
You can only export CDs, disk images, and partitions with geom_gate. You
cannot export tape drives, devices such as /dev/random or /dev/io, or terminal
devices. If you can’t mount(8) it, you cannot export it. A device cannot be
mounted while exported, however; it must be mountable but unmounted.
To export a device, list it in the geom_gate exports file, /etc/gg.exports.
Each entry in the exports file has this format:
host

permissions

device

The host can be either an IP address (e.g., 192.168.1.2), a network (e.g.,
192.168.1.0/24), or a hostname (such as pesty.blackhelicopters.org).
Assign each exported device read-write (RW), read-only (RO), or writeonly (WO) privileges.
Finally, list the exact device node you want to export. For example, here
we permit read-only access to the server’s CD drive from every host on my
private network and then assign read-write access to the disk device /dev/da0
from my laptop’s IP address:
192.168.1.0/24
192.168.1.202

RO
RW

/dev/acd0
/dev/da0

Now start ggated(8). As of this writing, ggated(8) is not integrated into
the rc.d startup system. Fortunately, you’ve all read Chapter 12 so you know
exactly how to write a rc.d script that integrates perfectly with FreeBSD.
ggated(8) should work without any arguments:
# ggated

geom_gate is now listening for client requests on TCP port 3080.

562

C h ap te r 1 8

geom_gate Client Setup
Now that we have something exporting devices, let’s fire up a client. The
client doesn’t need a configuration file, just the ggatec(8) command. Use
this syntax to create a local device node for a device node exported from
a remote server:
# ggatec create -o permissions servername devicename

For example, suppose we want read-only access to the server’s CD drive.
The server is named sardines.blackhelicopters.org, and the CD is /dev/acd0. This
device is listed in /etc/gg.exports on the server.
# ggatec create -o ro sardines /dev/acd0
ggate0

ggatec(8) replies with the local device node (/dev/ggate0) that corresponds
to the remote device node (/dev/acd0). If this is really an exported CD device,
I should be able to see the CD in that drive.
# mount -t cd9660 /dev/ggate0 /cdrom/

A simple ls /cdrom shows that the server’s CD is now mounted on the
client. Pretty cool, huh?
The next question is, how far can you take this? Let’s export an actual
hard drive and see what sort of abuses we can inflict upon it.
# ggatec create -o ro sardines /dev/da0
ggate1

The obvious thing to try is to use this as a regular hard drive. Can
we fdisk(8), disklabel(8), and newfs(8) it? The short answer is, no, we can’t.
fdisk(8) can’t write the first sector of the drive. We can’t use disklabel(8)
because disklabels belong on slices, not on whole disks, and we didn’t export
/dev/da0s1, only /dev/da0. If we export /dev/da0s1, we won’t get the various
partitions beneath it, only the whole slice. You must be very specific about what
you export, and even then you cannot perform low-level operations on it.
Once you have partitions and a filesystem on the server disk, however, you can
mount those filesystems as you desire and access data on them transparently.
# ggatec create -o ro sardines /dev/da0s1a
ggate3
# mount /dev/ggate3 /mnt

The remote disk /dev/da0s1a is now mounted locally at /mnt.

D is k Tr ick s wit h GEO M

563

Identifying geom_gate Devices
After you’ve been using geom_gate for a while, it’s easy to forget exactly
which devices you have mounted from which machines. The device name
/dev/ggate isn’t a too helpful mnemonic for the actual device. ggatec(8) has
a list command that gives details on locally mounted geom_gate devices.
Use the -v flag to get detailed information on the remote devices:
# ggatec list -v
NAME: ggate0
info: sardines:3080 /dev/acd0
access: read-only
timeout: 0
queue_count: 0
...

While ggatec(1) lists a great deal of information about each device, the
important bits for most of us are the remote device server, the remote device
name, and the type of access we have to that device.

Shutting Down geom_gate
When you’re finished with the exported devices, remove them from the
client. Unmount the device on the client locally and then use ggatec(8)’s
destroy function. For example, if we have /dev/ggate0 mounted as /cdrom:
# umount /cdrom
# ggatec destroy -u 0

The -u flag supplies the number of the ggate device we are eliminating.
We use -u 0 because we are shutting down device ggate0.
On the server side, just stop ggated(8) with pkill(1):
# pkill ggated

Poof! Your network devices are no longer exported.

Oops! Rescuing geom_gate
Suppose you weren’t quite done with geom_gate when you killed it. For
example, I unmounted /cdrom before destroying /dev/ggate0, but I still had
/dev/ggate3 mounted at /mnt. I cannot access that partition now, I cannot
unmount it, and if I try to copy data to it, the command hangs. What to
do? ggatec(8) has a rescue command for exactly this sort of situation. Restart
ggated(8) on the server and then start a new ggatec(8) process on the client.

564

C h ap te r 1 8

The syntax for rescue is much like that for create, except that you can specify
a ggate device node number. Here we reactivate /dev/ggate3 for our /mnt
mount point:
# ggatedc rescue -o rw -u 3 sardines /dev/da0s1a

There. Now, nobody needs to know that you goofed!

Mirroring Disks Across the Network
Two servers can share a single disk image. When data is written to one disk, it is
transparently copied to the other. You can use this to achieve a certain degree
of high availability with commodity hardware. Use ggated(8) to provide disk
exporting and gmirror(8) to handle the replication.
Remember that network speed is much lower than disk speed, and that
mirroring disks across the network is a bad choice for I/O-intensive applications. For example, as you read this, I expect 3Gbps SATA-II disks to be
common. That’s three gigabits a second, or three times the speed of a gigabit
network! When combined with the fact that not all gigabit network hardware
actually runs at gigabit speeds, if you’re actually using those 3Gbps of disk
I/O, a networked backup will sadly disappoint you. For low-I/O applications,
however, a network-mirrored disk of vital data might be useful.
In the example below, we mirror a 100MB file-backed filesystem across
the network. This is large enough to be useful for, say, a small database, but
small enough to run on my modest network.

Backup Server Setup
Create a 100MB file on one server and then copy it to the other server. This
way, you start with the same data file on both sides, hopefully reducing the
initial synchronization time. The simplest way to create a file of a given size
is with truncate(1):8
# truncate -s100m /var/db/dbmirrorbackup.img

truncate(1)’s -s flag dictates the size of the file to create. You can use k
for KB, m for MB, or G for GB. (You can also use t for TB, but we don’t quite
have disks of that size. Yet.)
Now, configure /etc/gg.exports to allow our primary server to access this disk
image. The primary server (in this example, at 192.168.1.2) needs read/write
privileges on this device:
192.168.1.2

RW

/var/db/dbmirrorbackup.img

Now start ggated(8). Your backup server is ready.

8
If truncate(1) is the easiest way to make a file, why did I make you use dd(1) back in Chapter 8?
Because you need to know dd(1). It was for your own good.

D is k Tr ick s wit h GEO M

565

Primary Server Setup
The primary server is slightly more complicated—but not by much. First,
attach a ggate device to the remote file:
# ggatec create backupserver /var/db/dbmirror.img
ggate0

Our remote filesystem file is available locally as /dev/ggate0. Now, load
gmirror(8) and label the ggate device as a mirror. Here, our mirror device is
called remotemirror0:
# gmirror load
# gmirror label remotemirror0 /dev/ggate0

We now have a mirror device. We need a local filesystem file, created just
like the backup filesystem:
# truncate -s100m /var/db/dbmirrorprimary.img

Attach the local file to a memory device, so it gets a device node, and then
insert that device node into the mirror:
# mdconfig -a -t vnode -f /var/db/dbmirrorprimary.img
md0
# gmirror insert remotemirror0 /dev/md0

You’ll see that the mirror quickly becomes up to date. There’s no data in
it, after all!
# gmirror status
Name
mirror/remotemirror0

Status
DEGRADED

Components
ggate0
md0 (33%)

When the mirror is up to date, which should not take more than a
couple of minutes for a filesystem of this size, use newfs(8) and mount it:
# newfs -U /dev/mirror/remotemirror0
# mount /dev/mirror/remotemirror0 /database

You now have a distributed mirror mounted on /database.
Depending on your system load and the throughput on this networkmirrored disk, you might find it useful to mount the disk noatime to minimize
disk writes.

566

C h ap te r 1 8

Mirror Failover and Recovery
The purpose of a mirrored disk is to allow you to recover data after a
disk failure, and the purpose of mirroring a disk across the network is
to allow you to recover after a server failure. You can use heartbeat
(/usr/ports/sysutils/heartbeat) in combination with mirrored disks to create a
low-cost, high-availability solution, automatically making one system take
over when the other fails. With heartbeat, one system watches another and
can take over operations when the master system disappears.
Your client machine would be the primary server and would handle system
activity most of the time. In the event of a system failure, however, the backup
machine takes on the role of the master machine. The backup machine can
activate its copy of the mirror with mdconfig(8) and mount(8):
# mdconfig -a -t vnode -f dbmirrorbackup.img
md0
# fsck -B /dev/md0
# mount /dev/md0 /database

You can then activate your database server program and start using the
data on the mirrored partition.
When the master system recovers, it reattaches the backup file as a ggate
device, relabels the mirror, inserts the local partition into the mirror, and lets
the mirror rebuild from the backup. Once the mirror is completely rebuilt,
heartbeat can switch services back to the master machine. Scripting and
automating failover are not terribly difficult but highly implementationand environment-dependent, so we’ll have to leave that for you.
High availability like this works best if you use a transactional database,
such as PostgreSQL as opposed to MySQL. It’s also best if you can keep the
amount of “live” data on your mirrored partition to a minimum, relying on
periodically archiving data to the other server’s remote disk. High availability
requires careful application design and should not be taken lightly.
This concludes our tour of the most popular GEOM modules. While
you’ll find several other modules of interest in FreeBSD, with the background
you’ve gained from this chapter deploying any of them will be fairly straightforward and pain free. Let’s move on into a topic near every system administrator’s heart: performance.

D is k Tr ick s wit h GEO M

567

19
SYSTEM PERFORMANCE AND
MONITORING

Even if “It’s slow!” is not the most dreaded
phrase a system administrator will ever hear,
it’s pretty far up on that list. The user doesn’t
know why the system is slow and probably can’t
even quantify or qualify the problem any further than
that. It just feels slow. Usually there’s no test case, no set
of reproducible steps, and nothing particularly wrong. A slowness complaint
can cause hours of work as you dig through the system trying to find problems
that might or might not even exist. One phrase is more dreadful still, especially
after you’ve invested those hours of work: “It’s still slow.”
An inexperienced system administrator accelerates slow systems by buying
faster hardware. This exchanges “speed problems” for costly parts and even
more expensive time. Upgrades just let you conceal problems without actually
using the hardware you already own, and sometimes they don’t even solve
the problem at all.

FreeBSD includes many tools designed to help you examine system
performance and provide the information necessary to learn what’s actually
slowing things down. Once you understand where a problem is, identifying
the solution to the problem becomes much simpler. You might actually need
faster hardware, but sometimes shifting system load or reconfiguring software
might solve the problem at much less expense. In either case, the first step is
understanding the problem.

Computer Resources
Performance problems are usually caused by running more tasks than the
computer can handle. That seems obvious, but think about it a moment.
What does that really mean?
A computer has four basic resources: input/output, network bandwidth,
memory, and CPU. If any one of them is filled to capacity, the others cannot
be used to their maximum. For example, your CPU might very well be waiting
for a disk to deliver data or for a network packet to arrive. If you upgrade
your CPU to make your system faster, you will be disappointed. Buying a
whole new server might fix the problem, but only by expanding the existing
bottleneck. The new system probably has more memory, faster disks, a better
network card, and faster processors than the old one. You have deferred the
problem until the performance reaches some new limit. However, by identifying where your system falls short and addressing that particular need, you
can stretch your existing hardware much further. After all, why purchase a
whole new system when a few gigabytes of relatively inexpensive memory
would fix the problem? (Of course, if your goal is to retire this “slow” system
to make it your new desktop, that’s another matter.)
Input/output is a common bottleneck. A PCI bus has a maximum
throughput, and while you might not be pushing your disk or your network
to their limits, you might be saturating the PCI bus by heavily using them
both simultaneously.
One common cause of system slowdowns is running multiple large
programs simultaneously. For example, I once thoughtlessly scheduled a
massive database log rotation that moved and compressed gigabytes of data
at the same time as the daily periodic(8) run. Since the job required shutting
down the main database and caused application downtime, speed was
crucial. Both the database job and the periodic(8) run slowed unbearably.
Rescheduling one of them made both jobs go more quickly.
We’re going to look at several FreeBSD tools for examining system performance. Armed with that information, we’ll consider how to fix performance
issues. Each potential bottleneck can be evaluated with the proper tools.
FreeBSD changes continually, so later systems might have new tuning options
and performance features. Read tuning(7) on your system for current performance tips.

570

C h ap te r 1 9

W H AT I S N OR M A L?
One word you’ll keep tripping over in this chapter is abnormal. As the system
administrator, you’re supposed to know what is normal for your system. It’s somewhat
like art; you might not be able to define normal, but you need to recognize abnormal
when you see it. Use these tools regularly when the system is behaving itself, so you
can have a good idea of which results are out of whack during system slowdowns.
Pay attention to your hardware!

Checking the Network
If your system’s slowness stems from network problems, you need more bandwidth or you must make better use of the bandwidth you have. This can be
summarized as, “You can’t fit ten pounds of bandwidth in a five-pound circuit.”
If your T1 is full, making everything else slow, stop filling it!
To check for bandwidth problems, begin by monitoring how much
bandwidth your system is using. Consult netstat -m and netstat -s, and look
for errors or places where you’re out of memory or buffers. Use a packet
sniffer or Netflow to catch your big traffic generators, and use MRTG or
Cricket to track utilization on a long-term basis.
Other system conditions are much more complicated. Start by checking
where the problem lies with vmstat(8).

General Bottleneck Analysis with vmstat(8)
FreeBSD includes several programs for examining system performance.
Among these are vmstat(8), iostat(8), and systat(1). We’ll discuss vmstat
because I find it most helpful; iostat(8) is similar to vmstat(8), and systat(1)
provides the same information in an ASCII graphical format.
vmstat(8) shows the current virtual memory statistics. While its output
takes getting used to, vmstat(8) is very good at showing large amounts of data
in a small space. Type vmstat at the command prompt and follow along.
# vmstat
procs
memory
r b w
avm
fre
3 0 0 580416 1388912

page
flt re
234 11

pi
7

po
0

fr
209

disks
sr ad0
0
0

faults
cpu
in
sy cs us sy id
171 5796 3816 3 10 87

vmstat divides its display into six sections: process (procs), memory, paging
(page), disks, faults, and cpu. We’ll look at all of them quickly and then discuss
in detail those parts that are the most important for investigating performance
issues.

Sy s te m Perf or ma nc e a n d M on it or in g

571

Processes
vmstat(8) has three columns under the procs heading.
r

The number of processes that are blocked waiting for CPU time.
These processes are ready to run but cannot get access to the CPU.
If this number is high, your CPU is the bottleneck.

b

The number of processes that are blocked waiting for system input
or output—generally, waiting for disk access. These processes will
run as soon as they get their data. If this number is high, your disk
is the bottleneck.

w

The number of processes that are runnable but are entirely
swapped out. If you regularly have processes swapped out, your
memory is inadequate for the work you are doing on the system.

Memory
FreeBSD breaks memory up into chunks called pages. These pages, allocated
to processes, are all of uniform size. The size of a page is hardware- and
OS-dependent. The system treats each page as a whole—if FreeBSD must
shift memory into swap, for example, it does that on a page-by-page basis.
The memory section has two columns.
avm

The average number of pages of virtual memory that are in use.
If this value is abnormally high or increasing, your system is
actively consuming swap space.

fre

The number of memory pages available for use. If this value is
abnormally low, you have a memory shortage.

Paging
The page section shows how hard the virtual memory system is working. The
inner workings of the virtual memory system are an arcane science that I won’t
describe in detail here.

572

C h ap te r 1 9

flt

The number of page faults, where information needed was not
in real memory and had to be fetched from swap space or disk.

re

The number of pages that have been reclaimed or reused from
cache.

pi

Short for pages in; this is the number of pages being moved from
real memory to swap.

po

Short for pages out; this is the number of pages being moved
from swap to real memory.

fr

How many pages are freed per second.

sr

How many pages are scanned per second.

Moving memory into swap is not bad, but consistently recovering pagedout memory indicates a memory shortage. Having high fr and flt values can
indicate lots of short-lived processes—for example, a CGI script that starts
many other processes or a cron job scheduled too frequently.

Disks
The disks section shows each of your disks by device name. The number shown
is the number of disk operations per second, a valuable clue to determining
how well your disks are handling their load. You should divide your disk
operations between different disks whenever possible and arrange them on
different buses when you can. If one disk is obviously busier than the others,
and the system has operations waiting for disk access, consider moving some
frequently accessed files from one disk to another. One common cause of
high disk load is a coredumping program that can restart itself. For example,
a faulty CGI script that dumps core every time someone clicks on a link will
greatly increase your disk load.
If you have a lot of disks, you might notice that they don’t all appear on
the vmstat display. vmstat(8) is designed for 80-column display and therefore
cannot list every disk on a large system. If, however, you have a wider display
and don’t mind exceeding the 80-column limit, use the -n flag to set the
number of drives you want to display.

Faults
Faults are not bad; they’re just received system traps and interrupts. An
abnormally large number of faults is bad, of course—but before you tackle this
problem, you need to know what’s normal for your system.
in

The number of system interrupts (IRQ requests) received in the
last five seconds.

sy

The number of system calls in the last five seconds.

cs

The number of context switches in the last second, or a persecond average since the last update. (For example, if you have
vmstat update its display every five seconds, this column displays
the average number of context switches per second over the last
five seconds.)

CPU
Finally, the cpu section shows how much time the system spent doing user
tasks (us), system tasks (sy), and how much time it spent idle (id). top(1)
presents this same information in a friendlier format, but only for the
current time, whereas vmstat lets you view system utilization over time.

Using vmstat
So, how do you use all this information? Start by checking the first three
columns to see what the system is waiting for. If you’re waiting for CPU access
Sy s te m Perf or ma nc e a n d M on it or in g

573

(the r column), then you’re short on CPU horsepower. If you’re waiting for
disk access (the b column), then your disks are the bottleneck. If you’re
swapping (the w column), you’re short on memory. Use the other columns
to explore these three types of shortages in more detail.

Continuous vmstat
You’re probably more interested in what’s happening over time, rather than
in a brief snapshot of system performance. Use vmstat’s -w flag and a number
to run it as an ongoing display updating every so many seconds. FreeBSD
recalculates its virtual memory paging counters every five seconds, but other
counters are updated continuously:
# vmstat -w 5
procs
memory
r b w
avm
fre
1 1 0 184180 12876
0 1 0 184180
8360
0 1 0 184180
7984
0 1 0 184180 14892
...

page
flt re
8
0
1
0
0
0
0
0

pi
0
0
0
0

po
0
0
0
0

disks
faults
cpu
fr sr ad0 ad4 in
sy
cs us sy id
30 11
0
0 1143 122 517 0 1 99
260
0 373
0 1513 419 2287 0 5 95
224
0 387
1 1528 428 2318 0 5 95
62 346 188
1 1328 221 1322 0 3 96

Every five seconds, an updated line appears at the end. You can sit there
and watch how your system’s performance changes when scheduled jobs kick
off or when you start particular programs. Hit CTRL-C when you’re done. In
this example, processes are sometimes waiting for CPU time (as shown by
the occasional 1 in the r column), but we have something waiting for disk
access all the time.
An occasional wait for a system resource doesn’t mean you must upgrade
your hardware; if performance is acceptable, don’t worry about it. Otherwise,
however, you must look further. In this case, I’m looking at vmstat(8) because
someone reported a performance problem. vmstat(8) shows that something
is waiting for disk all of the time. Let’s see just how busy our disks are.

Disk I/O
We looked at disk management in Chapter 8, and again in Chapter 18. When
it comes to performance, disk speed is a common bottleneck. If programs
repeatedly wait for disk activity to complete before proceeding, they run
more slowly. This is commonly called blocking on disk, meaning that the disk
is preventing program activity. The only real solution for this is to use faster
disks, install more disks, or reschedule the load.
While FreeBSD provides several tools to check disk activity, my favorite is
gstat(8), so we’ll use that. Just enter gstat without arguments for a display that
updates every second or so:
L(q)
0

574

C h ap te r 1 9

ops/s
0

r/s
0

kBps
0

ms/r
0.0

w/s
0

kBps
0

ms/w
0.0

%busy Name
0.0| ad0s1b.eli

1
0
1
0
0
0
0
0
1

213
0
213
0
0
0
0
2
211

211
0
211
0
0
0
0
0
211

1066
0
1066
0
0
0
0
0
1066

4.6
0.0
4.7
0.0
0.0
0.0
0.0
0.0
4.7

2
0
2
0
0
0
0
2
0

4
0
4
0
0
0
0
4
0

9.2
0.0
9.2
0.0
0.0
0.0
0.0
9.2
0.0

97.9|
0.0|
98.0|
0.0|
0.0|
0.0|
0.0|
0.9|
98.2|

ad0
acd0
ad0s1
ad0s1a
ad0s1b
ad0s1c
ad0s1d
ad0s1e
ad0s1f





We see a line for each disk device, slice, and partition, and various information for each. gstat(8) shows all sorts of good stuff, such as the number of
reads per second (r/s), writes per second (w/s), the kilobytes per second of
reading and writing, as well as the number of milliseconds each read or write
takes. The first thing to look for is the %busy column, second from the right.
The swap space partition , our first entry, is idle. Disk ad0 is running
at 97 percent of capacity, however. No wonder our system feels slow! Slightly
further down, we see that the big disk activity on disk ad0 is actually concentrated on slice ad0s1 . Most of the slices are idle, and there’s just a smidge
of activity on ad0s1e. Partition ad0s1f , however, is very busy, at 98.2 percent
utilization and hundreds of reads per second. Checking /etc/fstab or mount(8)
shows that ad0s1f is /usr on this server. Something is reading lots and lots of
data from this disk . . . but what? To find out, we need other tools.





CPU, Memory, and I/O with top(1)
The top(1) tool provides a good overview of system status, displaying information about CPU, memory, and disk usage. Just type top to get a full-screen
display of system performance data. The display updates every two seconds,
so you have a fairly accurate, close to real-time system view.

last pid: 977; load averages: 1.14, 0.80, 0.18 up 0+00:18:03 18:30:34
41 processes: 2 running, 39 sleeping
CPU states: 0.0% user, 0.0% nice, 2.1% system, 0.0% interrupt, 97.9% idle
Mem: 106M Active, 139M Inact, 130M Wired, 8164K Cache, 68M Buf, 1607M Free
Swap: 4096M Total, 4096M Free
PID
840
873
903
853
762
859
...

USERNAME
mwlucas
mwlucas
mwlucas
mwlucas
root
mwlucas

THR PRI NICE
SIZE
RES STATE
1 96
0
278M 24740K select
1 96
0
113M 81916K CPU0
6 20
0
169M
103M kserel
1 96
0 7924K 6608K select
1 96
0 3208K
964K select
1
4
0 18496K 10544K select

C
1
0
0
1
1
0

TIME
0:33
0:51
0:31
0:04
0:02
0:00

WCPU
1.76%
0.59%
0.00%
0.00%
0.00%
0.00%

COMMAND
Xorg
acroread
soffice.bin
wmaker
moused
yank

Very tightly packed, isn’t it? top(1) crams as much data as possible into
a standard 80 × 25 terminal window or X terminal. Let’s take this apart and
learn how to read it.
Sy s te m Perf or ma nc e a n d M on it or in g

575

PID Values
Every process on a Unix machine has a unique process ID, or PID. Whenever
a new process starts, the kernel assigns it a PID one greater than the previous
process. The last PID value is the last process ID assigned by the system. In
the previous example, our last PID is 977 . The next process created will be
978, then 979, and so on. Watch this number to see how fast the system
changes. If the system is running through PIDs more quickly than usual, it
might indicate a process forking beyond control1 or something crashing and
restarting.



Load Average
The load average is a somewhat vague number that offers a rough impression
of the amount of CPU load on the system. The load average is the average
number of threads waiting for CPU time. (Other operating systems have
different load average calculation methods.) An acceptable load average
depends on your system. If the numbers are abnormally high, you need to
investigate. Many Pentiums feel bogged down at a load average of 3, while
some modern systems are still snappy with a load average of 10.
top(1) lists three load averages. The first (1.14 in our example) is the load
average over the last minute, the second (0.80) is for the last five minutes, and
the last (0.18) is for the last 15 minutes. If your 15-minute load average is high,
but the 1-minute average is low, you had a major activity spike that has since
subsided. On the other hand, if the 15-minute value is low but the 1-minute
average is high, something happened within the last 60 seconds and might
still be going on now. If all of the load averages are high, the condition has
persisted for at least 15 minutes.

Uptime



The last entry on the first line is the uptime , or how long the system has
been running. This system has been running for 18 minutes, and the current
time is 18:30:34. I’ll leave it up to you to calculate when I booted this system.

Process Counts
On the second line, you’ll find information about the processes currently
running on the system . Running processes are actually doing work—
they’re answering user requests, processing mail, or doing whatever your
system does. Sleeping processes are waiting for input from one source or
another; they are just fine. You should expect a fairly large number of
sleeping processes at any time. Processes in other states are usually waiting
for a resource to become available or are hung in some way. Large numbers
of non-sleeping, non-running processes hint at trouble. The ps(1) command
shows the state of all processes.



1
Some users actually try to consume system resources by starting too many programs. This is
called forkbombing. These users are like script kiddies, but not as educated and without the sense
of self-preservation.

576

C h ap te r 1 9

Process Types
The CPU states line indicates what percentage of available CPU time the
system spends handling different types of processes. It shows five different
process types: user, nice, system, interrupt, and idle.
The user processes are average everyday programs—perhaps daemons
run by root, or commands run by regular users, or whatever. If it shows up in
ps -ax, it’s a user process.
The nice processes are user processes whose priority has been deliberately
manipulated. We’ll look at this in detail in “Reprioritizing with Niceness” on
page 584.
The system value gives the total percentage of CPU time spent by FreeBSD
running kernel processes and the userland processes in the kernel. These
include things such as virtual memory handling, networking, writing to disk,
debugging with INVARIANTS and WITNESS, and so on.
The interrupt value shows how much time the system spends handling
interrupt requests (IRQs).
Last, the idle entry shows how much time the system spends doing
nothing. If your CPU regularly has a very low idle time, you might want to
think about rescheduling jobs or getting a faster processor.

T O P AN D S M P
On an SMP system, top(1) displays the average use among all the processors. You
might have one processor completely tied up compiling something, but if the other
processor is idle, top(1) shows the CPU usage of only 50 percent.

Memory



The Mem line represents the usage of physical RAM. FreeBSD breaks memory
usage into several different categories.
Active memory is the total amount of memory in use by user processes.
When a program ends, the memory it had used is placed into inactive memory
and the data pulled from the disk is put into cache memory. If the system has
to run this program again, it can retrieve the software from memory instead
of disk.
Similarly, the Buf entry shows the size of the memory buffer. This buffer
contains most of the data recently read from the disk. The memory in Buf is
actually a subset of the active, inactive, and cache entries, not an entirely
separate category.
Free memory is totally unused. It might be memory that has never been
accessed, or it might be memory released by a process. This system has
1,607MB of free RAM. If you have a server that’s been up for months, and it
still has free memory, you might consider putting some of that RAM in a
machine that’s hurting for memory. This top(1) snapshot comes from my
laptop, which needs every bit of RAM it can get just on general principles.
Sy s te m Perf or ma nc e a n d M on it or in g

577

FreeBSD will shuffle memory between the inactive, cache, and free categories as needed to maintain a pool of available memory. Memory in the cache
is most easily transferred to the free pool. When cache memory gets low and
FreeBSD needs still more free memory, it picks pages from the inactive pool,
verifies that it can use them as free memory, and moves them to the free
pool. FreeBSD tries to keep the total number of free and cache pages above
the sysctl vm.v_free_target. (Remember, the page size is given by the sysctl
hw.pagesize which is 4,096 on i386 and amd64 systems.)
This means that having free memory does not mean that your system has
enough memory. If vmstat(8) shows that you are swapping at all, you are out
of memory. You might have a program that releases memory on a regular
basis. Also, FreeBSD will move some pages from cache to free in an effort to
maintain a certain level of free memory.
FreeBSD uses wired memory for in-kernel data structures, as well as for
system calls that must have a particular piece of memory immediately available.
Wired memory is never swapped or paged.

Swap



The Swap line gives the total swap available on the system and how much is
in use. Swapping is using the disk drive as additional memory. We’ll look at
swap in more detail later in the chapter.

Process List
Finally, top(1) lists the processes on the system and their basic characteristics.
The table format is designed to present as much information as possible in
as little space as possible. Every process has its own line.
PID First, we have the process ID number, or PID. Every running process has its own unique PID. When you use kill(1), specify the process by
its PID. (If you don’t know the PID of a process, you can use pkill(1) to
kill the process by its name.)
Username Next is the username of the user running the process. If
multiple processes consume large amounts of CPU or memory, and they
are all owned by the same user, you know who to talk to.
Priority and niceness The PRI (priority) and NICE columns are interrelated and indicate how much precedence the system gives each process.
We’ll talk about priority and niceness a little later in this chapter.
Size Size gives the amount of memory that the system has set aside for
this process.
Resident memory The RES column shows how much of a program is
actually in memory at the moment. A program might reserve a huge
amount of memory, but only use a small fraction of that reservation at
any time. For example, as I type this, my OpenOffice.org program has
allocated 169MB but is only using 103MB. (As an interesting side note,
X.org’s memory usage includes the memory on your video card, which is
separate from your main system memory.)
578

C h ap te r 1 9

State The STATE column shows what a process is doing at the moment.
A process can be in a variety of states—waiting for input, sleeping until
something wakes it, actively running, and so on. You can see the name of
the system call a process is waiting on, such as select, pause, or ttyin. On
an SMP system, when a process runs you’ll see the CPU it’s running on.
In our example output, acroread is running on CPU0.
Time The TIME column shows the total amount of CPU time the process
has consumed.
WCPU The weighted CPU usage shows the percentage of CPU time
that the process uses, adjusted for the process’s priority and niceness.
Command

Finally, we have the name of the program that’s running.

Looking at top(1)’s output gives you an idea of where the system is
spending its time.

top(1) and I/O
In addition to the standard CPU display, top(1) has an I/O mode that displays
which processes are using the disk most actively. While top(1) is running, hit M
to enter the I/O mode. The upper portion of the display still shows memory,
swap, and CPU status, but the lower portion changes considerably.
PID
3064
767
1082
1092
904
...

USERNAME
root
root
mwlucas
root
root

VCSW
89
0
2
0
0

IVCSW
0
0
1
0
0

READ
89
0
0
0
0

WRITE
0
0
0
0
0

FAULT
0
0
0
0
0

TOTAL PERCENT COMMAND
89 100.00% tcsh
0
0.00% nfsd
0
0.00% sshd
0
0.00% tcsh
0
0.00% sendmail

The PID is the process ID, of course, and the USERNAME column shows who
is running the process.
VCSW stands for voluntary context switches; this is the number of times this
process has surrendered the system to other processes. IVCSW means involuntary
context switches and shows how often the kernel has told the process, “You’re
done now, it’s time to let someone else run for a while.”
Similarly, READ and WRITE show how many times the system has read from
disk and written to disk. The FAULT column shows how often this process has
had to pull memory pages from swap, which makes for another sort of disk
read. These last three columns are aggregated in the TOTAL column.
The PERCENT column shows what percent of disk activity this process is
using. Unlike gstat(8), top(1) displays each process’s utilization as a percentage of the actual disk activity, rather than the possible disk activity. If you have
only one process accessing the disk, top(1) displays that process as using 100
percent of disk activity, even if it’s only sending a trickle of data. gstat(8) tells
you how busy the disk is, but top(1) tells you what’s generating that disk
activity and where to place the blame. Here we see that process ID 3064 is
generating all of our disk activity. It’s a tcsh(1) process, also known as “some
user’s shell.” Let’s track down the miscreant.
Sy s te m Perf or ma nc e a n d M on it or in g

579

M O RE T O P F EA T U R ES
top can alter its display in many ways. You can view processes for a particular user,
include or exclude kernel threads, exclude idle processes, and so on. Read the man
page for details.

Following Processes
On any Unix-like system, every userland process has a parent-child relationship
with other processes. When FreeBSD boots, it creates a single process by
starting init(8) and assigning it PID 1. This process starts other processes,
such as the /etc/rc startup script and the getty(8) program that handles your
login request. These processes are children of process ID 1. When you log in,
getty(8) starts a new shell for you, making your shell a child of the getty
process. Commands you run are either children of your shell process or part
of your shell. You can view these parent-child relationships with ps(1) using
the -ajx flags (among others).
# ps -ajx
USER
root
root
root
...
haldaemon
root
...

PID PPID PGID
0
0
0
1
0
1
2
0
0

SID JOBC STAT
0
0 WLs
1
0 ILs
0
0 DL

TT
??
??
??

TIME
0:00.01
0:00.01
0:00.79

826
827

826
826

??
??

0:02.80 /usr/local/sbin/hald
0:00.06 hald-runner

1
826

826
826

0 Ss
0 I

COMMAND
[swapper]
/sbin/init -[g_event]

At the far left we have the username of the process owner, then the PID
and parent PID (PPID) of the process. This is the most useful thing we see here,
but we’ll briefly cover the other fields.
The PGID is the process group ID number, which is normally inherited
from its parent process. A program can start a new process group, and that
new process group will have a PGID equal to the process ID. Process groups
are used for signal processing and job control. A session ID, or SID, is a grouping of PGIDs, usually started by a single user or daemon. Processes may not
migrate from one SID to another. JOBC gives the job control count, indicating
if the process is running under job control (i.e., in the background).
STAT shows the process state—exactly what the process is doing at the
moment you run ps(1). Process state is very useful as it tells you if a process is
idle, what it’s waiting for, and so on. I highly recommend reading the section
on process state from ps(1).
TT lists the process’s controlling terminal. This column shows only the end
of the terminal name, such as v0 for ttyv0 or p0 for ttyp0. The example shows
only processes without a controlling terminal, indicated by ??, but we’ll see
another example with terminals shortly.
580

C h ap te r 1 9

The TIME column shows how much processor time the process has used,
both in userland and in the kernel.
Finally, we see the COMMAND name, as it was called by the parent process.
Processes in square brackets are actually kernel threads, not real processes.
So, how can this help us track a questionable process? In our top(1) I/O
example, we saw that process 3064 was generating almost all of our disk activity. Run ps -ajx and look for this process:




...
root
chris
root
root
root
mwlucas
...

3035
2981
2989
2990
3064
2996

3034
2980
2981
2989
2990
2995

3035
2981
2989
2990
3064
2996

2969
2981
2981
2981
2981
2996

1
0
1
1
1
0

S+
Is
I
D
DV+
Is

p0
p1
p1
p1
p1
p2

0:00.03
0:00.03
0:00.01
0:00.05
0:00.15
0:00.02

_su -m (tcsh)
-tcsh (tcsh)
su -m
_su -m (tcsh)
_su -m (tcsh)
-tcsh (tcsh)



Our process of interest is owned by root and is a tcsh(1) instance, just
as top’s I/O mode said. It’s actually running under su(1), however, which
means that there’s a real person behind this performance problem. Process
3064 is a child of process 2990 , which is a child of process 2989 , both of
which are owned by root. Process 2989 is a child of 2981 , however, which is
a shell run by a real user. You might also note that these processes are all
parts of session 2981, showing that they’re probably all run in the same login
session. The TT column shows p1, which means that the user is logged in on
/dev/ttyp1, the second virtual terminal on this machine.
It’s normal for a system to experience brief periods of total utilization. If
nobody else is using the system and nobody’s complaining about performance,
why not let this user run his job? If this process is causing problems for other
users, however, we can either deprioritize it, use our root privileges to kill the
job, or show up at the user’s cubicle with a baseball bat.





Paging and Swapping
Using swap space isn’t bad in and of itself, as FreeBSD uses swap as virtual
memory. Swap space is much slower than chip memory, but it does work, and
many programs don’t need to have everything in RAM in order to run. A
typical program spends 80 percent of its time running 20 percent of its code.
Much of the rest of its code covers startup and shutdown, error handling,
and so on. You can safely let those bits go to swap space with minimal performance impact.
Swap caches data that it has handled. Once a process uses swap, that
swap remains in use until the process either exits or calls the memory back
from swap.
Swap usage occurs through paging and swapping. Paging is all right;
swapping is not so good, but it’s better than crashing.

Sy s te m Perf or ma nc e a n d M on it or in g

581

Paging
Paging occurs when FreeBSD moves a portion of a running program into
swap space. Paging can actually improve performance on a heavily loaded
system because unused bits can be stored on disk until they’re needed—if
ever. FreeBSD can then use the real memory for actual running code. Does it
really matter if your system puts your database startup code to swap once the
database is up and running?

Swapping
If the computer doesn’t have enough physical memory to store a process that
isn’t being run at that particular microsecond, the system can move the entire
process to swap. When the scheduler starts that process again, FreeBSD fetches
the entire process from swap and runs it, probably consigning some other
process to swap.
The problem with swapping is that disk usage goes through the roof and
performance drops dramatically. Since requests take longer to handle, there
are more requests on the system at any one time. Logging in to check the
problem only makes the situation worse, because your login is just one more
process. Some systems can handle certain amounts of swapping, while on
others the situation can quickly degenerate into a death spiral.
When your CPU is overloaded, the system is slow. When your disks are a
bottleneck, the system is slow. Memory shortages can actually crash your computer. If you’re swapping, you must buy more memory or resign yourself to
appalling performance.
vmstat(8) specifically shows the number of processes swapped out at any
one time.

Performance Tuning
FreeBSD caches recently accessed data in memory because a surprising
amount of information is read from the disk time and time again. If this
information is cached in physical memory, it can be accessed very quickly. If
the system needs more memory, it dumps the oldest cached chunks in favor
of new data.
For example, the top(1) output we’re discussing is from my laptop. On
it, I had started Firefox so I could check my comics.2 The disk had to work for
a moment or two to read in the program. I then shut the browser off so I could
focus on my work, but FreeBSD leaves Firefox in the system buffer cache.
If I start Firefox again, FreeBSD will just pull it from memory instead of troubling the disk, which dramatically reduces its startup time. Had I started
another large process, FreeBSD would have dumped the web browser from
the cache to read in the new program.

2
Sluggy Freelance (http://www.sluggy.com), Help Desk (http://www.ubersoft.net), and User Friendly
(http://www.userfriendly.org), if anyone cares. Fear the bunny.

582

C h ap te r 1 9

If your system is operating well, you will have at least a few megabytes
of free memory. The sysctls vm.v_free_target and hw.pagesize tell you how
much free memory FreeBSD thinks it needs on your system. If you consistently have more memory than these two sysctls multiplied, your system is not
being used to its full potential. For example, on my laptop I have:
# sysctl vm.v_free_target
vm.v_free_target: 13745
# sysctl hw.pagesize
hw.pagesize: 4096

My system wants to have at least 13,745 × 4,096 = 56,299,520 bytes, or
about 54MB, of free memory. I could lose a gigabyte of RAM from my laptop
without flinching, if it wasn’t for the fact that I suffer deep-seated emotional
trauma about insufficient RAM.

Memory Usage
If you have a lot of memory in cache or buffer, you don’t have a memory
shortage. You might make good use of more memory, but it isn’t strictly
necessary. If you have low free memory, but a lot of active and wired memory,
your system is devouring RAM. Adding memory would let you take advantage
of the buffer cache.
When you’re out of free space and have little or no memory in cache or
buffer, investigate your memory use further. You might well have a memory
shortage. Once you start to use swap, this memory shortage is no longer
hypothetical.

Swap Space Usage
Swap space helps briefly cover RAM shortages. For example, if you’re untarring a huge file you might easily consume all of your physical memory and
start using virtual memory. It’s not worth buying more RAM for such occasional tasks when swap suffices.
In short, swap space is like wine. A glass or two now and then won’t
hurt you and might even be a good choice. Hitting the bottle constantly is
a problem, however.

CPU Usage
A processor can only do so many things a second. If you run more tasks than
your CPU can handle, requests will start to back up, you’ll develop a processor
backlog, and the system will slow down. That’s CPU usage in a nutshell.
If performance is unacceptable and top(1) shows your CPU hovering
around 100 percent all the time, CPU utilization is probably your problem.
While new hardware is certainly an option, you do have other choices. For
example, investigate the processes running on your system to see if they’re
all necessary. Did some junior sysadmin install a SETI@Home client
Sy s te m Perf or ma nc e a n d M on it or in g

583

(/usr/ports/astro/setiathome) to hunt for aliens with your spare CPU cycles?
Is anything running that was important at one time, but not any longer?
Find and shut down those unnecessary processes, and make sure that they
won’t start the next time the system boots.
Once that’s done, evaluate your system performance again. If you still
have problems, try rescheduling or reprioritizing.

Rescheduling
Rescheduling is easier than reprioritizing; it is a relatively simple way to
balance system processes so that they don’t monopolize system resources.
As discussed in Chapter 15, you and your users can schedule programs to
run at specific times with cron(1). If you have users who are running massive
jobs at particular times, you might consider using cron(1) to run them in
off hours. Frequently, jobs such as the monthly billing database search can
run between 6 PM and 6 AM and nobody will care—finance just wants the
data on hand at 8 AM on the first day of the month, so they can close out
last month’s accounting. Similarly, you can schedule your make buildworld
&& make buildkernel at 1 AM.

Reprioritizing with Niceness
If rescheduling won’t work, you’re left with reprioritizing, which can be a little
trickier. When reprioritizing, you tell FreeBSD to change the importance of a
given process. For example, you can have a program run during busy hours,
but only when nothing else wants to run. You’ve just told that program to be
nice and step aside for other programs.
The nicer a process is, the less CPU time it demands. The default
niceness is 0, but niceness runs from 20 (very nice) to -20 (not nice at all).
This might seem backwards; you could argue that a higher number should
mean a higher priority. That would lead to a language problem, however;
calling this factor “selfishness” or “crankiness” instead of “niceness” didn’t
seem like a good idea at the time.3
top(1) displays a PRI column for process priority. FreeBSD calculates a
process’s priority from a variety of factors, including niceness, and runs highpriority processes first whenever possible. Niceness affects priority, but you
cannot directly edit priority.
If you know that your system is running at or near capacity, you can
choose to run a command with nice(1) to assign the process a niceness.
Specify niceness with nice -n and the nice value in front of the command.
For example, to start a very selfish make buildworld at nice 15, you would run:
# nice -n 15 make buildworld

3

584

C h ap te r 1 9

Besides, “selfish” and “cranky” are already taken by the system administrators themselves.

Only root can assign a negative niceness to a program, as in nice -n -5. For
example, if you have a critical kernel patch that must be applied urgently and
you want the compile to finish as quickly as possible, use a negative niceness:
# cd /usr/src
# nice -n -20 make kernel

N I CE V S . T C S H
The tcsh(1) shell has a nice command built in. That built-in nice uses the renice(8)
syntax, which is different from nice(1). I’m sure there’s a reason for that other than
annoying tcsh users, but that rationale escapes me at the moment. If you specifically
want to use nice(1), use the full path /usr/bin/nice.

Usually, you don’t have the luxury of telling a command to be nice when
you start it, but instead have to change its niceness when you learn that it’s
absorbing all of your system capacity. renice(8) reprioritizes running processes
by their process IDs or owners. To change the niceness of a process, run renice
with the new niceness and the PID as arguments.
For example, one of my systems is a logging host that also runs several
instances of softflowd(8), flow-capture, Nagios, and other critical network
awareness systems. I also have other assorted pieces of software running on
this system, such as a CVSup server for my internal hosts. If I find that intermittent load on the CVSup server is interfering with my Nagios-based network
monitoring or my syslogd(8) server, I must take action. Renicing the cvsupd(8)
server will make the clients run more slowly, but that’s better than slowing
down monitoring. Use pgrep(1) to find cvsupd(8)’s PID:
# pgrep cvsupd
993
# renice 10 993
993: old priority 0, new priority 10

Boom! FreeBSD now schedules the cvsupd process less often than other
processes. This greatly annoys the users of that service, but since I’m the main
user and I’m already annoyed that’s all right.
To renice every process owned by a user, use the -u flag. For example, to
make my processes more important than anyone else’s, I could do this:
# renice -5 -u mwlucas
1001: old priority 0, new priority -5

Sy s te m Perf or ma nc e a n d M on it or in g

585

The 1001 is my user ID on this system. Again, presumably I have a very
good reason for doing this, beyond my need for personal power.4 Similarly,
if that user who gobbled up all my disk I/O insists on being difficult, I could
make his processes very, very nice, which would probably solve other users’
complaints.

Investigating Software
We’ve spent a lot of time discussing tuning FreeBSD to solve performance
problems. That’s only logical, as this is a FreeBSD book. Don’t forget about
your software, however! You can frequently solve performance problems by
tweaking the software that’s causing the problems. For example, I have a set
of Perl CGI scripts that thrash disks and choke CPUs. A Google search showed
that these scripts behave much better running under mod_perl2. I changed
the web server configuration, and the problem went away. This is far, far
easier than changing the operating system configuration.

T H E B O T T L E N EC K S H U F F L E
Every system has bottlenecks. If you eliminate one bottleneck, performance will
increase until another bottleneck is hit. The system’s performance is bound by the
slowest component in the computer. For example, a web server is frequently
network-bound because the slowest part of the system is the Internet connection.
If you upgrade your 1.5Mb/s T1 to a 2.4Gb/s OC-48, the system will hand out its
sites as fast as its other components allow. The hypothetical “eliminating bottlenecks”
that management often demands is really a case of “eliminating bottlenecks that
interfere with your usual workload.”

Now that you can look at system problems, let’s learn how to hear what
the system is trying to tell you.

Status Mail
If you look in /etc/crontab, you’ll see that FreeBSD runs maintenance jobs
every day, week, and month through periodic(8). These jobs perform basic
system checks and notify the administrators of changes, items requiring attention, and potential security issues. The output of each scheduled job is mailed
daily to the root account on the local system. The simplest way to find out
what your system is doing is to read this mail; many very busy system administrators just like you have collaborated to make these messages useful. While
you might get a lot of these messages, with a little experience you’ll learn
how to skim the reports looking for critical or unusual changes only.
The configuration of the daily, weekly, and monthly reports is controlled
in periodic.conf, as discussed in Chapter 10.

4

586

C h ap te r 1 9

Being selfish doesn’t count as a good reason to renice -20 your processes. Or so I’ve been told.

You probably don’t want to log in as root on all of your servers every day
just to read email, so forward root’s mail from every server to a centralized
mailbox. Make this change in /etc/aliases, as discussed in Chapter 16.
The only place where I recommend disabling these jobs is on embedded systems, which should be managed and monitored through some other
means, such as your network monitoring system. On such a system, disable the
periodic(8) checks in /etc/crontab.
While these daily reports are useful, they don’t tell the whole story. Logs
give a much more complete picture.

Logging with syslogd
The FreeBSD logging system is terribly useful. Any Unix-like operating system
allows you to log almost anything at almost any level of detail. While you’ll
find default system logging hooks for the most common system resources,
you can choose a logging configuration that meets your needs. Almost all
programs integrate with the logging daemon, syslogd(8).
The syslog protocol works through messages. Programs send individual
messages, which the syslog daemon syslogd(8) catches and processes.
syslogd(8) handles each message according to its facility and priority level,
both of which client programs assign to messages. You must understand both
facilities and levels to manage system logs.

Facilities
A facility is a tag indicating the source of a log entry. This is an arbitrary label,
just a text string used to sort one program from another. In most cases, each
program that needs a unique log uses a unique facility. Many programs or
protocols have facilities dedicated to them—for example, FTP is such a
common protocol than syslogd(8) has a special facility just for it. syslogd also
supports a variety of generic facilities that you can assign to any program.
Here are the standard facilities and the types of information they’re
used for.
auth Public information about user authorization, such as when people
logged in or used su(1).
authpriv

Private information about user authorization, accessible only

to root.
console

Messages from the system process scheduler.

cron
daemon
ftp
kern
lpr
mail

Messages normally printed to the system console.
A catch-all for all system daemons without other explicit handlers.

Messages from FTP and TFTP servers.
Messages from the kernel.
Messages from the printing system.
Mail system messages.

Sy s te m Perf or ma nc e a n d M on it or in g

587

mark This facility puts an entry into the log every 20 minutes. This is
useful when combined with another log.
news
ntp

Messages from the Usenet News daemons.
Network Time Protocol messages.

security

Messages from security programs such as pfctl(8).

syslog Messages from the log system about the log system itself. Don’t
log when you log, however, as that just makes you dizzy.
user The catch-all message facility. If a userland program doesn’t specify
a logging facility, it uses this.
uucp Messages from the Unix-to-Unix Copy Protocol. This is a piece of
pre-Internet Unix history that you’ll probably never encounter.

These are provided for the system administrator.
Many programs have an option to set a logging facility; choose one of
these if at all possible. For example, you might tell your customer service
system to log to local0.

local0 through local7

While most programs have sensible defaults, it’s your job as the system
administrator to manage which programs log to which facility.

Levels
A log message’s level represents its relative importance. While programs send
all of their logging data to syslogd, most systems only record the important
stuff that syslogd receives and discard the rest. Of course, one person’s trivia
is another’s vital data, and that’s where levels come in.
The syslog protocol offers eight levels. Use these levels to tell syslogd
what to record and what to discard. The levels are, in order from most to
least important:
emerg System panic. Messages flash on every terminal. The computer is
basically hosed. You don’t even have to reboot—the system is doing it
for you.5

This is bad, but not an emergency. The system can continue to
function, but this error should be attended to immediately.

alert

crit Critical errors include things such as bad blocks on a hard drive or
serious software issues. You can continue to run as is, if you’re brave.
err These are errors that require attention at some point, but they won’t
destroy the system.
warning These are miscellaneous warnings that probably won’t stop the
program that issued them from working just as it always has.
notice This includes general information that probably doesn’t require
action on your part, such as daemon startup and shutdown.

5

588

C h ap te r 1 9

You might think this is funny now, but just wait until it happens to you.

info This includes program information, such as individual transactions
in a mail server.

This level is usually only of use to programmers and occasionally
to sysadmins who are trying to figure out why a program behaves as it
does. Debugging logs can contain whatever information the programmer
considered necessary to debug the code, which might include information
that violates user privacy.

debug

none This means, “Don’t log anything from this facility.” It’s most commonly used to exclude information from wildcard entries, as we’ll see
shortly.

By combining level with priority, you can categorize messages quite
narrowly and treat each one according to your needs.

Processing Messages with syslogd(8)
The syslogd(8) daemon catches messages from the network and compares
them to entries in /etc/syslog.conf. That file has two columns; the first describes
the log message, either by facility and level, or by program name. The second
tells syslogd(8) what to do when a log message matches the description. For
example, look at this entry from the default syslog.conf:
mail.info

/var/log/maillog

This tells syslogd(8) that when it receives a message from the mail
facility with a level of info or higher, the message should be appended to
/var/log/maillog.
Wildcards
You can also use wildcards as an information source. For example, this line
logs every message from the mail facility:
mail.*

/var/log/maillog

To log everything from everywhere, uncomment the all.log entry and
create the file /var/log/all.log :
*.*

/var/log/all.log

This works, but I find it too informative to be of any real use. You’ll find
yourself using complex grep(1) statements daisy-chained together to find
even the simplest information. Also, this would include all sorts of private
data.

Sy s te m Perf or ma nc e a n d M on it or in g

589

Excluding Information
Use the none level to exclude information from a log. For example, here we
exclude authpriv information from our all-inclusive log. The semicolon allows
you to combine entries on a single line:
*.*; authpriv.none

/var/log/most.log

Comparison
You can also use the comparison operators < (less than), = (equals), and >
(greater than) in syslog.conf rules. While syslogd defaults to recording all
messages at the specified level or above, you might want to include only a
range of levels. For example, you could log everything of info level and
above to the main log file while logging the rest to the debug file:
mail.info
mail.=debug

/var/log/maillog
/var/log/maillog.debug

The mail.info entry matches all log messages sent to the mail facility at
info level and above. The second line only matches the messages that have
a level of precisely debug. You can’t use a simple mail.debug, because the debugging log will then duplicate the content of the previous log. This way, you
don’t have to sort through debugging information for basic mail logs, and
you don’t have to sort through mail transmission information to get your
debugging output.
Local Facilities
Many programs offer to log via syslog. Most of these can be set to a facility of
your choice. The various local facilities are reserved for these programs. For
example, by default dhcpd(8) (see Chapter 15) logs to the facility local7. Here,
we catch these messages and send them to their own file:
local7.*

/var/log/dhcpd

If you run out of local facilities, you can use other facilities that the system
is not using. For example, I’ve once used the uucp facility on a busy log server
on a network that had no uucp services.
Logging by Program Name
If you’re out of facilities, you can use the program’s name as a matching term.
An entry for a name requires two lines: the first line contains the program
name with a leading exclamation mark and the second line sets up logging.
For example, FreeBSD uses this to log ppp(8) information:
!ppp
*.*

590

C h ap te r 1 9

/var/log/ppp.log

The first line specifies the program name and the second one uses
wildcards to tell syslogd(8) to append absolutely everything to a file.
The !programname syntax affects all lines after it, so you must put it last
in syslogd.conf.
Logging to User Sessions
When you log to a user, any messages that arrive appear on that user’s screen.
To log to a user session, list usernames separated by commas as the destination. To write a message to all users’ terminals, use an asterisk (*). For
example, the default syslog.conf includes this line:
*.emerg

*

This says that any message of emergency level will appear on all users’
terminals. Since these messages usually say “Goodbye” in one way or another,
that’s appropriate.
Sending Log Messages to Programs
To direct log messages to a program, use a pipe symbol (|):
mail.*

|/usr/local/bin/mailstats.pl

Logging to a Logging Host
My networks habitually have a single logging host that handles not only the
FreeBSD boxes but also Cisco routers and switches, other Unix boxes, and
any syslog-speaking appliances. This greatly reduces system maintenance and
saves disk space. Each log message includes the hostname, so you can easily
sort them out later.
Use the at symbol (@) to send messages to another host. For example, the
following line dumps everything your local syslog receives to the logging host
on my network:
*.*

@loghost.blackhelicopters.org

The syslog.conf on the destination host determines the final destination
for those messages.
On the logging host, you can separate logs by the host where the log
message originated. Use the plus (+) symbol and the hostname to indicate
that the rules that follow apply to this host:
+dhcpserver
local7.*
+ns1
local7.*

/var/log/dhcpd
/var/log/named

If you’re using this configuration, I recommend placing your generic
rules at the top of syslog.conf and per-host rules near the bottom.

Sy s te m Perf or ma nc e a n d M on it or in g

591

Logging Overlap
syslog doesn’t log on a first-match or last-match basis; instead, it logs according
to every matching rule. This means you can easily have one log message in
several different logs. Consider the following snippet of log configuration.
(The first line is taken from the default FreeBSD syslog.conf but is trimmed to
fit legibly on paper.)
*.notice;authpriv.none
local7.*

/var/log/messages
/var/log/dhcp

Every message of level notice or more is logged to /var/log/messages.
We have our DHCP server logging to /var/log/dhcp. This means that any
DHCP messages of notice level or above will be logged to both /var/log/
messages and /var/log/dhcpd. I don’t like this; I want my DHCP messages
only in /var/log/dhcpd. I can deliberately exclude DHCP messages from
/var/log/messages by using the none facility:
*.notice;authpriv.none;local7.none

/var/log/messages

My /var/log/messages syslog configuration frequently grows quite long as I
exclude every local facility from it, but that’s all right.

S P AC E S AN D T A B S
Traditional Unix-like operating systems require tabs between the columns in
syslog.conf, but FreeBSD permits you to use spaces. Be sure to use only tabs if
you share the same syslog.conf between different operating systems.

syslogd Customization
FreeBSD runs syslogd by default, and out of the box it can be used as a logging
host. You can customize how it works through the use of command-line flags.
You can specify flags either on the command line or in rc.conf as syslogd_flags.
Allowed Log Senders
You can specify exactly which hosts syslogd(8) accepts log messages from.
This can be useful so you don’t wind up accepting logs from random people
on the Internet. While sending you lots of logs could be used to fill your hard
drive as a preparation for an attack, it’s more likely to be the result of a
misconfiguration. Your log server should be protected by a firewall in any case.
Use the -a flag to specify either the IP addresses or the network of hosts that
can send you log messages, as these two (mutually exclusive) examples show:
syslogd_flags="-a 192.168.1.9"
syslogd_flags="-a 192.168.1.0/24"

592

C h ap te r 1 9

While syslogd(8) would also accept DNS hostsnames and domain names
for this restriction, DNS is really an unsuitable access control device.
You can entirely disable accepting messages from remote hosts by
specifying the -s flag, FreeBSD’s default. If you use -ss instead, syslogd(8)
also disables sending log messages to remote hosts. Using -ss removes
syslogd(8) from the list of network-aware processes that show up in sockstat(1)
and netstat(1). While this half-open UDP socket is harmless, some people
feel better if syslogd(8) doesn’t appear attached to the network at all.
Attach to a Single Address
syslogd(8) defaults to attaching to UDP port 514 on every IP address the
system has. Your jail server needs syslogd, but a jail machine can only run
daemons that bind to a single address. Use the -b flag to force syslogd(8) to
attach to a single IP:
syslogd_flags="-b 192.168.1.1"

Additional Log Sockets
syslogd(8) can accept log messages via Unix domain sockets as well as over
the network. The standard location for this is /var/run/log. No chrooted
processes on your system can access this location, however. If you want those
chrooted processes to run, you must either configure them to log over the
network or provide an additional logging socket for them. Use the -l flag for
this and specify the full path to the additional logging socket:
syslogd_flags="-l /var/named/var/run/log"

The named(8) and ntpd(8) programs come with FreeBSD and are
commonly chrooted. The /etc/rc.d/syslogd is smart enough to add the
appropriate syslogd sockets if you chroot these programs through rc.conf.
Verbose Logging
Logging with verbose mode (-v) prints the numeric facility and level of each
message written in the local log. Using doubly verbose logging prints the name
of the facility and level instead of the number:
syslogd_flags="-vv"

These are the flags I use most commonly. Read syslogd(8) for the
complete list of options.

Log File Management
Log files grow, and you must decide how large they can grow before you trim
them. The standard way to do this is through log rotation. When logs are
rotated, the oldest log is deleted, the current log file is closed up and given a
new name, and a new log file is created for new data. FreeBSD includes a
Sy s te m Perf or ma nc e a n d M on it or in g

593

basic log file processor, newsyslog(8), which also compresses files, restarts
daemons, and in general handles all the routine tasks of log file shuffling.
cron(1) runs newsyslog(8) once per hour.
When newsyslog(8) runs, it reads /etc/newsyslog.conf and examines each
log file listed therein. If the conditions for rotating the log are met, the log is
rotated and other actions are taken as appropriate. /etc/newsyslog.conf uses
one line per log file; each line has seven fields, like this:
/var/log/ppp.log

root:network

640

3

100

*

JC

Let’s examine each field in turn.

Log File Path
The first entry on each line (/var/log/ppp.log in the example) is the full path
to the log file to be processed.

Owner and Group
The second field (root:network in our example) lists the rotated file’s owner
and group, separated by a colon. This field is optional and is not present in
many of the standard entries.
newsyslog(8) can change the owner and group of old log files. By default,
log files are owned by the root user and the wheel group. While it’s not
common to change the owner, you might need this ability on multi-user
machines.
You can also choose to change only the owner, or only the group. In
these cases, you use a colon with a name on only one side of it. For example,
:www changes the group to www, while mwlucas: gives me ownership of the file.

Permissions
The third field (640 in our example) gives the permissions mode in standard
Unix three-digit notation.

Count
This field specifies the oldest rotated log file that newsyslog(8) should keep.
newsyslog(8) numbers archived logs from newest to oldest, starting with the
newest as log 0. For example, with the default count of 5 for /var/log/messages,
you’ll find the following message logs:
messages
messages.0.bz
messages.1.bz
messages.2.bz
messages.3.bz
messages.4.bz
messages.5.bz

594

C h ap te r 1 9

Those of you who can count will recognize that this makes six archives,
not five, plus the current log file, for a week of logs. As a rule, it’s better to
have too many logs than too few; however, if you’re tight on disk space,
deleting an extra log or two might buy you time.

Size
The fifth field (100 in our example) is the file size in kilobytes. When
newsyslog(8) runs, it compares the size listed here with the size of the file. If
the file is larger than the size given here, newsyslog(8) rotates the file. If you
don’t want the file size to affect when the file is rotated, put an asterisk here.

Time
So far this seems easy, right? The sixth field, rotation time, changes that. The
time field has four different legitimate types of value: an asterisk, a number,
and two different date formats.
If you rotate based on log size rather than age, put an asterisk here.
If you put a plain naked number in this field, newsyslog(8) rotates the
log after that many hours have passed. For example, if you want the log to
rotate every 24 hours but don’t care about the exact time when that happens,
put 24 here.
The date formats are a little more complicated.
ISO 8601 Time Format
Any entry beginning with an @ symbol is in the restricted ISO 8601 time
format. This is a standard used by newsyslog(8) on most Unix-like systems;
it was the time format used in MIT’s primordial newsyslog(8). Restricted
ISO 8601 is a bit obtuse, but every Unix-like operating system supports it.
A full date in the restricted ISO 8601 format is 14 digits with a T in the
middle. The first four digits are the year, the next two the month, the next
two the day of the month. The T is inserted in the middle as a sort of decimal
point, separating whole days from fractions of a day. The next two digits are
hours, the next two minutes, the last two seconds. For example, the date of
March 2, 2008, 9:15 and 8 seconds PM is expressed in restricted ISO 8601 as
20080302T211508.
While complete dates in restricted ISO 8601 are fairly straightforward,
confusion arises when you don’t list the entire date. You can choose to specify
only fields near the T, leaving fields further away as blank. Blank fields are
wildcards. For example, 1T matches the first day of every month. 4T00 matches
midnight of the fourth day of every month. T23 matches the twenty-third
hour, or 11 PM, of every day. With a newsyslog.conf time of @T23, the log rotates
every day at 11 PM.
As with cron(1), you must specify time units in detail. For example,
@7T, the seventh day of the month, rotates the log once an hour, every hour,
on the seventh day of the month. After all, it matches all day long! A time of

Sy s te m Perf or ma nc e a n d M on it or in g

595

@7T01 would rotate the log at 1 AM on the seventh day of the month, which is

probably more desirable. You don’t need more detail than an hour, however,
as newsyslog(8) only runs once an hour.
FreeBSD-Specific Time
The restricted ISO 8601 time system doesn’t allow you to easily designate
weekly jobs, and it’s impossible to specify the last day of the month. That’s why
FreeBSD includes a time format that lets you easily perform these common
tasks. Any entry with a leading cash sign ($) is written in the FreeBSD-specific
month week day format.
This format uses three identifiers: M (day of month), W (day of week), and
H (hour of day). Each identifier is followed by a number indicating a particular
time. Hours range from 0 to 23, while days run from 0 (Sunday) to 6 (Saturday).
Days of the month start at 1 and go up, with L representing the last day of the
month. For example, to rotate a log on the fifth of each month at noon I
could use $M5H12. To start the month-end log accounting at 10 PM on the last
day of the month, use $MLH22.

ROTATING ON SIZE AND TIME
You can rotate logs at a given time, when they reach a certain size, or both. If you
specify both size and time, the log rotates whenever either condition is met.

Flags
The flags field dictates any special actions to be taken when the log is rotated.
This most commonly tells newsyslog(8) how to compress the log file, but you
can also signal processes when their log is rotated out from under them.
Log File Format and Compression
Logs can be either text or binary files.
Binary files can only be written to in a very specific manner. newsyslog(8)
starts each new log with a “logfile turned over” message, but adding this text
to a binary file would damage it. The B flag tells newsyslog(8) that this is a
binary file and that it doesn’t need this header.
Other log files are written in plain old ASCII text, and newsyslog(8) can
and should add a timestamped message to the top of the file indicating when
the log was rotated. Additionally, compressing old log files can save considerable space. The J flag tells newsyslog(8) to compress archives with bzip(1),
while the Z flag specifies gzip compression.
Special Log File Handling
newsyslog(8) can perform a few special tasks when it creates and rotates
log files. Here are the most common; you can read about the others in
newsyslog.conf(5).
596

C h ap te r 1 9

If you don’t want to back up certain logs, use the D flag to set the NODUMP
flag on newly created log files.
Perhaps you have many similar log files that you want to treat identically.
The G flag tells newsyslog that the log file name at the beginning of the line is
actually a shell glob, and that all log files that match the expression are to be
rotated in this manner. To learn about regular expressions, read regexp(3).
Bring aspirin.
You might want newsyslog to create a file if it doesn’t exist. Use the C flag
for this. The syslogd program will not log to a nonexistent file.
The N flag explicitly tells newsyslog to not send a signal when rotating
this log.
Finally, use a hyphen (-) as a placeholder when you don’t need any of
these flags. However, you will need one of the newsyslog.conf flags further
down the line, such as when specifying the path to a pidfile.

Pidfile
The next field is a pidfile path (not shown in our example, but look at
/etc/newsyslog.conf for a couple of samples). A pidfile records a program’s
process ID so that other programs can easily view it. If you list the full path to
a pidfile, newsyslog(8) sends a kill -HUP to that program when it rotates the
log. This signals the process to close its logfiles and restart itself. Not all
processes have pidfiles, and not all programs need this sort of special care
when rotating their logs.

Signal
Most programs perform logfile rotation on a SIGHUP, but some programs need a
specific signal when their logs are rotated. You can list the exact signal necessary in the last field, after the pidfile.

Sample newsyslog.conf Entry
Let’s slap all this together into a worst-case, you-have-got -to-be-kidding
example. A database log file needs rotation at 11 PM on the last day of the
month. The database documentation says that you must send the server an
interrupt signal (SIGINT , or signal number 2) on rotation. You want the
archived logs to be owned by the user dbadmin and only viewable by that
user. You need six months of logs. What’s more, the logs are binary files.
Your newsyslog.conf line would look like this:
/var/log/database

dbadmin:

600

6

*

$MLH23

B

/var/run/db.pid

2

This is a deliberately ugly example; in most cases you just slap in the
filename and the rotation condition and you’re done.

Sy s te m Perf or ma nc e a n d M on it or in g

597

FreeBSD and SNMP
Emailed reports are nice but general, and logs are difficult to analyze for
long-term trends. The industry standard for network, server, and service
management is Simple Network Management Protocol, or SNMP. Many different
vendors support SNMP as a protocol for gathering information from many
different devices across the network. FreeBSD includes an SNMP agent,
bsnmpd(8), that not only provides standard SNMP functions but also gives
visibility to FreeBSD-specific features.
bsnmpd (short for Begemot SNMPD) is a minimalist SNMP agent specifically
designed to be extensible. All actual functionality is provided via external
modules. FreeBSD includes the bsnmpd modules for standard network
SNMP functions and modules for specific FreeBSD features, such as PF and
netgraph(4). bsnmpd does not try to be all things to all people, but rather
offers a foundation where everyone can build an SNMP implementation that
does only what they need, no more and no less.

SNMP 101
SNMP works on a classic client-server model. The SNMP client, or agent,
sends a request across the network to an SNMP server. The SNMP server,
bsnmpd, gathers information from the local system and returns it to the
client.
An SNMP agent can also send a request to make changes to the SNMP
server. If the system is properly (or improperly, depending on your point of
view) configured, you can issue commands via SNMP. This “write” configuration is most commonly used in routers, switches, and other embedded
network devices. Most Unix-like operating systems have a command-line
management system and don’t usually accept instruction via SNMP. Writing
system configuration or issuing commands via SNMP requires careful setup
and raises all sorts of security issues; it’s an excellent topic for an entire book.
No system administrator I know is comfortable managing their system via
SNMP. With all of this in mind, we’re going to focus specifically on readonly SNMP.
In addition to having an SNMP server answer requests from an SNMP
client, the client can transmit SNMP traps to a trap receiver elsewhere on the
network. An SNMP agent generates these traps in response to particular
events on the server. SNMP traps are much like syslogd(8) messages, except
that they follow the very specific format required by SNMP. FreeBSD does
not include an SNMP trap receiver at this time; if you need one, check out
snmptrapd(8) from net-snmp (/usr/ports/net-mgmt/net-snmp).
SNMP MIBs
SNMP manages information via a Management Information Base, or MIB, a treelike structure containing hierarchical information in ASN.1 format. We’ve
seen an example of a MIB tree before: the sysctl(8) interface discussed in
Chapter 5.
598

C h ap te r 1 9

Each SNMP server has a list of information it can extract from the local
computer. The server arranges these bits of information into a hierarchical
tree. Each SNMP MIB tree has very general main categories: network, physical,
programs, and so on, with more specific subdivisions in each. Think of the
tree as a well-organized filing cabinet, where individual drawers hold specific
information and files within drawers hold particular facts. Similarly, the uppermost MIB contains a list of MIBs beneath it.
MIBs can be referred to by name or by number. For example, here’s a
MIB pulled off a sample system:
interfaces.ifTable.ifEntry.ifDescr.1 = STRING: "fxp0"

The first term in this MIB, interfaces, shows us that we’re looking at this
machine’s network interfaces. If this machine had no interfaces, this first
category would not even exist. The ifTable is the interface table, or a list of all
the interfaces on the system. ifEntry shows one particular interface, and ifDescr
means that we’re looking at a description of this interface. This MIB can be
summarized as, “Interface number 1 on this machine is called fxp0.”
MIBs can be expressed as numbers, and most SNMP tools do their work
natively in numerical MIBs. Most people prefer words, but your poor brain
must be capable of working with either. A MIB browser can translate between
the numerical and word forms of an SNMP MIB for you, or you could install
/usr/ports/net-mgmt/net-snmp and use snmptranslate(1), but for now just trust
me. The preceding example can be translated to:
.1.3.6.1.2.1.2.2.1.2.1

Expressed in words, this MIB has 5 terms separated by dots. Expressed
in numbers, the MIB has 11 parts. That doesn’t look quite right if they’re
supposed to be the same thing. What gives?
The numerical MIB is longer because it includes the default .1.3.6.1.2.1,
which means .iso.org.dod.internet.mgmt.mib-2. This is the standard subset of
MIBs used on the Internet. The vast majority of SNMP MIBs (but not all)
have this leading string in front of them, so nobody bothers writing it down
any more.
If you’re in one of those difficult moods, you can even mix words and
numbers:
.1.org.6.1.mgmt.1.interfaces.ifTable.1.2.1

At this point, international treaties permit your co-workers to drive you
from the building with pitchforks and flaming torches. Pick one method of
expressing MIBs and stick to it.
MIB Definitions and MIB Browsers
MIBs are defined according to a very strict syntax and are documented in MIB
files. Every SNMP agent has its own MIB files; bsnmpd’s are in /usr/share/snmp.
These files are very formal plaintext. While you can read and interpret them
Sy s te m Perf or ma nc e a n d M on it or in g

599

with nothing more than your brain, I highly recommend copying them to a
workstation and installing a MIB browser so that you can comprehend them
more easily.
MIB browsers interpret MIB files and present them in their full tree-like
glory, complete with definitions of each part of the tree and descriptions of
each individual MIB. Generally speaking, a MIB browser lets you enter a
particular MIB and displays both the numerical and word definitions of that
MIB, along with querying an SNMP agent for the status of that MIB.
If you have a FreeBSD (or lesser Unix-like) workstation, use mbrowse
(/usr/ports/net-mgmt/mbrowse) for MIB browsing. Windows systems have many
options, but Getif (http://www.wtcs.org) is a decent no-cost choice. If you
don’t want to use a graphical interface for SNMP work, check out net-snmp
(/usr/ports/net-mgmt/net-snmp) for a full assortment of command-line SNMP
client tools.
SNMP Security
Many security experts state that SNMP really stands for “Security: Not My
Problem!” This is rather unkind but very true. SNMP needs to be used only
behind firewalls on trusted networks. If you must use SNMP on the naked
Internet, use packet filtering to keep the public from querying your SNMP
service. SNMP agents run on UDP port 161.
SNMP provides basic security through communities. If you go looking
around you’ll find all sorts of explanations why a community is not the same
thing as a password, but a community is a password. Most SNMP agents have
two communities by default: public (read-only access) and private (read-write
access). Yes, there’s a default that provides read-write access. Your first task
whenever you provision an SNMP agent on any host, on any OS, is to disable
those default community names and replace them with ones that haven’t
been widely documented for decades.
SNMP comes in different versions. Version 1 was the first attempt, and
version 2c is the modern standard. You’ll see references to SNMP version 3,
which uses advanced encryption to protect data on the wire. Very few vendors
actually implement SNMPv3, however. FreeBSD’s bsnmpd(8) uses SNMPv2c.
This means that anyone with a packet sniffer can capture your SNMP community name, so be absolutely certain you’re only using SNMP on a private
network. Making SNMP queries over an untrusted network is a great way to
have strangers poking at your system management.

Configuring bsnmpd
Before you can use SNMP to monitor your system, you must configure the
SNMP daemon. Configure bsnmpd(8) in /etc/snmpd.config. In addition to
including the default communities of public and private, the default configuration does not enable any of the FreeBSD-specific features that make
bsnmpd(8) desirable.

600

C h ap te r 1 9

bsnmpd Variables
bsnmpd uses variables to assign values to configuration statements. Most
high-visibility variables are set at the top of the configuration file, as you’ll
see here:
location := "Room 200"
contact := "sysmeister@example.com"
system := 1
# FreeBSD
traphost := localhost
trapport := 162

These top variables define values for MIBs that should be set on every
SNMP agent. The location describes the physical location of the machine.
Every system needs a legitimate email contact. bsnmpd(8) runs on operating
systems other than FreeBSD, so you have the option of setting a particular
operating system here. Lastly, if you have a trap host, you can set the server
name and port here.
Further down the file, you can set the SNMP community names:
# Change this!
read := "public"
# Uncomment begemotSnmpdCommunityString.0.2 below that sets the community
# string to enable write access.
write := "geheim"
trap := "mytrap"

The read string defines the read-only community of this SNMP agent.
The default configuration file advises you to change it. Take that advice. The
write string is the read-write community name, which is disabled by default
further down in the configuration file. You can also set the community name
for SNMP traps sent by this agent.
With only this configuration, bsnmpd(8) will start, run, and provide basic
SNMP data for your network management system. Just set bsnmpd_enable="YES"
in /etc/rc.conf to start bsnmpd at boot. You won’t get any special FreeBSD
functionality, however. Let’s go on and see how to manage this.
Detailed bsnmpd Configuration
bsnmpd(8) uses the variables you set at the top of the configuration file to
assign values to different MIBs later in the configuration. For example, at the
top of the file you set the variable read to public. Later in the configuration
file, you’ll find this statement:
begemotSnmpdCommunityString.0.1 = $(read)

This sets the MIB begemotSnmpdCommunityString.0.1 equal to the value of
the read variable.

Sy s te m Perf or ma nc e a n d M on it or in g

601

Why not just set these values directly? bsnmpd(8) is specifically designed
to be extensible and configurable. Setting a few variables at the top of the file
is much easier than directly editing the rules further down the file.
Let’s go back to this begemotSnmpdCommunityString MIB set here. Why are
we setting this? Search for the string in your MIB browser, and you’ll see that
this is the MIB that defines an SNMP community name. You probably could
have guessed that from the assignment of the read variable, but it’s nice to
confirm that.
Similarly, you’ll find an entry like this:
begemotSnmpdPortStatus.0.0.0.0.161 = 1

Checking the MIB browser shows that this dictates the IP address and the
UDP port that bsnmpd(8) binds to (in this case, all available addresses, on
port 161). All MIB configuration is done in this manner.
Loading bsnmpd Modules
Most of bsnmpd’s interesting features are configured through modules.
Enable modules in the configuration file by giving the begemotSnmpdModulePath
MIB a class that the module handles and the full path to the shared library
that implements support for that feature. For example, in the default configuration you’ll see a commented-out entry for the PF bsnmpd(8) module:
begemotSnmpdModulePath."pf"

= "/usr/lib/snmp_pf.so"

This enables support for PF MIBs. Your network management software
will be able to see directly into PF when you enable this, letting you track
everything from dropped packets to the size of the state table.
As of this writing, FreeBSD’s bsnmpd(8) ships with the following
FreeBSD-specific modules. All of these modules are disabled by default,
but you can enable them by just uncommenting their configuration file
entries.
Provides visibility into all Netgraph-based network features,
documented in snmp_netgraph(3).

Netgraph
PF

Provides visibility into the PF packet filter.

Hostres

Implements the Host Resources SNMP MIB, snmp_hostres(3).

bridge Provides visibility into bridging functions, documented in
snmp_bridge(3).

Restart bsnmpd(8) after enabling any of these in the configuration file.
If the program won’t start, check /var/log/messages for errors.
With bsnmpd(8), syslogd(8), status emails, and a wide variety of
performance analysis tools, you can make your FreeBSD system the bestmonitored device on the network. Now that you can see everything your
system offers, grab a flashlight as we explore a few of FreeBSD’s darker
corners.
602

C h ap te r 1 9

20
THE FRINGE OF FREEBSD

If you hang around the FreeBSD community for any length of time, you’ll hear
mention of all sorts of things that can be
done if you know how. People build embedded
FreeBSD devices and ship them to customers all over
the world, who don’t even know that they have a
Unix-like server inside the little box running their air conditioner or radio
relay station. People run FreeBSD on machines without hard drives, supporting hundreds or thousands of diskless workstations from a single server.
You’ll find bootable CDs and USB devices that contain complete FreeBSD
systems, including all the installed software you could ever want. These
things aren’t difficult to do, once you know the tricks.
In this chapter, we’re headed into the fringes of FreeBSD—the really
cool things that are done by FreeBSD users, but aren’t necessarily supported
by the mainstream FreeBSD Project. While you can find support and assistance
through the usual channels, you must be prepared to debug and troubleshoot
everything in this chapter even more than usual.

/etc/ttys
The file /etc/ttys controls how and where users may log into your FreeBSD
system. While you can set per-user restrictions on account logins elsewhere,
the /etc/ttys file controls which devices are available for logins and how they
can be used. While most people use only the console and SSH connections
that FreeBSD ships with, you can configure other devices to allow logins.
Remember, tty is short for teletype. Once upon a time, the main output
interface for a Unix-like system was a teletype. Anything you can log in on is
considered a tty. FreeBSD supports many different types of terminal devices,
from old-fashioned serial terminals to the usual keyboard/video/mouse
console to SSH and telnet sessions. FreeBSD systems offer four standard
login devices, or terminals: the console, virtual terminals, dial-up terminals,
and pseudoterminals.
The console is the only device available in single-user mode. On most
FreeBSD systems, this is either a video console that includes the monitor and
keyboard, or a serial console accessed from another system. Once the system
hits multi-user mode, the console is usually attached to a virtual terminal
instead. The console device is /dev/console.
A virtual terminal is attached to the physical monitor and keyboard.
If you’re not using X, you can have multiple terminals on your one physical
terminal. Switch between them with ALT and the function keys. For example,
the next time you’re at a console, hit ALT-F2. You’ll see a fresh login screen,
with ttyv1 after the host name. This is the second virtual terminal. Hitting
ALT-F1 takes you back to the main virtual terminal. By default, FreeBSD has
eight virtual terminals and reserves a ninth for X Windows. You can use
the eight virtual text terminals even when you’re in X, and some X desktops
provide multiple X virtual terminals. The virtual terminals are the /dev/ttyv
devices.
A dial-up terminal is connected via serial line. You can attach modems
directly to your serial ports and let users dial into your server. This is not so
common these days, but the same functionality supports logging in over a
serial console. Dial-up terminals are the /dev/ttyd devices.
Finally, a pseudoterminal is implemented entirely in software. When you
SSH into your server, you don’t need any actual hardware, but the software
still needs a device node for your login session. Pseudoterminals are the
/dev/ttyp devices.
Configure access to terminal devices in /etc/ttys. Most terminal devices
don’t require configuration—after all, if you can get a pseudoterminal it’s
because you’ve been authenticated and authorized to connect through
SSH, telnet, or some other network protocol. The useful things you can do
here are setting the console to require a password and enabling logins over
a serial console.

/etc/ttys Format
A typical entry in /etc/ttys looks like this:
ttyv0

604

C h ap te r 2 0

"/usr/libexec/getty Pc"

cons25

on

secure

The first entry is the console device. In this case, ttyv0 is the first virtual
terminal on the system.
The second field is the program that is spawned to process login requests
on this terminal. FreeBSD uses getty(8) for this on everything except pseudoterminals. Pseudoterminals process their login requests through whatever
daemon the user logs in through.
The third entry is the terminal type. You’ve probably heard of the vt100
terminal, or even the Sun terminal. On FreeBSD, the monitor uses a type of
cons25, which is a 25-character by 80-character screen. Pseudoterminals use
the terminal type of network; their features are determined by the server
daemon and the client software.
The fourth entry determines if the terminal is available for logins or not.
This is either on for accepting logins or off for not allowing them. Pseudoterminals are activated upon demand.
This example also includes the keyword secure, which tells getty(8) that
root may log in to this console.
Offering terminals is a low-level system task handled directly by init(8).
Changes to /etc/ttys do not take effect until you restart init(8). init is always
PID 1.
# kill -1 1

Insecure Console
When you boot FreeBSD in single-user mode, you get a root command
prompt. This is fine for your laptop and works nicely for servers in your
corporate datacenter, but what about machines in untrusted facilities? If
you have a server in a co-location center, for example, you probably don’t
want just anyone to be able to get root-level access to a machine. You can
tell FreeBSD that the physical console is insecure and make it require the
root password to enter single-user mode. The system will then boot from
power-on to multi-user mode without requiring a password, but will require
the password when you explicitly boot in single-user mode.
Requiring a password in single-user mode does not completely protect
your data, but it does raise the bar considerably. A lone tech working late,
when nobody’s looking, could boot your system into single-user mode and
add an account for himself in only fifteen minutes or so. Dismantling your
machine, removing the hard drives, mounting them into another machine,
making changes, and bringing your server back online requires much more
time, is far more intrusive, and is much more likely to be noticed by co-location
management.
Find the console entry in /etc/ttys:
console none

unknown off secure

T he Fr in g e of F ree BSD

605

You’ll see that the console terminal isn’t as full-featured as other terminals;
it doesn’t run getty(8) and uses the generic unknown terminal type. The console
is intended for use only in single-user mode or when attached to another
terminal, however, so that’s fine.
To make the console require a root login when booted into single-user
mode, change the secure to insecure.
Serial Logins
We talked about serial consoles back in Chapter 3. In addition to providing
console access, FreeBSD lets you log in through a serial port. In many Unixlike systems in the 1970s and 1980s, this was the only way to get a login prompt.
While it’s fallen into disfavor today thanks to the ubiquitous network, a serial
login is very useful on a system where you already have a serial console or
where the system does not support a video console. Systems without the
standard video monitor and keyboard are called headless systems.
To enable logging in on a serial line, find the /etc/ttys entry for your serial
port. The default configuration lists four dial-up terminals, ttyd0 through
ttyd3. These correspond to serial ports sio0 through sio3, or COM1 through
COM4. Let’s assume you have a serial console attached to the first serial port.
ttyd0

"/usr/libexec/getty std.9600"

dialup

off secure

This is pretty easy to configure. The port is off . Change it to on.
Serial logins are slower than network logins. They’re limited by the speed
of the serial port. This can be a problem when you have a serial console
and a serial login. The console prints debugging messages to your serial login
session, which takes up bandwidth on the serial port. If you have a lot of log
messages arriving quickly, they can prevent you from using the serial console.
Even so, a serial login is invaluable for headless servers and embedded devices,
as we’ll see later in this chapter.

Diskless FreeBSD
While FreeBSD isn’t difficult to manage, dozens or hundreds of nearly identical systems can become quite a burden. One way to reduce your maintenance
overhead is to use diskless systems. Diskless systems are not forbidden to have
hard drives; rather, they load their kernel and operating system from an NFS
server elsewhere on the network.
Why use a diskless system for your server farm? Multiple systems can boot
off of a single NFS server, centralizing all patch and package management.
This is excellent for collections of terminals, computation clusters, and other
environments where you have large numbers of identical systems. Rolling
out an operating system update becomes a simple matter of replacing files
on the NFS server. Similarly, when you discover that an update has problems,
reverting it is as simple as restoring files on the NFS server. In either case, the
only thing you have to do at the client side is reboot. As the clients have

606

C h ap te r 2 0

read-only access to the server, untrusted users cannot make any changes to
their local systems that couldn’t be undone by rebooting. If you have only a
couple of systems, running diskless might be too much work for you, but any
more than that and diskless is a clear winner.
Before you can run diskless systems, you must have an NFS server, a DHCP
server, a TFTP server, and hardware that supports diskless booting. Let’s go
through each and see how to set it up.

T E S T , T ES T , T ES T !
Your first diskless setup will be much like your first firewall setup: error-prone, troublesome, and infuriating. I strongly suggest that you test each step of the preparation so
that you can find and fix problems more easily. Test instructions are provided for
each required service.

Diskless Clients
Machines that run diskless must have enough smarts to find their boot loader
and operating system over the network. There are two standard ways of doing
this: BOOTP and PXE. BOOTP, the Internet Bootstrap Protocol, is an older
standard that fell out of favor long ago. PXE, Intel’s Preboot Execution Environment, has been supported on almost every new machine for years now, so
we’ll concentrate on that.
Boot your diskless client machine and go into the BIOS setup. Somewhere
in the BIOS you will find an option to set the boot device order. If the machine
supports PXE, one of those options will be the network. Enable that option
and have the machine try it first.
Your diskless client is ready. Now let’s get the server ready.

DHCP Server Setup
While most people think of DHCP as a way to assign IP addresses to clients,
it can provide much more than that. You can configure your DHCP server to
provide the locations of a TFTP server, an NFS server, and other network
resources. Diskless systems make extensive use of DHCP, and you’ll find that
we use DHCP options you never tried before. Configuring the ISC DHCP
server to handle diskless systems is pretty straightforward, once you have the
MAC address of your diskless workstation.
MAC Address
To assign configuration information to a DHCP client, you need the MAC
address of that client’s network card. Some BIOS implementations provide
the MAC addresses of integrated network cards, and some server-grade hardware has labels with the MAC address printed on them. Those options, however, are too easy, so we’ll try the hard way.

T he Fr in g e of F ree BSD

607

When a machine tries to boot off the network, it makes a DHCP request
for its configuration information. While you don’t have a diskless configuration yet, any DHCP server logs the MAC address of clients. For example,
when I boot my diskless client, my DHCP log shows:
Jul 27 10:15:49 sardines dhcpd: DHCPDISCOVER from 00:00:24:c1:cb:a4 via fxp0
Jul 27 10:15:49 sardines dhcpd: DHCPOFFER on 192.168.1.78 to 00:00:24:c1:cb:a4 via fxp0

This client has a MAC address of 00:00:24:c1:cb:a4 and has been offered
IP address 192.168.1.78. Given this information, we can create a DHCP configuration to assign this host a static IP address and provide its boot information.
DHCP Configuration: Specific Diskless Hosts
We configured basic DHCP services in Chapter 15. Here’s a sample dhcpd(8)
configuration for a diskless client. This does not go inside a subnet statement,
but is a top-level statement on its own.







group diskless {
next-server 192.168.1.1;
filename "pxeboot";
option root-path "192.168.1.1:/var/diskless/1/";
host diskless1.blackhelicopters.org {
hardware ethernet 00:00:24:c1:cb:a4 ;
fixed-address 192.168.1.99 ;
}
}

We define a group called diskless . The group will allow us to assign
certain parameters to the group and then just add hosts to the group. Every
host in the group gets those same parameters.
The next-server setting tells the DHCP clients the IP address of a
TFTP server, and the filename option tells clients the name of the boot
loader file to request from that TFTP server. Remember from Chapter 3
that the boot loader is the software that finds and loads the kernel. Finally,
option root-path
tells the boot loader where to find the root directory for
this machine. All of these options and settings are given to all clients in the
diskless group.
We then assign our diskless client to the diskless group using the host statement and the hostname of this system . Our first client is called diskless1.
This client is identified by its MAC address
and is assigned a static IP .
It also receives the standard configuration for this group.
Create additional host entries just like this for every diskless host on your
network.
Restart dhcpd(8) to make this configuration take effect. Now reboot
your diskless client. The DHCP log should show that you have offered this
client its static address. However, the DHCP client cannot boot any further
without a boot loader, which means you need a TFTP server.







608

C h ap te r 2 0





DHCP Configuration: Diskless Farms
Perhaps you have a large number of identical diskless hosts, such as thin clients
in a terminal room. It’s perfectly sensible to not want to make a static DHCP
entry for each thin client. Let these hosts get their boot information from the
DHCP server, but without specifying a host address. They’ll just take an
address out of the DHCP pool.
You can also specifically identify hosts that are requesting DHCP information from PXE and assign those hosts to a specific group of addresses.
A host booting with PXE identifies itself to the DHCP server as a client of
type PXEclient. You can write specific rules to match clients of that type and
configure them appropriately. Look in the DHCP manual for information
on how to match on vendor-class-identifier and dhcp-client-identifier .

tftpd and the Boot Loader
We covered configuring a TFTP server in Chapter 15. The TFTP server must
provide the pxeboot file for your diskless clients. FreeBSD installs pxeboot in the
/boot directory.
# cp /boot/pxeboot /tftpboot
# chmod +r /tftpboot/pxeboot

Try to download pxeboot via TFTP from your workstation. If that works,
reboot your diskless client and watch it try to boot. The console should show
a message like this:
Building the boot loader arguments
Relocating the loader and the BTX
Starting the BTX loader

You’ve seen this message before, when a regular FreeBSD boots off its
hard drive. Your diskless client will stop here, because it can’t mount the
userland that doesn’t yet exist. That’s a job for your NFS server.

The NFS Server and the Diskless Client Userland
Many tutorials on diskless operation suggest using the server’s userland and
root partition for diskless clients. That might be easy to do, but it’s not even
vaguely secure. Your diskless server probably has programs on it that you
don’t want the clients to have access to, and it certainly has sensitive security
information that you don’t want to hand out to a whole bunch of workstations.
Providing a separate userland is a much wiser option.

T he Fr in g e of F ree BSD

609

While you can provide a separate userland in many ways, I find that
the simplest is to slightly modify the jail(8) construction process from
Chapter 9. First, make a directory for our diskless clients, and then install
a userland and kernel in that directory. Here we install our new userland
in /var/diskless/1:
#
#
#
#
#

mkdir -p /var/diskless/1
cd /usr/src
make installworld DESTDIR=/var/diskless/1
make installkernel DESTDIR=/var/diskless/1
make distribution DESTDIR=/var/diskless/1

This installs a vanilla FreeBSD userland in /var/diskless/1.
Now tell your NFS server about this directory. I intend to install several
diskless systems on this network, so I offer this directory via NFS to my entire
subnet. The clients do not need write access to the NFS root, so I export it
read-only. The following /etc/exports line does this:
/var/diskless/1 -ro -maproot=0 -alldirs -network 192.168.1.0 -mask 255.255.255.0

Restart mountd(8) via /etc/rc.d/mountd to make this share available and
try to mount it from a workstation. Confirm that the directory contains a
basic userland visible from the client and that you cannot write to the filesystem. After that, reboot your diskless client and see what happens. It should
find the kernel and boot into an unconfigured multi-user mode. Depending
on the server, client, and network speed, this might take a while to complete.
My main laptop netboots nearly as quickly as it boots from the hard drive,
while my 266 MHz Soekris net4801 takes several minutes to boot.

I M P R OV I N G D I S K L E S S N F S S EC U R I T Y
Once your diskless farm is running, you could go back and assign a different user for
the NFS root account. Running find /var/diskless/1 -user 0 -exec chown nfsroot
changes the owner of all files owned by root to be owned by the user nfsroot. You can
then edit the exports file to map root to the nfsroot user. When you’re first learning,
however, don’t try to get fancy.

At this point, you could configure your userland to specifically match
your single diskless client. You could make changes in /etc, such as creating
/etc/fstab that reflects your needs, and copy password files into place. That
would suffice for one client, but FreeBSD has infrastructure designed
specifically to support dozens or hundreds of clients off the same filesystem.
Let’s look into how this is done.

610

C h ap te r 2 0

Diskless Farm Configuration
One of the benefits of diskless systems is that multiple machines can share
the same filesystem. However, even on machines that are mostly identical,
you’ll probably find that you must make certain configuration files slightly
different. FreeBSD includes a mechanism for offering personalized configuration files on top of a uniform userland by remounting directories on memory filesystem (MFS) partitions and copying custom files to these partitions.
FreeBSD’s default diskless setup lets you set up diskless workstations
across multiple networks and subnets—an invaluable feature when you have
a large network. If you only have a few diskless systems, however, you might
find it slightly cumbersome at first. Over time, however, you’ll find that you
make more and more use of it. Diskless systems are a convenient solution to
many problems.
On boot, FreeBSD checks to see if it’s running diskless by looking at the
vfs.nfs.diskless_valid sysctl. If the sysctl equals 0, you’re running off a hard
drive; otherwise you’re running diskless. On diskless systems, FreeBSD runs
the /etc/rc.initdiskless script.
Diskless configuration is kept in the /conf directory on the NFS client,
which is the conf directory under the NFS root. While we refer to this directory
as /conf, in our example this directory could be found on the server in
/var/diskless/1/conf.
You must create a few subdirectories: at least /conf/base and /conf/default,
as well as, possibly, separate directories for subnets and/or individual IP
addresses. The contents of these directories are used to build memory filesystems on top of the mounted root partition, so you can make changes for
individual hosts.
The easiest way to explain this is by example. We’ll make a common
change and configure /etc as a memory filesystem on our diskless hosts.

The /conf/base Directory
The /conf/base directory contains basic information about the partitions we’re
going to remount. Anything in this directory affects all of our diskless clients.
Every directory that gets remounted on a memory filesystem needs a further
subdirectory under /conf/base. As we want to remount our /etc filesystem, we
create /conf/base/etc.
Activating Diskless Remounting
In our /conf/base/etc directory, we must tell FreeBSD to remount /etc on our
clients. Create a file named diskless_remount containing the name of the
directory to remount. This sounds complicated, but it just means that the
file /conf/base/etc/diskless_remount contains the single word:
/etc

This tells FreeBSD to build an MFS /etc and mount it appropriately on
our diskless client.

T he Fr in g e of F ree BSD

611

Populating and Trimming Remounted Filesystems
By default, rc.initdiskless copies the server directory to populate the memory
filesystem. That is, our MFS /etc is a copy of what’s in /etc on the read-only
NFS filesystem. Some files are not necessary on your MFS filesystem, however.
For example, diskless systems shouldn’t keep logs locally so they don’t need
newsyslog or /etc/newsyslog.conf. You don’t back up diskless clients, so /etc/
dumpdates is also unnecessary. Browsing /etc will reveal quite a few files that
you don’t need on your MFS /etc. Keeping your diskless /etc small saves
memory, which can be very useful on small systems. If you remove too much,
however, your system will not boot, and the list of necessary files is not intuitive.
For example, if your remove /etc/mtree, the machine will hang in single-user
mode because it cannot repopulate the MFS /var partition.
Put the full paths to your unwanted files and directories in the file
/conf/base/etc.remove. For example, the following entries remove the /etc/gss
and /etc/bluetooth directories as well as the file /etc/rc.firewall:
/etc/gss
/etc/bluetooth
/etc/rc.firewall

Not so hard, is it? Now let’s put some things back into our configuration.

The /conf/default Directory
The default directory contains files issued to all diskless clients. Many files are
the same on all clients, such as /etc/fstab. You put those files in an etc directory
under /conf/default. For example, to distribute /etc/fstab to all of your clients,
you would store it as /conf/default/etc/fstab.
Anything you put in the default directory overrides the same files provided
by remounting the base system.

Per-Subnet and Per-Client Directories
Very few files are unique to certain diskless hosts. You might have a diskless
DNS server that needs a particular named.conf file, or a diskless NTP server
that needs a unique ntp.conf. If nothing else, every diskless host should have
unique SSH keys.
The diskless system provides the ability to override the base configuration
on a per-subnet and a per-IP basis. The per-subnet override requires a directory named after the broadcast address of the subnet. For example, our
diskless box has an IP of 192.168.1.99 and a netmask of 255.255.255.0.
The broadcast address on its network is 192.168.1.255. Any files that
need copying to every diskless client on this network should be placed in
/conf/192.168.1.255/etc.
Similarly, you can specify overrides on a per-IP basis by using a directory
with the same name as the IP address. To place a custom rc.conf on our test
diskless system and only that system, save it as /conf/192.168.1.99/etc/rc.conf.
612

C h ap te r 2 0

Anything in the per-IP directories overrides the same files in the persubnet directory. Similarly, anything in the per-subnet directory overrides
the same files in the default directory, which override the files in the base
directory.

D I S T R I B U T I N G RC . C ON F T O D I S K L ES S H OS T S
While most FreeBSD administrators use /etc/rc.conf as the sole configuration point
for the system, remember that you can also use /etc/rc.conf.local. This is especially
useful for diskless systems. If you want to have a diskless farm where all of the
machines run identical services, you can distribute a single rc.conf to all the hosts.
Place anything that is truly local to an individual machine in /etc/rc.conf.local and
distribute it only to that diskless client. This is the easiest way to keep a large number
of systems synchronized.

Diskless Packages and Files
Naked systems without software aren’t very useful. Inevitably, you’ll want to
install packages on your diskless systems. You also must install configuration
files on diskless clients.

Installing Packages
The key to installing software on the diskless clients is to install it in the
directory the diskless clients will use as their filesystem. You can install
packages in nonstandard directories with pkg_add(8)’s chroot function.
You must copy the software to the diskless client directory before starting.
# cp /usr/ports/packages/All/jdk-1.5.0.11p5,1.tbz /var/diskless/1
# pkg_add -C /var/diskless/1 /jdk-1.5.0.11p5,1.tbz

Remember, chroot changes the root directory of a process. Before
pkg_add(8) does anything, it changes its root to /var/diskless/1. As far
as pkg_add(8) is concerned, the package is now in the root directory.
You can also use ports on your diskless clients, but it’s slightly more
complicated. If you need a customized port, I recommend building it into
a package and then installing with pkg_add -C to cleanly chroot it.

Diskless Configuration Files
You must decide if you want identical configuration files distributed amongst
all of your diskless clients or on a client-by-client basis. Are your configuration
files unique to each diskless client, or do you manage them separately?
Here’s how you can configure the most common configuration files for
diskless operation.

T he Fr in g e of F ree BSD

613

/etc/rc.conf
Every host needs a basic rc.conf. While the hostname and IP address are already
set by DHCP, you must tell the system to create a writable /var and /tmp
filesystem if nothing else.
/etc/fstab
Every FreeBSD machine needs a filesystem table. Even diskless machines
have filesystems, after all! Here’s the /etc/fstab from our sample diskless
machine:
192.168.1.1:/var/diskless/1
md
md
md

/
/tmp
/var
/etc

nfs
mfs
mfs
mfs

ro
-s=30m,rw
-s=30m,rw
-s=2m,rw

0
0
0
0

0
0
0
0

If you provide other partitions (such as home directories), you’ll want
to list them here as well. You might also choose to use local hard disks to
provide swap and scratch space. If you use local disk on a public or semipublic machine, however, I recommend using newfs(8) on those partitions
at each boot so that any data left behind by previous users does not pose a
security risk to the next user.
SSH Keys
If your diskless hosts provide SSH services, they need unique SSH keys. Now
and then you will hear of someone who uses the same SSH keys across his
entire diskless farm. While the traffic is encrypted and hence cannot be read
in transit, shared keys make it impossible to tell one remote machine from
another.
Left to its own devices, FreeBSD automatically creates new SSH keys for
a host at boot. As these keys are written to a memory filesystem, they are lost
at every shutdown. Remember that the SSH client software caches the key for
each remote host it connects to and refuses to connect if the key changes. If
the SSH keys on your diskless hosts keep changing, you will quickly become
annoyed. The simplest thing to do is go into each client’s configuration
directory on the server and create the keys there:
#
#
#
#
#
#

cd /var/diskless/1/conf/192.168.1.99/etc
mkdirs ssh
cd ssh
touch ssh_host_key
ssh-keygen -t dsa -f ssh_host_dsa_key -N ''
ssh-keygen -t rsa -f ssh_host_rsa_key -N ''

If you have many diskless clients, write a script for this.
As the permanent filesystem is mounted read-only, root can actually do
very little damage to a diskless workstation. Considering this, you might
decide that you want to SSH to your diskless workstations as root instead of
creating passwords for each of your administrators. Copy /etc/ssh/shhd_config
to your /conf/default/etc/ssh directory and set PermitRootLogin to yes to enable
614

C h ap te r 2 0

this. (While doing this on a server with a read-write filesystem is a terrible
security risk, the fact that a diskless system is ephemeral changes the rules.)
Password Files
Presumably you’ll want a password file on your diskless system. FreeBSD does
not have a good method for creating password files independent of the main
system. It’s best to create the password files on a test machine or in a jail, then
copy the four key password files (master.passwd, passwd, pwd.db, and spwd.db)
from /etc into the appropriate conf directory. (You can use pwd_mkdb(8)’s -d
option to create and install the database files, but you still have to perform
some drudgery to create a suitable password file in the first place.) You could
also run chroot(8) to lock yourself into your diskless root directory on the
server and run adduser(8) there.
syslogd.conf
Remember, you have no hard drive. If you want your diskless system to use
syslog, you must provide a logging host.
You’ll certainly find other files you need, but this should get you started
in diskless networking.

D I S K L ES S I N S T AL L AT I ON
One common headache these days is that small servers don’t always come with CD
drives. While the machine doesn’t need a CD during operation, its absence complicates installation. Diskless booting makes that problem go away. NFS can export
a FreeBSD installation CD image and have the installation target boot with the
CD image as its root directory. You’ll boot directly into sysinstall!

NanoBSD: Building Your Own Appliances
News flash: Computers are expensive.
Many people dedicate a computer to a single purpose, such as a
firewall/NAT device. It’s quite true that a high-end 486 running FreeBSD
is perfectly capable of handling network connectivity in a small office, but
computers have costs beyond the base hardware. These systems are bulky,
use a lot of power for the job they do, and frequently, their ancient hard
drives are staggering on the verge of death. If you’re in a business environment, your manager does not want to hear, “You know that old desktop
that was so slow you wouldn’t let the receptionist endure it any longer?
I put it back into service as the web server.” FreeBSD lets you cluster fairly
easily, but there’s a reason industry publications never mention a RAIC
(Redundant Array of Inexpensive Crap).1 Of course, you can always use your

1
That’s because the industry would rather sell you a Redundant Array of Expensive Crap, but
that’s another story.

T he Fr in g e of F ree BSD

615

hard-earned IT budget to purchase a new machine with full warranty, but we
all have other things we’d rather buy than a computer that will sit 99 percent
idle. All this adds up to “expensive.” It’s enough to make even the most rabid
open source advocate consider purchasing an inexpensive firewall appliance,
exchanging flexibility and certainty for low-power, cheap-to-replace silence.
You can get the best of both worlds by building your own appliance on
low-power, inexpensive hardware. FreeBSD is used heavily in the embedded
market. Companies such as NetApp and Juniper build high-performance
devices on top of FreeBSD. Tools for building FreeBSD images for embedded devices are integrated with the core FreeBSD source tree. You’ll come
across many methods of doing this, but the most widely used method is
NanoBSD (/usr/src/tools/tools/nanobsd).
Many companies manufacture small, inexpensive, low-end computers,
but my favorite is Soekris (http://www.soekris.com). Soekris boxes are roughly
the size of a paperback book. They have no fan, no video card, very low
power requirements, and are designed specifically to work with open source
operating systems. The CPU, memory, serial and USB ports, and network
interfaces are soldered directly onto the motherboard. You can buy them as a
plain motherboard or with a case. You can use a hard drive or mini-PCI card
with them, or use a flash card as a drive. Best of all, they’re only a couple
hundred dollars—cheap enough to slide through most accounting departments without you having to fill out a mass of paperwork. You don’t get a
warranty with them, but for the price of one small rackmount server you
can buy ten Soekris boxes. If one dies, throw it away and plug the next one
in. Mind you, these systems are so simple that I’ve never had one die in
production.2
We’ll use one of these systems with a flash card instead of a hard drive.
All the examples in this section were implemented on a Soekris net4801,
with a 266 MHz processor, 128MB RAM, and an AC/DC power supply. For
average use, I recommend either a 128MB or 256MB flash card. Anything
more than 256MB is a waste, unless you have really really big programs on
your embedded system. As I write this, however, even 128MB cards are getting
difficult to find; you will probably have to use the smallest flash drive you can
find. Other vendors offer similar hardware, but you’ll have to modify these
instructions slightly. (We’re on the fringe of FreeBSD, remember?)
For my example here, I’m building the smallest possible DNS server,
with only those programs that are strictly necessary for supporting DNS.

What Is NanoBSD?
NanoBSD is a shell script that builds a stripped-down version of FreeBSD
suitable for simple devices. A simple device means that the system is built to
perform a very particular task or tasks. A NanoBSD system can make an
excellent nameserver, or an excellent firewall, or an excellent router. It
makes a lousy desktop, however, as desktop systems are expected to support
2
The one in my test lab that I spilled the Slurpee on doesn’t count. The sparks were really nifty,
though.

616

C h ap te r 2 0

many complicated functions and add functions over time. Similarly, most
multipurpose servers are poor NanoBSD candidates. If you want to convert
your NanoBSD firewall into a NanoBSD DNS server, you’ll need to build a
new NanoBSD image and reinstall from scratch. That’s not very difficult,
but you would never expect to do that on a server.
NanoBSD can be complicated, and you’ll probably need several attempts
to make your first working NanoBSD disk. What’s worse, the configuration
process isn’t easy to break apart for step-by-step explanation. You will probably
want to read this section twice.
Building a NanoBSD image has two main steps: building the operating
system and disk image, and then customizing that disk image for your
environment. We’re going to learn about NanoBSD the same way, by first
focusing on getting a NanoBSD image that boots on your hardware, and
then looking at customizing that image.

Your Hardware and Your Flash Drive
NanoBSD builds a version of FreeBSD tuned to your specifications and then
installs it to a disk image suitable for copying directly to a flash drive. Before
you can build the disk image, you must know your hardware’s idea of your
flash card’s geometry. Flash drives don’t have cylinders or heads, but they tell
the computer that they do. Worse, different computers might see the geometry differently. While tools such as diskinfo(8) might work, they might provide
incorrect answers depending on the exact hardware. The only way to authoritatively identify your flash card’s geometry is by mounting it in the target
system, powering it on, and checking what the flash card reports.
A Soekris uses the serial port as a default console. Unlike many other
embedded systems, however, a Soekris uses a default port speed of 19,200bps
instead of 9,600bps. Attach a null modem cable to the Soekris serial port,
plug the other end into your computer’s serial port, and open up a tip(1)
session at 19,200bps.
# tip -19200 sio0

Now power up your Soekris. After the POST messages, you’ll see a screen
much like this:
comBIOS ver. 1.28

20050529

Copyright (C) 2000-2005 Soekris Engineering.

net4801
0128 Mbyte Memory
Pri Sla

KODAK ATA_FLASH

CPU Geode 266 Mhz
LBA 984-4-32

63 Mbyte

Slot
Vend Dev ClassRev Cmd Stat CL LT HT Base1
Base2
Int
------------------------------------------------------------------0:00:0 1078 0001 06000000 0107 0280 00 00 00 00000000 00000000
...
T he Fr in g e of F ree BSD

617

This informative screen tells you interesting things about the hardware,
but what we’re most concerned with is the disk information. This flash drive
shows up as the primary slave device . Also, the machine sees this 63MB flash
card as having 984 cylinders, 4 heads, and 32 sectors . We can prepare a
NanoBSD image to fit this geometry. Shut down your Soekris box and remove
the flash drive. The next time you power it on, the flash drive will boot it into
FreeBSD.

N A N OB S D AN D D I S K L ES S S Y S T EM S
If you have a diskless infrastructure available, netboot your NanoBSD system diskless
one time. This will give you the dmesg.boot for the system, which will help you make
a good kernel configuration. It will also tell you what device name your system uses
for the flash drive, saving you a round of customization later on.

The NanoBSD Toolkit
You’ll find NanoBSD in /usr/src/tools/tools/nanobsd. The base toolkit includes
a directory called Files, the shell script nanobsd.sh, and the file FlashDevice.sub.
The Files directory contains files that will be copied to your NanoBSD
image. NanoBSD comes with a few scripts to make managing NanoBSD
systems a little easier, and you can add your own later.
The file FlashDevice.sub contains geometry descriptions of various flash
storage devices. This file is woefully incomplete due to the innumerable flash
drive vendors, but very easy to expand.
The real brains of NanoBSD are in the shell script nanobsd.sh. This is
actually a driver for the build-from-source process discussed in Chapter 13.
In addition to these files, you’ll need a NanoBSD configuration file and
possibly a packages directory.

Expanding FlashDevice.sub
Chances are, your flash card is not already listed in FlashDevice.sub. Adding it
is very straightforward and only takes a few moments. All you do is copy an
existing entry and modify it to match your card. You must know the disk brand
and size, as well as the disk geometry as seen by the target device. You can get
the brand and size off the flash card label. The Soekris displays the geometry
in its BIOS screen, as do most hardware vendors. An entry for a new card
will look like this:





618

C h ap te r 2 0

samsung)
#Source: mwlucas@freebsd.org
case $a2 in
128|128mb)
NANO_MEDIASIZE=`expr 130154496 / 512`
NANO_HEADS=8
NANO_SECTS=32



;;
*)
echo "Unknown Samsung Corp Flash Capacity"
exit 2
;;
esac
;;



First, we list the card manufacturer , in alphabetical order within the file.
We know one size of flash drive, the one in our hand. This is a 128MB
device. We want the script to accept either 128 or 128mb as an argument, so we
specify both of them.
NANO_MEDIASIZE
equals the number of cylinders, times the number of
heads, times the number of sectors per track, times by 512. Calculate this
number and make the substitution in the expr(1) statement. (Yes, NanoBSD
immediately divides this value by 512 again.)
The NANO_HEADS and NANO_SECTS statements are taken directly from the
heads and sectors seen by the target device.
We also have a default statement , so that if you try to use a Samsung
256MB flash card without configuring it, NanoBSD will abort early on instead
of building a bogus image.
Add an entry like this to FlashDevice.sub, and you can proceed to configuring your NanoBSD build.








NanoBSD Configuration Options
Customize a NanoBSD build with a configuration file. FreeBSD doesn’t
include a NanoBSD configuration file by default, for a good reason: Every
NanoBSD install is slightly different. You can find many sample files on the
Internet, and I’ll include my own sample file here. As FreeBSD changes,
however, you must tweak your configuration. A NanoBSD configuration that
works with FreeBSD 7.0 won’t be quite right for FreeBSD 7.5 and will quite
probably fail on FreeBSD 8.0. You can have multiple configuration files for
different types of NanoBSD builds.
Here are some of the useful configuration settings and how they can
be used.
NANO_NAME=full

This is the name you assign to this particular build of NanoBSD. NanoBSD
uses this to assign the names to work directories and to the disk image file.
NANO_SRC=/usr/src

This is the location of the source tree NanoBSD uses to build its disk
images. If you want to use a different location, change this variable. You
can build NanoBSD on any version of FreeBSD reasonably close to the
version you’re running on.
NANO_TOOLS=tools/tools/nanobsd

The NanoBSD scripts and files are considered to be part of the FreeBSD
source code. This means that when you run csup, the files that came with
T he Fr in g e of F ree BSD

619

NanoBSD can be updated. If you make changes to any of the NanoBSD
files, csup(1) will undo those changes and return the files to their default
state. If you change the NanoBSD core components, you’ll want to copy
the NanoBSD directory somewhere else under the source tree (such as
/usr/src/tools/tools/local-nanobsd). Tracking the changes between your custom NanoBSD tools and the system base NanoBSD tools is your problem.
NANO_PACKAGE_DIR=${NANO_SRC}/${NANO_TOOLS}/Pkg

NanoBSD looks in this directory for any packages to install on the completed disk image. By default, this is /usr/src/tools/tools/nanobsd/Pkg.
NANO_PMAKE="make -j 3"
This defines the -j flag NanoBSD gives to make(1) during the buildworld
and buildkernel processes. If you have trouble building NanoBSD during
either buildworld or buildkernel, try setting this to make -j 1.
CONF_BUILD=' '

NanoBSD uses any options defined here at the make buildworld stage. You
can find a complete list of options that work on your version of FreeBSD
in src.conf(5).
CONF_INSTALL=' '

NanoBSD uses any options defined here at the make installworld stage.
Again, see src.conf(5) for the supported options.
CONF_WORLD=' '

NanoBSD uses anything in this variable as flags during the whole build
process.
NANO_KERNEL=GENERIC

You’ll almost certainly want a custom kernel for your NanoBSD
image. Specify it here. This should be the name of a file under
/usr/src/sys/i386/conf.
NANO_CUSTOMIZE=""

By default, NanoBSD does not apply any customization scripts to the disk
image building process. We’ll talk about customizing disk images later
in this chapter. Specify any customization scripts that you want to use in
this variable.
NANO_NEWFS="-b 4096 -f 512 -i 8192 -O1 -U"

NanoBSD builds disk image filesystems with newfs(8). Specify any options
you want here. Remember that NanoBSD runs its disk mounted read-only,
so you should not need any custom options here.
NANO_DRIVE=ad0

You must identify how your device sees its flash drive. The easiest way to do
this is to boot the target device and see what it thinks the hard drive is.
NANO_MEDIASIZE=1000000

This gives the size of your flash drive in 512-byte sectors.

620

C h ap te r 2 0

NANO_IMAGES=2

NanoBSD lets you put multiple operating system images on a single flash
drive. This can make upgrades very easy, but uses twice as much space. As
flash drives are now more than large enough to hold almost any FreeBSD
image you can build, there’s rarely any reason to change this.
NANO_CODESIZE=0

You can define how much of your flash drive you want to allocate for
your NanoBSD operating system slices. If you set this to 0, the disk image
building script makes the operating system slices as large as possible.
NANO_CONFSIZE=2048

NanoBSD also creates a slice for configuration data, as we’ll see later.
This specifies the size of this slice, in 512-byte sectors. 2,048 sectors equals
1 megabyte.
NANO_DATASIZE=0

NanoBSD can optionally create a fourth slice for data storage. It is conceivable, though not common, to build a NanoBSD device that stores
data on a long-term basis. If set to 0, nanobsd.sh doesn’t create a data partition. If set to a negative number, NanoBSD makes the data slice as large as
possible.
NANO_RAM_ETCSIZE=10240

This is the size of the memory disk used for /etc, in 512-byte sectors.
10240 is 5MB.
NANO_RAM_TMPVARSIZE=10240

This is the size of the memory disk shared between /var and /tmp. 5MB
is large enough so long as you don’t try to write to your logs.
NANO_SECTS=63 and NANO_HEADS=16

These define the geometry of the flash drive you’re using. Do not set
these in your configuration file! Configure your flash card correctly in
FlashDevice.sub instead.
FlashDevicesamsung128

This tells NanoBSD to take the disk device geometry from the
FlashDevice.sub file, instead of using the NANO_SECTS and NANO_HEADS values.
This sample entry means to use a 128MB flash device made by Samsung.

A Sample NanoBSD Configuration
All this looks impressive, but how do we use it? Let’s create a configuration
file and walk through it. Remember, I’m building a DNS server on a Soekris
device. Any variable I don’t set gets the default value.
#configuration for building a Soekris DNS server

 NANO_NAME=SoekrisDNS


NANO_IMAGES=2
NANO_KERNEL=SOEKRIS
NANO_DRIVE=ad1

T he Fr in g e of F ree BSD

621





622

C h ap te r 2 0

NANO_PMAKE="make -j 3"
FlashDevice samsung 128
customize_cmd cust_comconsole
customize_cmd cust_allow_ssh_root
customize_cmd cust_install_files
CONF_INSTALL='
WITHOUT_TOOLCHAIN=YES
'
CONF_WORLD='
COMCONSOLE_SPEED=19200
NO_MODULES=YES
WITHOUT_ACPI=YES
WITHOUT_ASSERT_DEBUG=YES
WITHOUT_ATM=YES
WITHOUT_AUDIT=YES
WITHOUT_AUTHPF=YES
#WITHOUT_BIND=YES
WITHOUT_BLUETOOTH=YES
WITHOUT_CALENDAR=YES
WITHOUT_CPP=YES
WITHOUT_CVS=YES
WITHOUT_CXX=YES
WITHOUT_DICT=YES
WITHOUT_DYNAMICROOT=YES
WITHOUT_EXAMPLES=YES
WITHOUT_FORTH=YES
WITHOUT_FORTRAN=YES
WITHOUT_GAMES=YES
WITHOUT_GCOV=YES
WITHOUT_GDB=YES
WITHOUT_GPIB=YES
WITHOUT_GROFF=YES
WITHOUT_HTML=YES
WITHOUT_I4B=YES
WITHOUT_INET6=YES
WITHOUT_INFO=YES
WITHOUT_IPFILTER=YES
WITHOUT_IPX=YES
WITHOUT_KERBEROS=YES
WITHOUT_LIBPTHREAD=YES
WITHOUT_LIBTHR=YES
WITHOUT_LPR=YES
WITHOUT_MAILWRAPPER=YES
WITHOUT_MAN=YES
WITHOUT_NCP=YES
WITHOUT_NETCAT=YES
WITHOUT_NIS=YES
WITHOUT_NLS=YES
WITHOUT_NLS_CATALOGS=YES
WITHOUT_NS_CACHING=YES
WITHOUT_OBJC=YES
WITHOUT_PAM_SUPPORT=YES
WITHOUT_PF=YES
WITHOUT_PROFILE=YES
WITHOUT_RCMDS=YES

WITHOUT_RCS=YES
WITHOUT_RESCUE=YES
WITHOUT_SENDMAIL=YES
WITHOUT_SHAREDOCS=YES
WITHOUT_SPP=YES
WITHOUT_SYSCONS=YES
WITHOUT_USB=YES
#WITHOUT_WPA_SUPPLICANT_EAPOL=YES
WITHOUT_ZFS=YES
'

I start by giving this disk image a unique name with the NANO_NAME
variable . I have a custom kernel configuration for this device. Where
did I get this kernel configuration? I booted the Soekris diskless to capture
/var/run/dmesg.boot off of a GENERIC kernel, and then I searched the Internet for other net4801 kernels to identify the potentially useful options other
people used. The only annoying thing about booting this device diskless is
that you must disable ACPI in the boot loader.
Thanks to the diskless boot, I know that the net4801 sees the flash slot
as drive /dev/ad1 . If I don’t set this in my configuration file, on my first
NanoBSD I would get complaints that the kernel could not find the root
partition on /dev/ad0.
I then list the various customize_ scripts I want to use on this image. We’ll
talk about customizations later in this chapter. All of these scripts are in the
default NanoBSD toolkit, and I recommend using them for your first image.
We then have special options for make(1) to use during the install
process. We’ll look at the options a little later, but for now you just need to
know that WITHOUT_TOOLCHAIN tells make(1) to not install the compiler, libraries,
or other tools needed to build software. We won’t be building ports on an
embedded system!
Take careful note of how the CONF_INSTALL variable is set. We have a
single tick mark (') after the equal sign, our variable on a line of its own, and
then a single tick mark (') afterwards. This helps keep multiple options separate and makes the file readable. We do something similar for CONF_WORLD ,
in the next section. NanoBSD expects that each variable will be listed on its
own line, as it builds a private make.conf out of them.
Each of the options in CONF_WORLD tells NanoBSD to not build a particular
section of FreeBSD. Most are self-explanatory; for example, WITHOUT_ACPI says
to not build ACPI. NO_MODULES means to not build kernel modules. I am deliberately not building systems that I don’t want to put on my finished disk image,
such as Sendmail, games, and packet filters.
These options are all buried within FreeBSD’s build structure. Some
of them have been around for years; others are new. Some of them will
disappear—there’s no guarantee than any of them will work a few releases
down the road. Your src.conf man page has the current list of build variables
for the installed version of FreeBSD. I include the whole list just so you can
see how aggressive you can be in cutting. Yes, FreeBSD is so much smaller
than many Linux versions; but this doesn’t mean it can’t be made smaller
still! Some of the build options listed in src.conf(5) should not be used for







T he Fr in g e of F ree BSD

623

NanoBSD, however. For example, WITHOUT_SETUID_LOGIN has bad effects on
your login. WITHOUT_SYMVER removes symbol versioning, which makes it
impossible to use binaries compiled on other FreeBSD machines on your
NanoBSD image. Be certain you understand what a build option really
does before enabling it.
Also, note that you might have to disable some of these build options for
particular NanoBSD images. I’m building a DNS server, so I need BIND.
I commented out the WITHOUT_BIND
entry in my configuration file for this
particular build.
In the CONF_INSTALL option, I list the only option that is used not with
builds but with installations of FreeBSD 7.0. The WITHOUT_TOOLCHAIN tells make
installworld to not install the compiler, headers, or related tools on the disk
image. We need to build a new compiler to build the new FreeBSD, but we
don’t need it installed on our final image. The installation process won’t try to
install anything that we stripped out of the build, but running make buildworld
requires an up-to-date compiler.
This is a complete NanoBSD configuration file. Now, let’s use it to build
our custom disk image.



Building NanoBSD
Run nanobsd.sh as a shell script, specifying the configuration file with -c:
# /bin/sh nanobsd.sh -c mynanoconfig.txt

A NanoBSD build prints fairly generic log messages as it proceeds:
## Clean and create object directory (/usr/obj/nanobsd.SoekrisDNS/)
## Construct build make.conf (/usr/obj/nanobsd.SoekrisDNS//make.conf)
## run buildworld
### log: /usr/obj/nanobsd.SoekrisDNS//_.bw
## build kernel (SOEKRIS)
...
## build diskimage
### log: /usr/obj/nanobsd.SoekrisDNS//_.di
# NanoBSD image completed

If you get the NanoBSD image completed message, the build succeeded. If
not, the build failed at the last stage you see in the log. It’s not uncommon to
perform multiple rounds of troubleshooting to get a configuration file that
works properly, especially when aggressively using WITHOUT_ options to reduce
FreeBSD’s size.
NanoBSD Build Directory
NanoBSD uses a build directory under /usr/obj named after NANO_NAME. Our
configuration file specifies that we’re building a NanoBSD called SoekrisDNS,
so our build directory is /usr/obj/nanobsd.SoekrisDNS. Go to that directory and
look at the files. You’ll find several oddly named files, some large directories,
and disk images.
624

C h ap te r 2 0

NanoBSD uses files beginning with _. as markers, logs, and temporary
directories. Despite the terseness of the messages you saw during the build
process, NanoBSD doesn’t actually discard the output of the commands it
runs to build your disk image; it just redirects it to other files.
Table 20-1: NanoBSD Build Files
File

Purpose

_.bk

make buildkernel log

_.bw

make buildworld log

_.cust.

Installing customization X

_.di

fdisk disk image

_.dl

Disklabel disk image

_.env

The environment used while building NanoBSD

_.etc

make distribution log

_.fdisk

fdisk(8) configuration

_.ik

make installkernel log

_.iw

make installworld log

_.w

Newly built world

disk.image

Disk image of a single slice

disk.full

Disk image of complete device

Use the log files to help solve build and configuration problems.
NanoBSD Build Troubleshooting
NanoBSD’s error messages when a build fails aren’t terribly helpful. For
example, here’s a complete output of a failed NanoBSD build:
# /bin/sh nanobsd.sh -c soekris-dns.conf
## Clean and create object directory (/usr/obj/nanobsd.SoekrisDNS/)
## Construct build make.conf (/usr/obj/nanobsd.SoekrisDNS//make.conf)
## run buildworld
### log: /usr/obj/nanobsd.SoekrisDNS//_.bw



You can see that the build process failed, but not why.
To get the actual error, look at the log file specified in the message. In
this case, the log is the file /usr/obj/nanobsd.SoekrisDNS/_.bw . The end of
that log file shows the last output from the failed build process. You’ll see the
usual sort of error for that part of the upgrade process.
One thing to remember is that NanoBSD uses make -j3 by default. This
can obscure error messages in the log, as the error that caused the crash might
be several lines before the apparent end of the build.
Why would a NanoBSD install fail when a regular upgrade from source
works fine from the same source code? The leading cause of build failures is
a choice of incompatible build options for the source tree. My NanoBSD
configuration file includes many different WITHOUT_ options. You might notice



T he Fr in g e of F ree BSD

625

that some of them are commented out. At the time I built this disk image,
those options broke the build. While my embedded DNS server doesn’t really
need the software provided by the options I commented out, not including
these options broke my NanoBSD build. (WITHOUT_BIND, of course, is commented out because I actually want BIND on this image.)
Another problem you can have when you’re building a NanoBSD image
is when the build completes, but the disk image construction dies with:
/usr/obj/nanobsd.SoekrisDNS/_.mnt: write failed, filesystem is full

Your files take up more space than is available on the drive. By default,
the /conf slice takes up 1 megabyte. NanoBSD splits the remaining space in
half. If you have a 128MB flash card, each of the two NanoBSD images should
get 63.5MB of disk space. If your FreeBSD build, including any additional files
and the installed packages, is larger than that, image construction will fail.
Don’t forget that flash card vendors measure disk space the same way disk
drive manufacturers do, in base 10 rather than base 2. Your 128MB flash card
is actually a bit smaller than that. You need to put only one NanoBSD image
on the disk, reduce the amount of stuff you include, or get a larger flash card.
The Completed Build
So, your build finished! Take a look at the disk image and see what’s in it:
# mdconfig -af _.disk.full
md0

If you run fdisk(8) against the disk image, you’ll see that it has three slices
in use, two large ones and a very small third one. The two large slices are our
NanoBSD images. Mount one and look around in it.
# mount /dev/md0s1a /mnt

Some poking around reveals that your NanoBSD system is complete but
doesn’t really do anything. It’s an unconfigured FreeBSD install. Nevertheless,
I recommend copying it to a flash disk and test-booting it on your target
hardware to make sure it runs. Install NanoBSD on the flash disk /dev/da0
by running:
# dd if=disk.full of=/dev/da0

Now remove the flash drive, install it in your Soekris or other device, fire
up your serial console, and turn on the power.
Serial Console Speeds
If you boot a vanilla FreeBSD install on a Soekris box, you’ll have problems.
Soekris hardware uses a default serial console speed of 19,200bps, while
FreeBSD uses a more common 9,600bps. This means that if you connect
626

C h ap te r 2 0

your terminal at 19,200, you’ll see all of the Soekris boot messages, but when
FreeBSD starts, you’ll see garbage. If you connect your terminal at 9,600,
you’ll see the FreeBSD messages but nothing but garbage from the Soekris.
Changing the FreeBSD console speed will be one of the first customizations we perform on our FreeBSD system. For an initial test of your
FreeBSD image, however, boot your Soekris and console in at 9,600bps.

Customizing NanoBSD
Now that you have a vanilla FreeBSD install that boots on your hardware, let’s
customize it to meet our needs. Theoretically, you can customize NanoBSD in
two ways: by editing the finished disk image or by changing the build to
create the desired disk image. The first method is easy, but undesirable. If
you’re actually using NanoBSD in a production environment, one day you’ll
have to build a new disk image. Perhaps you must apply a security patch, or
you’ll find an application where a second NanoBSD box would be ideal. By
modifying the build, you greatly ease producing a correct and usable updated
image. You have three tools to modify the build: customization scripts, copying
files, and adding packages.
Customization Scripts
Customization scripts are shell script functions inside nanobsd.sh or within
your configuration file. Use a customization script to change a file that’s
likely to change as FreeBSD grows. For example, FreeBSD updates the
included version of SSH every few releases. You don’t want to write a custom
sshd(8) configuration file and load it unilaterally, as that would quite possibly be incompatible with newer versions of OpenSSH. The sensible thing
to do is use a configuration script to make changes to the sshd_config included
with the system, no matter what version it might be. NanoBSD includes a
customization script just for that.
cust_allow_ssh_root () (
sed -i "" -e '/PermitRootLogin/s/.*/PermitRootLogin yes/' \
${NANO_WORLDDIR}/etc/ssh/sshd_config
)

This simple sed(1)3 script changes one line in /etc/ssh/sshd_config to
allow the root user to SSH into the NanoBSD system. This script would break
if the PermitRootLogin option was renamed or removed, but that will happen
rarely if ever. You can include anything you like in a customization script,
even spawning a complicated Perl script that takes NANO_NAME as an argument
and writes a custom rc.conf for your image.

3
sed(1) and awk(1) expertise is another thing that separates the Real Sysadmins from the
young punks. Reading shell scripts is an excellent way to pick up enough about these tools to
fool people into thinking you’re an expert.

T he Fr in g e of F ree BSD

627

NanoBSD includes four customization scripts:
cust_comconsole
Enables serial console and serial login.
cust_allow_ssh_root
Allows root to SSH into the system.
cust_install_files
Copies files under the Files directory into completed image.
cust_pkg
Installs packages from the Pkg directory into completed image.
As you’ve probably gathered, customization scripts drive the other two
methods of tweaking your image.
Adding Packages
Create a Pkg directory in the NanoBSD directory. The customization script
cust_pkg installs any packages you place in this directory on your image.
That’s really it.
If you’re short on space, however, you don’t need to install the entire
package. Find the binary for the program you want to run, use ldd(1) to
identify the necessary libraries, and just copy those files to your image.
Adding Files
The cust_install_files customization script copies files from the tree under the
Files directory into your completed image. By default, the Files directory
contains a single subdirectory called root. The contents of this directory are
copied into /root on the image.
If you regularly build NanoBSD images, I recommend changing the
cust_install_files script. NanoBSD includes several files in the Files directory,
and those are considered system files. When you update your source tree,
csup(1) wipes out any changes you made to any files under the Files directory.
Copy cust_install_files into your configuration file, give it a name like
cust_install_dns_files, and point it to a directory other than Files.
I’m building a DNS server, so I need a custom named.conf and rndc.conf.
These files go in /var/named/etc/namedb, so I must create the directory
/usr/src/tools/tools/nanobsd/Files/var/named/etc/namedb (ick!) and put my
custom files in there.
Similarly, I need an rc.conf. I’ll create Files/etc/rc.conf for my configuration.
I need SSH keys, which I create exactly as I would for a diskless workstation.
Similarly, I want a syslog.conf that writes to my networked log server.
One last annoyance when working with Soekris hardware is the serial
console speed. We set the serial console speed in our configuration file with
the COMCONSOLE_SPEED variable, telling FreeBSD to use a default console speed
of 19,200. If you didn’t do that, the simplest way to accommodate the difference in port speed is to set FreeBSD to use the higher console speed.

628

C h ap te r 2 0

You can do this with the proper entries in /boot/loader.conf. Just create the
following entries in Files/boot/loader.conf :
comconsole_speed="19200"
console="comconsole,vidconsole"

This gives your NanoBSD install a serial console with a default speed of
19,200bps. It won’t take effect until the boot loader starts, however. You’re
really better off building your whole NanoBSD world with a console speed of
19,200 through a CONF_WORLD variable.
Perfecting Customizations
One of the headaches in building a NanoBSD image is getting the right set
of customizations for your purpose. You’ll add every file and every setting
you think you need in your image, copy the completed image to a flash drive,
and boot it on your device only to realize you forgot something and must add
a file to the image.
While building a NanoBSD image takes a while, the longest part of the
process is building the world and kernel. You can add a file to the customization image and rerun the image-building process without building a whole
new world and kernel by adding the -b flag to nanobsd.sh:
# /bin/sh nanobsd.sh -b -c myconfig.txt

This skips the buildworld and buildkernel stages, and assembles a fresh
disk image with your updated files.

Using NanoBSD
So you have a NanoBSD image that boots and includes all the software you
need. You’ve copied it to a flash drive and are running it without trouble.
What happens when you must make a minor change or install an upgrade?
Not to worry. NanoBSD has special provisions for both these cases.
Minor Updates
NanoBSD uses the cfg slice to handle minor changes. Any files on /cfg are
copied to the running NanoBSD’s /etc partition during boot. This means
that you can make changes to, say, /etc/rc.conf without building a whole
new image. For a good example of how this is used, take a look at the
change_password script in Files/root. This script runs passwd(1) to let you
change root’s password, mounts /cfg read-write, copies the vital password
files to /cfg, and unmounts /cfg. The next time the system boots, NanoBSD
copies those vital password files to /etc.
Whenever you make changes to your running NanoBSD system, I
recommend making corresponding changes to your NanoBSD build system.
This way, when you have to roll a new disk image, your changes will be all
captured and up to date.

T he Fr in g e of F ree BSD

629

Updating Disk Images
So, you’ve built a NanoBSD image and are running it on a Soekris box
nailed to the top of a windmill so you can use custom SNMP scripts to
manage power output. One day you have to do an upgrade. Do you break
out a ladder and a crowbar so you can climb to the top of the windmill and
install a new flash card? No, most FreeBSD developers are far too sedentary
to tolerate any such thing. 4
Remember, a NanoBSD system has two disk images. Only one of them
runs at any moment. The included upgrade scripts allow you to upgrade the
disk image that is not in use at the moment. You then reboot onto the new
image. If the new image doesn’t work, you can boot the original image,
restoring service while you work out what went wrong.
Upgrade a NanoBSD installation remotely over SSH with the scripts
updatep1 and updatep2. If your system is running slice 1, use the script updatep2.
If you’re on slice 2, run updatep1. In addition to providing a command prompt,
SSH lets you transfer files or even run commands on remote systems. Here, we
are logged into our NanoBSD system and want to update slice 2 with a disk
image on the server 192.168.1.5:
# ssh 192.168.1.5

cat _.disk.image | sh /root/updatep2

We connect to the remote system and feed it the output of cat(1) —in
this case, the image of a single partition. On the remote system, we run the
updatep2 script, which runs some basic sanity checking to confirm we aren’t
trying anything too daft and then copies the disk image to the unused slice.
It also sets the new boot default to the freshly upgraded slice. If the upgrade
doesn’t work correctly—that is, if the system boots but the application doesn’t
run correctly—use the serial console to boot on the other slice, or use
boot0cfg(8) on the running system to tell NanoBSD to boot the other
slice next time.
Once you start to experiment with NanoBSD, you can find any number
of uses for it. In addition to custom-built appliances, I use NanoBSD to
provide a userland for jails and diskless workstations. If you want more fullfeatured bootable FreeBSD media, though, take a look at FreeSBIE.

Live Media with FreeSBIE
Compared to NanoBSD, FreeSBIE takes a different approach to creating
purpose-built FreeBSD installs and has a different target. While NanoBSD is
trying to satisfy users of small media, FreeSBIE targets live read-only media
such as CDs, DVDs, and big flash devices. Having much more disk space gives
FreeSBIE a certain amount of flexibility lacking in NanoBSD. While I prefer
using NanoBSD for appliance functions, many people use FreeSBIE to build
bootable CDs that turn any old workstation into an appliance-style server.
4
Put a good drink at the top of that windmill, however, and you’ll get three out of four
developers up the ladder before you can blink. It’s all a question of alinging your needs with
their priorities.

630

C h ap te r 2 0

Using a live CD is as simple as booting off of the CD. The operating
system on the CD should identify your hardware, configure the installed
software, and in general boot into a fresh and usable operating system. If
you want detailed information about using a running FreeSBIE install,
check out the FreeSBIE website at http://www.freesbie.org. We’re going to
focus on building your own customized FreeSBIE disk.
FreeSBIE builds a fresh world and kernel from the source code in /usr/src,
but then takes its packages from the system you’re building on. This means
that your build system must have a fairly coherent and clean set of packages.
If you’ve abused your system and your package records are inaccurate, you
can’t use them to build a FreeSBIE live CD.
While we refer to a FreeSBIE live CD throughout this section, all of this
applies to building DVDs and flash devices as well.

F R EE S B I E AN D F R EE B S D
FreeSBIE is not an integrated part of FreeBSD. While the FreeSBIE developers work
very hard to support newer versions of FreeBSD, their work still lags slightly behind
mainstream FreeBSD development. This chapter was written using FreeSBIE 2, which
is designed for FreeBSD 6. I expect FreeBSD 7 support shortly after FreeBSD 7 is
released. When you start working with FreeSBIE, be prepared to do your own
investigation and troubleshooting.

Installing the FreeSBIE Toolkit
You can get the very latest FreeSBIE tools from the FreeSBIE CVS repository
available through http://www.freesbie.org, or, if you have an up-to-date Ports
Collection, you can install it from /usr/ports/sysutils/freesbie. Everything in the
FreeSBIE toolkit installs under /usr/local/share/freesbie.
The FreeSBIE toolkit includes very few instructions. Most of the documentation is available only on the FreeSBIE website, http://www.freesbie.org.

Configuring FreeSBIE
Under /usr/local/share/freesbie/conf you’ll find the default FreeSBIE configuration file freesbie.defaults.conf. Do not edit this file directly; instead, create a
freesbie.conf file in the same directory. Anything in freesbie.conf overrides the
same setting in freesbie.defaults.conf.
As of FreeSBIE 2.0, here are some commonly changed configuration
options. The example configuration below shows as you would configure it
in your customized freesbie.conf file, not as it appears in freesbie.defaults.conf.
(The default configuration file has special syntax to allow your custom
configuration to override those values.)

T he Fr in g e of F ree BSD

631

For your first FreeSBIE build, I recommend changing as little as possible.
While many options might be tempting to change, sticking as close to the
defaults as possible gives the best initial results. Overly ambitious initial
customization causes build failures. As with so many other things, I recommend you learn about FreeSBIE’s hidden catches one at a time.
BASEDIR="/usr/local/freesbie-fs"

This is the directory where FreeSBIE installs its freshly built world,
packages, and custom files.
CLONEDIR="/usr/local/freesbie-clone"

FreeSBIE uses this directory as a temporary staging area when building
images.
SRCDIR="/usr/src"

This is the source tree that FreeSBIE uses to build its world and kernel.
ISOPATH="/usr/obj/FreeSBIE.iso"

FreeSBIE puts the completed ISO at this location.
IMGPATH="/usr/obj/FreeSBIE.img"

This is the full path to a completed UFS FreeSBIE image for flash media.
MAKEJ_WORLD="-j3"

This determines the -j argument for make(1) to use when building world.
Set this to -j1 if you have a build failure and want to determine if it’s an
actual problem or just a parallelization issue.
MAKEJ_KERNEL="-j1"

This sets the parallelization of make(1) when building the kernel.
MAKEOPT="-DNO_CLEAN"

If you want to use any make(1) options for the world and kernel builds,
specify them here.
KERNELCONF="/usr/src/sys/i386/conf/CUSTOMFREESBIE"

This is the full path to the FreeSBIE kernel configuration file. I find that
the kernel configuration provided with FreeSBIE only works well with the
same version of FreeBSD that the FreeSBIE developers are using. As of
FreeBSD 7.0, a FreeSBIE build takes advantage of the following nonGENERIC options:
options
options
options
options

GEOM_UZIP
GEOM_LABEL
VESA
SC_PIXEL_MODE

I additionally remove all SCSI and RAID controllers, as I really
don’t need access to those disks. If I don’t want to touch the hard drive
I’ll also remove the atadisk device so that my disk image cannot find
any IDE disks. If you want firewall features, add the PF kernel options.

632

C h ap te r 2 0

MAKE_CONF="/etc/make.conf"

This defines the custom make.conf you want to use for your FreeSBIE
build. Note that as the packages are copied from the installed system,
any special make.conf settings will not apply to the packages. You might
as well use src.conf instead.
SRC_CONF="/etc/src.conf"

You can customize the source build with src.conf, just as you can with the
primary FreeBSD build. Specify the full path to your FreeSBIE-specific
src.conf here.
FILE_LIST="/home/mwlucas/freesbie-files.txt"

You can specify exactly which files you want to install on your FreeSBIE
disc. If you specify a file list, FreeSBIE installs only the files on that list
into the image. (Effectively, this means that only files on this list are
copied from the FreeSBIE base dir to the FreeSBIE clone dir.) This means
you must make a 100 percent complete list of all the files on the live CD.
This is most useful for NanoBSD-like small systems, in my opinion.
PRUNE_LIST="/home/mwlucas/freesbie-prune.txt"

This file contains a list of files and directories that are included in the
build, but must be removed before building the live CD image. Any files
listed in the prune list file are removed from the cloned directory before
the image is built from that directory. This can be very useful to reduce the
size of a live CD.
PKGFILE="/home/mwlucas/freesbie-packages"

This file contains a list of packages, one package per line, without version
information. FreeSBIE installs all of the packages on this list onto your live
image. If you do not set this, FreeSBIE’s make packageselect command
creates a package list in the file /usr/local/share/freesbie/conf/packages.
EXTRA="customroot rootmfs etcmfs sound xautostart"

FreeSBIE provides a variety of plug-ins to enable different functions on
the live CD. We’ll cover the standard plug-ins later.
MINIMAL=YES

If you define MINIMAL, FreeSBIE builds the smallest FreeBSD it can. This
creates a result similar to NanoBSD. A MINIMAL build only works reliably
with whatever version of FreeBSD the FreeSBIE developers are using.
NO_BUILDWORLD=YES

By defining this, you tell FreeSBIE to not build a new userland. You must
have a freshly compiled userland available in /usr/obj.
NO_BUILDKERNEL=YES

This tells FreeSBIE to not build a new kernel and use the previously compiled kernel in /usr/obj instead.

T he Fr in g e of F ree BSD

633

MAKEOBJDIRPREFIX="/usr/freesbie/obj"

While you can set this to provide a separate FreeSBIE build area, doing
so is not recommended. Many shell scripts assume that the recently built
userland and kernel are under /usr/obj.
NO_COMPRESSEDFS=YES

This tells FreeSBIE to use an uncompressed filesystem on the ISO image.

FreeSBIE Plug-ins
FreeSBIE provides plug-ins to more thoroughly customize disk images. While
a default FreeSBIE build is perfectly serviceable, plug-ins let you turn on and
off different features on your live CD. Plug-ins install extra configuration
scripts or enable different automated functions on your bootable image. All
of the plug-in scripts are in /usr/local/share/freesbie/extra. The README file
describes all of the default plug-ins.
List all of the plug-ins you want to use in the EXTRA value in freesbie.conf.
Here are some of the plug-ins I find most interesting or useful.
adduser
The adduser plug-in adds the user freesbie to the live CD. You can specify a username of your own choosing with the FREESBIE_ADDUSER variable in
freesbie.conf.
autologin
When you enable the autologin plug-in, the user freesbie is automatically
logged in at boot.
comconsole
Enabling the comconsole plug-in in the build adds the serial console as
a second console. Setting the configuration option SERIAL_ONLY=YES in
freesbie.conf builds an image that only uses the serial console. You could
also do this by adding a custom loader.conf into the image through the
customroot plug-in.
customroot
The customroot plug-in lets you add arbitrary files to your live CD. Any
files in the directory /usr/local/share/freesbie/extra/customroot are copied to
your new image. Use subdirectories as necessary to put the files in the
proper place. For example, if you want a custom /etc/rc.conf, put it in
/usr/local/share/freesbie/extra/customroot/etc/rc.conf. By adding a custom
/usr/local/share/freesbie/extra/customroot/var/named/etc/namedb/named.conf,
you can have an easily replaceable secondary nameserver that boots off
of read-only media. Any files added by this plug-in override any other file
source in FreeSBIE.
customscripts
The FreeSBIE build process runs any scripts found in /usr/local/freesbie/
extra/customscripts immediately before creating the ISO image. You can
add users or edit files automatically with this plug-in.
634

C h ap te r 2 0

etcmfs
This tells the system to use a memory filesystem for /etc.
l10n.sh
This plug-in lets the live CD user choose an international character
set. This is useful for non-English speakers.
mountdisks
With this plug-in enabled, FreeSBIE mounts any UFS, FAT, or NTFS
slices found on the system the live CD runs on.
pf
This plug-in creates and activates a simple “block all incoming, allow all
outgoing” PF ruleset upon boot.
rootmfs
This plug-in makes the live CD use a memory filesystem for root. While
the user can make changes to the root directory, they disappear upon a
reboot.
sound
Upon boot, the live CD autodetects the sound card and installs the correct sound driver.
swapfind
If the system running the live CD has any swap partitions, FreeSBIE finds
and uses them. This requires writing to the disk, of course, which might
be undesirable.
varmfs
The live CD uses a memory filesystem for /var. This makes local logging
possible, but the logs won’t survive a reboot.
xautostart
The live CD starts X at boot.
xconfig
The live CD automatically configures X at boot. This is desirable when
using xautostart.
xconfigure-probe
FreeSBIE uses an alternate method to automatically configure X at boot.
(As X offers multiple methods to configure a display, this dichotomy
isn’t anything the FreeSBIE team can solve.)

Choosing Packages
You can create a text file containing all the packages you want to install on
your live CD, or you can use make packageselect in /usr/local/share/freesbie. This
command creates a menu of all the packages installed on your build system
and lets you select which ones you want to install on your live CD. When you
select a package, all dependencies are pulled in automatically.

T he Fr in g e of F ree BSD

635

Building a FreeSBIE Image
To build a FreeSBIE CD from your configuration, just do:
# cd /usr/local/share/freesbie
# make iso

FreeSBIE builds a complete FreeBSD, trims and expands it as desired,
installs packages, runs the customization scripts, and compresses the whole
thing into a customized live CD image.
If you want to create a flash image instead of an ISO, use make flash.
FreeSBIE asks which flash device you want to use, initializes it, builds
a FreeSBIE userland, and installs it on the specified flash device.

Rebuilding FreeSBIE
FreeSBIE uses dotfiles much like ports to indicate when it’s finished with a
step. By default, these files are in /usr/obj/usr/local/share/freesbie.
If you want to rebuild FreeSBIE without rebuilding world, delete the files
for the stages that you’ve completed already.
This is the last stop in our exploration of FreeBSD’s darker corners. Now
let’s see what to do when things go really, really, really wrong.

636

C h ap te r 2 0

21
SYSTEM (AND SYSADMIN)
PANICS AND CRASHES

One of the nicest things about FreeBSD is
its stability; the only Blue Screen of Death
you can find is a screensaver. In fact, I ran
FreeBSD for almost a year before seeing a
machine crash for reasons other than bad hardware.
FreeBSD can crash, or panic, but it allows you to recover
from a panic fairly easily—so don’t, er, panic. You can even connect to the
remote console during a panic and force a reboot. FreeBSD provides the
tools you need to discover exactly what happened as well as extensive debugging information about the panic. Even if you don’t know what to do with
this information, you can submit a problem report and discuss the matter
with the FreeBSD development team.

What Causes Panics?
When does a system panic? Panicking is a choice made by the kernel when
it faces an unresolvable conflict. If the system achieves a condition that it
doesn’t know how to handle, or if it fails its own internal consistency checks,

it panics. Production versions of FreeBSD are increasingly difficult to panic,
but it can still happen. The easiest way to panic a system is to do something
daft, like pull out a non–hot swappable hard drive while it’s in use. Panics are
not uncommon when running -current; they’re not frequent, mind you, but
they’re not rarities.
FreeBSD is very complex, and neither its royal blood lineage nor the
open source development process can protect it from all bugs. Fortunately,
that heritage and the development process do give you the tools you need
to provide the information for other people to debug your problem. You
might begin with a cryptic error code, but you’ll quickly learn that your
string of garbage characters means something to someone.
I recommend that every system be prepared to capture a crash dump
before it is allowed to enter production. FreeBSD captures crash dumps by
default, but if you reconfigure your server or have special disk partitioning
you’ll want to confirm that crash dumps will still work. This precaution will
be wasted on most of your servers, but the preparation really takes very little
time. When one of your several servers panics, you’ll be glad that you have all
of the crash information on hand.

Recognizing Panics
When a system panics, it stops running all programs and stops listening to the
network. On a released version of FreeBSD, or when tracking -stable, a system
that panics automatically reboots. Not all unexplained reboots are panics—if
you have a bad power supply or bad memory, for example, the hardware
failure can cause a reboot without any sort of log or console message. You
might have messages in your error log if you panic, depending on the type
of panic. If you have your system configured to display panics rather that
reboot, you’ll see a console message much like this:
panic: _mtx_lock_sleep: recursed on non-recursive mutex ahc_lock @ /usr/src/
sys/dev/aic7xxx/aic7xxx_osm.h:203
cpuid = 0
KDB: enter: panic
[thread pid 12 tid 100002 ]
Stopped at
kdb_enter+0x32: leave
db>

The only part of this message that seems even vaguely sensible to me is
the first line. I know that a mutex is a kind of kernel lock (see Chapter 12),
and my SCSI card uses the ahc driver. This makes sense, actually; I triggered
this panic on a -current machine by running fdisk(8) on a SCSI hard drive.1

1
Also known as, “I guess I’m not writing Chapter 18 today, as it is FreeBSD’s will that I write
Chapter 21.” And here I was, wondering how I was going to get a good panic for this chapter.

638

C h ap te r 2 1

The db> at the bottom is a debugger command prompt. Hit ENTER a couple
of times and you’ll see that you can enter commands. They aren’t useful Unix
commands, but they will help you get more information out of the system.

Responding to a Panic
If you get a panic, the first thing to do is get a copy of the panic message. Since
FreeBSD is no longer running, the standard methods for copying data from
your machine won’t work—you cannot SSH in, and script(1) is no longer
viable. The console might even be completely locked up and unresponsive
instead of being in the debugger. In any event, you must have that error
message.
FreeBSD didn’t always automatically reboot after a panic; originally it just
sat there displaying the panic message. The first time I saw a panic, I scrambled
for paper and pen. Eventually I found an old envelope and a broken stub of
pencil that made marks if you held it at just the right angle, and crawled
between the server rack and the rough brick wall. I balanced the six-inch
black-and-white monitor in one hand, and with my other hand I held the
envelope against the wall. Apparently I grow a third hand under duress,
because I got the panic message onto the envelope somehow. Finally, scraped
and cramped, I slithered back out of the rack and victoriously typed the
whole mess into an email. Surely the FreeBSD Project’s Panic Emergency
Response Team would be able to look at this garbage and tell me exactly
what had happened.
I quickly learned that FreeBSD has no elite group standing by to take my
problem report. Instead, I got a lonesome email: “Can you send a backtrace?”
When I asked how, I was directed to a man page. (See Chapter 1.) Fortunately,
the panic was easily reproducible—the only thing that had to happen to
recreate the issue was for a customer to log in to the system. I spent the rest
of the day struggling to master serial logins and core dumps.
The problem with the panic message on my envelope was that it only
gave a tiny scrap of the story. It was so vague, in fact, that it was like describing
a stolen car as “red, with a scratch on the fender.” If you don’t give the car’s
make, model, VIN, and license plate number, you cannot expect the police
to make much headway. Similarly, without much more information from your
crashing kernel, the FreeBSD developers cannot catch the criminal code.
There’s a really simple way around this problem, however: Configure all
of your machines to handle a panic before the panic happens. Set them up
when you install the server. I now configure all my machines for proper panic
processing prior to permitting them in production. That way, you’ll be ready
for disaster and you’ll know exactly what to do if a machine ever panics. This
might seem like a novel idea, and it certainly isn’t emphasized in the FreeBSD
documentation, but it makes sense. If none of your systems ever panics, you
don’t have anything to complain about. If you get a panic, you’re ready and
you’ll be able to present the FreeBSD folks with complete debugging information without any trouble.

Sy st em ( an d Sy s ad mi n) Pa n ics a n d C r as h es

639

Preparations
While you can manage a panic in several ways, I prefer to dump the memory
to disk for leisurely processing. You need a swap partition slightly larger than
your kernel’s memory usage. While it’s possible that you might need to dump
the entire contents of your physical RAM, it seems FreeBSD 7.0 kernel dumps
are less than half a gigabyte in most cases. If you are using something that
requires a great deal of kernel memory, such as the experimental ZFS support,
your kernel dumps will be much larger. If you do not have a sufficiently large
swap partition, you must either reinstall or add another hard drive with an
adequate swap partition. (Remember I brought this up back in Chapter 2?
Don’t you wish you’d listened to me then?) While having a /var partition
sufficiently large to hold the kernel dump is helpful, it isn’t mandatory.

The Crash Dump in Action
The kernel crash-capturing process works somewhat like this: If a properly
configured system crashes, it saves a copy of the kernel memory. The copy is
called a dump. The system cannot save the dump straight to a file. The crashed
kernel doesn’t know anything about filesystems, for one thing, and the filesystem might be corrupt or a write could corrupt it. The kernel understands
partitions, however, so it can write the dump to a partition. The best partition
available is the swap partition, which is memory scratch space. FreeBSD defaults
to dumping on the first swap partition on the system, placing the dump as
close to the end of the partition as possible. After the dump, the computer
reboots.
FreeBSD enables swap during the reboot and then checks its filesystems for
cleanliness. The partitions will almost certainly be dirty after a panic. Perhaps
they’re journaled or use background fsck(8), but the system cannot guarantee
that. FreeBSD must enable swap before running fsck(8), because fsck(8) might
need swap space. Hopefully, you have enough memory for fsck(8) to not
require swapping, and if swapping is necessary, hopefully you have enough
swap space to avoid overwriting the dump file lurking at the end of the swap
partition.
Once FreeBSD has a filesystem where it can save a core dump, it checks the
swap partition for a dump. If it finds a core dump, FreeBSD runs savecore(8)
to copy the dump out of swap and into a proper filesystem file, clears the dump
from swap space, and continues rebooting. You now have a kernel core file
usable for debugging.

Configuring Crash Dumps
FreeBSD defaults to saving core dumps on the first available swap partition
and, on every reboot, checks that partition for a dump. You can change this
behavior as necessary to fit your system.

640

C h ap te r 2 1

The obvious change you might need is to place your dump in a different
swap partition, especially if you had to add a hard drive to get enough swap to
hold your dump. Set the dump device with the dumpdev variable in /etc/rc.conf:
dumpdev="/dev/ad4s1b"

Remember, /etc/fstab lists all the swap partitions.
savecore(8) automatically places kernel dumps in /var/crash. If your /var
partition is not large enough to contain the dump, however, set a different
directory with the dumpdir variable in rc.conf:
dumpdir="/usr/crash"

While savecore also supports a few other options, such as compression,
they aren’t usually necessary on modern systems.
S ER I A L C O N S OL ES AN D P A N I C S
While a serial console is not strictly necessary for panic debugging, it can be
invaluable when dealing with a stuck machine. The ability to capture everything with
script(1), if nothing else, makes a serial console worthwhile. On a remote machine,
a serial console is a necessity. If you really want to be prepared for a panic, make
sure all of your machines have serial consoles or at least dual consoles. If possible,
log the output of your serial consoles; this way, you’ll get the panic message even if
the system is not configured for crash dumps.

With a debugging kernel and a dump device, you’re as ready for a panic
as you can be.

Debugging Kernels
The FreeBSD kernel build process makes large amounts of debugging information available in the kernel. You’ll find the primary debugging information in the symbols files, which provide a map between the machine code and
the source code. This map also includes a complete list of source code line
numbers so that developers can learn exactly where a problem occurred.
Without this information, the developer is stuck trying to map a kernel core
to the source code by hand, which is somewhat like solving a million-piece
jigsaw puzzle without a picture or even a guarantee that you have all the
pieces. This would be an ugly job regardless, but it becomes even uglier when
you consider that the developer doing the work is an unpaid volunteer.
Symbols are important.
Including the debugger in the kernel eases the debugging process if you
have console access or if you have a serial console on a remote machine. While
you can use entirely off-line debugging, at times having the debugger handy
during a crash simplifies troubleshooting.
Sy st em ( an d Sy s ad mi n) Pa n ics a n d C r as h es

641

To retain debugging information and include the kernel debugger, ensure
your kernel configuration includes these lines:
makeoptions
options
options
options

DEBUG=-g
KDB
KDB_TRACE
DDB

When you add the DDB option to your kernel, FreeBSD will no longer automatically reboot after a panic. Do you want your system to reboot itself after a
panic, or do you want it to wait for you to intervene? If the system is at a
remote location, you’ll almost certainly want to trigger a dump and reboot
automatically after a panic, but if you’re at the console, you might want it
to wait for you to tell it to reboot.
To reboot automatically, use the kernel option KDB_UNATTENDED:
options

KDB_UNATTENDED

This option tells FreeBSD to reboot after a panic even if DDB is in the
kernel. When the system panics, you’ll see a message telling you to hit the
spacebar to enter the debugger; if you don’t do it, the system will reboot in
15 seconds.

When Panic Strikes: Manual Crash Dumps
Suppose you have a debugging kernel installed, your swap partition can contain your dump, and your machine is not configured to reboot automatically
after a panic. One day, one of your machines disappears from the network.
You look at the console only to find a cryptic message followed by a db> prompt.
Or, perhaps you hit ENTER on the serial console only to see the db> prompt
appear. FreeBSD is waiting for you to tell it to reboot after a panic.
While you have the machine in the debugger, take a moment to capture
some of the debugging output. If the panic message is there, copy it. If not,
tell the debugger to reprint it:
db> panic
panic: from debugger
cpuid = 0
Uptime: 2m38s
panic: _mtx_lock_sleep: recursed on non-recursive mutex ahc_lock @ /usr/src/
sys/cam/cam_periph.h:182
...

The debugger repeats the panic message for your convenience.
You could also ask the debugger for more information about the panic:
db> trace
Tracing pid 12 tid 100002 td 0xc2117c00
kdb_enter(c0a908ed,0,3e9,c0b9fe2c,0,...) at kdb_enter+0x32

642

C h ap te r 2 1

panic(c0a8f7ce,c0a54565,c0a4ba56,b6,c228ca30,...) at panic+0x124
_mtx_lock_sleep(c228ca30,c2117c00,0,c0a4ba56,b6,...) at _mtx_lock_sleep+0x47
_mtx_lock_flags(c228ca30,0,c0a4ba56,b6,780,...) at _mtx_lock_flags+0xef
...

This might go on for several pages. If you have a serial console, be certain
to capture it all.
Before you reboot the machine, dump the kernel.
db> continue
Physical memory: 243 MB
Dumping 62 MB: 47 31 15
Dump complete
= 0xf

FreeBSD counts down how much memory remains to be dumped and
reboots the system when finished.
If you don’t want to dump, you can just reboot the machine:
db> reset

Your server will reboot and return to service.
If you have set KDB_UNATTENDED, the machine will reboot without any of
this, just as if you had entered continue.

Using the Dump
If you’re a kernel developer, this is where you stop listening to me and rely
upon your own debugging experience. If you’re a system administrator, however, you probably don’t know enough about C and kernel internals to have
any real hope of debugging a complicated kernel issue. Therefore, we’ll
focus on extracting enough information to give a developer a good shot at
identifying the problem.
After a dumped panic, you’ll find the files vmcore.0 and info.0 in /var/crash.
(Each subsequent crash gets a consecutively higher number.) The info.0 file
contains basic information about the dump, such as the partition the dump
came from, the date, and if the dump succeeded. The vmcore.0 file is the
memory image of the kernel at the time of the crash. You have a complete
record of what the kernel was thinking at the time it panicked.
If your system crashes frequently, crash dumps can grow to fill your /var
partition. If you have multiple instances of a certain type of crash, you only
need one dump of that type. Crash dumps from older versions of FreeBSD
are probably useless and can be deleted.

Getting a Backtrace
Go to the directory containing the kernel that panicked. This might be your
currently running kernel, or perhaps a test kernel. The kernel directory has
two files for every module, one named after the kernel module and the other
Sy st em ( an d Sy s ad mi n) Pa n ics a n d C r as h es

643

with the word .symbols appended. The .symbols files are the debugging images.
Once you’re in the kernel directory, run kgdb(1) giving it the kernel.symbols
filename and the full path to the vmcore for your crash:
# cd /boot/kernel.panicked/
# kgdb kernel.symbols /var/crash/vmcore.0

kgdb(1) prints a synopsis of gdb(1)’s copyright and license (kgdb(1) is
actually a modified gdb(1) specifically tweaked for debugging kernels). Then
you’ll see a copy of the original panic message and finally get a kgdb prompt.
Here, at long last, you can get the requested backtrace.
(kgdb) backtrace
#0 doadump () at pcpu.h:195
#1 0xc048c629 in db_fncall (dummy1=-875263368, dummy2=0, dummy3=524358,
dummy4=0xcbd489e4 "ÐèHÀ") at /usr/src/sys/ddb/db_command.c:486
#2 0xc048cb95 in db_command_loop () at /usr/src/sys/ddb/db_command.c:401
#3 0xc048e305 in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_main.c:222
#4 0xc0772426 in kdb_trap (type=3, code=0, tf=0xcbd48b88)
at /usr/src/sys/kern/subr_kdb.c:502
#5 0xc09faa2b in trap (frame=0xcbd48b88) at /usr/src/sys/i386/i386/trap.c:620
#6 0xc09e053b in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#7 0xc07725a2 in kdb_enter (msg=0xc0a908ed "panic") at cpufunc.h:60
#8 0xc074b954 in panic (
fmt=0xc0a8f7ce "_mtx_lock_sleep: recursed on non-recursive mutex %s @
%s:%d\n") at /usr/src/sys/kern/kern_shutdown.c:547
#9 0xc07400b7 in _mtx_lock_sleep (m=0xc228ca30, tid=3255925760, opts=0,
file=0xc0a51887 "/usr/src/sys/dev/aic7xxx/aic7xxx_osm.h", line=203)
at /usr/src/sys/kern/kern_mutex.c:311
#10 0xc07402df in _mtx_lock_flags (m=0xc228ca30, opts=0,
file=0xc0a51887 "/usr/src/sys/dev/aic7xxx/aic7xxx_osm.h", line=203)
at /usr/src/sys/kern/kern_mutex.c:187
...

This is a complete list of everything the kernel did, in reverse order. Line 0
shows that the debugger called the doadump function, which performs the crash
dump to disk. Immediately below that we see assorted debugging calls. Line 8
includes our panic message and the word panic itself; this is where our server
started turning belly-up.
A panicking kernel will call a function called panic and sometimes trap.
You’ll see variants on these two words, such as db_trap, kdb_trap, calltrap, and
so on, but we’re only interested in the plain unadorned trap or panic call.
Search through your kgdb(1) output for either of these functions. In our
previous example, line 8 includes a call to panic. Lines 3, 4, and 5 contain
functions that the panic helper calls when attempting to figure out exactly
what happened and what to do about it.
Line 9 shows what happened immediately before the system decided to
panic in line 8. Line 9 reads:
#9

644

C h ap te r 2 1

0xc07400b7 in _mtx_lock_sleep (m=0xc228ca30, tid=3255925760, opts=0,
file=0xc0a51887 "/usr/src/sys/dev/aic7xxx/aic7xxx_osm.h", line=203)
at /usr/src/sys/kern/kern_mutex.c:311

The hex numbers don’t mean much to me, but this sure looks a lot like
our panic message. FreeBSD got confused while executing code from line 203
of /usr/src/sys/dev/aic7xxx/aic7xxx_osm.h. This gives the developer a pretty
good idea of where to start looking for the problem. We can look a little
closer, however. Enter the up command and the line you’re interested in:
(kgdb) up 9
#9 0xc07400b7 in _mtx_lock_sleep (m=0xc228ca30, tid=3255925760, opts=0,
file=0xc0a51887 "/usr/src/sys/dev/aic7xxx/aic7xxx_osm.h", line=203)
at /usr/src/sys/kern/kern_mutex.c:311
311
KASSERT((m->lock_object.lo_flags & LO_RECURSABLE) != 0,

Here we see the actual line of code that was compiled into the panicking
code. At this point, I must surrender you to a kernel developer’s tender mercies. Including this in a problem report will save a round of email, however,
and it’s quite possible that a developer will just say “Oops!” and send you a fix
with no further troubleshooting. Use the quit command to leave kgdb(1):
(kgdb) quit

If you build another kernel, be certain to save the panicked kernel for
further debugging. You can only debug a kernel dump with the kernel that
panicked.
The above is a pretty good start on a problem report. You can see exactly
where the system broke and what line of code it broke on. Interested developers would quite probably write you back and tell you other things to type at
the kgdb prompt, but you are well on your way to getting the problem solved
and helping the FreeBSD folks squash a bug.

vmcore and Security
The vmcore file contains everything in your kernel memory at the time of the
panic, which might include sensitive security information. Someone could conceivably use this information to break into your system. A FreeBSD developer
might request a copy of the vmcore file and the bad kernel for many legitimate
reasons; it makes debugging easier and can save countless rounds of email.
Still, carefully consider the potential consequences of someone having this
information. If you don’t recognize the person who asks or if you don’t trust
him, don’t send the file!
If the panic is reproducible, however, you can cold-boot the system to
single-user mode and trigger the panic immediately. If the system never starts
any programs containing confidential information and nobody types any
passwords into the system, the dump cannot contain that information. Reproducing a panic in single-user mode generates a security-information-free,
sanitized dump. Boot into single-user mode and then run:
# mount -ar
# /etc/rc.d/dumpon start
# command_that_panics_the_system
Sy st em ( an d Sy s ad mi n) Pa n ics a n d C r as h es

645

The first command mounts the filesystems as read-only, so that you won’t
have to fsck(8) yet again after a panic. The second command tells FreeBSD
where to put a dump. Finally, run the command that triggers the panic. Triggering the panic might require more than one command, but this should get
a clean dump for you in most cases. If your panic requires that you load
confidential information into memory, that information will be present in
the dump.

Submitting Problem Reports
You might argue that I should have included this in Chapter 1 on “Getting
More Help.” Problem Report (PR) sounds impressive, doesn’t it? A problem
report is not where you say that you have a problem, however, it’s where you
prove that FreeBSD has a problem. Yes, I said prove. It’s not that FreeBSD is
innocent until proven guilty, but for you to submit a proper problem report,
you must substantiate your claim. Problem reports filed without evidence
will be closed with a terse response such as “not a bug” or “useless PR.” The
proper forum to cry for help is a search engine, or perhaps even FreeBSDquestions@FreeBSD.org.
You can also file a PR to submit an improvement to the FreeBSD Project.
The key word here is improvement, not wish. This sort of PR needs actual
code attached along with a history of its testing and any other necessary
information.

W H A T I S N O T A P RO B L EM RE P OR T ?
Any variation on “I don’t know what I’m doing” doesn’t belong in a problem report.
This includes “FreeBSD doesn’t work the way I think it should” or “Something bad
happens when I do something dumb.” If you break your arm, the hospital has to take
you no matter what sort of foolish thing you did, but the FreeBSD team is much more
choosy. Similarly, any variation on “The software included in FreeBSD isn’t at the
version I would like” is not a problem report. FreeBSD doesn’t update to the latest and
greatest compiler the second it becomes available, for example, and might never
update to that particular compiler. There’s always a reason for these decisions, and
opening a PR to whine about them will get you nothing but a comment like “Install it
from ports” and an immediately closed PR.

Finally, problem reports are collaborative. By filing a PR, you are indicating a willingness to work with the FreeBSD developers to resolve your
issue. This might mean applying a patch, or trying a different command, or
running debugging commands and sending the output to the developers.
Filing a PR and expecting an answer like, “Fixed, go do this” is unrealistic.
If you are not willing to work with the development team, don’t open a PR.
Of course, including the proper information in a PR helps resolve the issue
much more quickly. If you can include everything the developer needs to
know in your initial report, you’ll get a much more prompt answer.
646

C h ap te r 2 1

Repeat after me: “Free software. Donated support time.” Remember
this when your RAID card device driver makes your hard drives spin to a
conga beat.
Finally, it’s best if you’re running a recent version of FreeBSD when filing
a PR. Someone will look at a problem report based on a version of FreeBSD a
release or two back, but nobody’s going to look at a PR for FreeBSD 2.2-stable.

Before Filing a PR
Ideally, you won’t ever have to file a problem report. Not only is a proper
problem report a lot of work for you, it’s a lot of work for the developers. The
FreeBSD Project has a mailing list dedicated to assessing the PR database and
guiding problem reports to the likely owners. While sending an email to a
FreeBSD mailing list announces your woes to thousands of people, opening
a PR announces your woes to thousands of people and demands that they
handle virtual paperwork for you. Before filing a PR, be absolutely certain
that both you and the FreeBSD Project need it.
First, treat your issue as a general problem and look through the usual
FreeBSD resources. Review the FAQ and the Handbook. Search mailing list
archives and the Internet for other people who have had this problem. Check
the problem report system for an already open PR on this. Then ask on
FreeBSD-questions@FreeBSD.org if anyone else has seen this behavior. Is this
expected, or should you open a PR? The questions people ask about your
problem will be invaluable in troubleshooting your problem and in creating
your problem report.
Before starting your PR, gather every bit of information that might possibly
be helpful. This includes:
Verbose boot output
System version
Custom kernel configuration (if any)
Debugging output from the problem
Can you reproduce this problem? For a developer to actually investigate
the problem, you need a reproducible test case. If your server starts singing
show tunes at 3 AM, that is a problem. If it only happened once and you can’t
reproduce it, you’re best served by keeping your mouth shut so people don’t
think you’re a loony. If it happens whenever you run a particular combination
of commands on certain hardware, however, the matter can be verified and
investigated, so that either the problem is resolved or someone offers your
server a recording contract.
FreeBSD tracks problem reports with GNATS, a long-standing bug
tracking system. While newer and flashier systems exist, none of them meets
the FreeBSD Project’s requirements. Before you submit a problem report,
take a look at the existing PR database to see what’s included in existing
problem reports. You can search the PR database at http://www.freebsd.org/
support.html. Does your problem resemble any problems in the PR database?
Are there any comments on those PRs? How about any debugging output?
Sy st em ( an d Sy s ad mi n) Pa n ics a n d C r as h es

647

Can you reference another PR in yours, saying that it might be related? Or,
does another PR exactly match your issue, and can you use any information
in there?
Assuming you get this far, you might actually need to file a problem
report. Let’s see what not to put in it.

Bad PRs
The easiest way to understand a good PR is by reading some bad ones and
identifying what qualifies them as such. While bad PRs are easy to find, I
picked one and rewrote it so as to not embarrass anyone (with my luck, the
original author is reading this right now!). I came across this problem report
a few days ago:
When I boot the FreeBSD 6.2-REL ISO image, I can’t get past the
“Welcome to FreeBSD!” options screen. The boot menu is stuck,
and each time the screen refreshes it stays at 10. It doesn’t matter
what I press, the system never boots. If I press a whole bunch of
buttons, I eventually get a kernel panic. The same ISO image
launches in VMWare, the countdown appears, and I can install it
to a partition on the disk. I’ve also tried enabling/disabling USB
keyboard legacy support in BIOS without success.

The PR includes the model number of the motherboard, keyboard, and
mouse. The instructions on repeating the problem suggest booting the ISO
image with a similar configuration of hardware.
First off, the reader obviously has a problem with FreeBSD. It might even
be that FreeBSD has a problem. I have no doubt that the system fails at boot
exactly as advertised. But there’s no evidence and no diagnostic information.
The reproduction process is not very useful; if every 6.2 installation CD caused
this behavior on common hardware such as this, the development team would
never have signed off on the release.
Including the hardware make and model is not as useful as you might
think. Vendors occasionally change chipsets in a piece of hardware without
changing the model number. The verbose boot information identifies the
actual hardware in the machine in a way that the model number never can.
If I experienced this behavior, I would first try burning a second CD.
Perhaps the first write of the disk was bad. If the behavior persisted, I would
download a FreeBSD 6.1 ISO. If the 6.1 ISO also failed, I would ask on the
-questions@ mailing list for “further advice before I file a PR.” If I could install
FreeBSD 6.1, but not 6.2, I would include the kernel and verbose boot information from the 6.1 install in my problem report. I would also include a link
to the mailing list archive of any discussion on my problem.
As you might guess, nobody has followed up on this problem report. Your
goal is to file a problem report that is so complete and compelling that the
developers will want to work on it. Show that you are easy for a volunteer to
work with.
The FreeBSD FAQ includes a joke by Dag-Erling Smørgrav: “How many
-current users does it take to change a light bulb?” The answer is one thousand,
one hundred, and sixty nine, and includes “three to submit PRs about it, one
648

C h ap te r 2 1

of which is misfiled under doc and consists only of ‘it’s dark.’” If your PR
amounts to “it’s dark,” it’s a bad problem report.

Good PRs
You can enter PRs using the web interface, but I find that rather clunky. If you
like graphic interfaces, you might try gtk-send-pr (/usr/ports/sysutils/gtk-send-pr).
I always use send-pr(1) for problem reports. I find that send-pr(1) lets me
consider my problem report carefully, even coming back to it a day later after
I’ve had a chance to think about it. Begin by printing a blank PR form with -P
and placing it in a file:
# send-pr -P > problemreport.txt

Now open the problem report in a text editor. Here’s a sample of the template:
To: FreeBSD-gnats-submit@freebsd.org
From: Michael W Lucas 
Reply-To: Michael W Lucas 
Cc:
X-send-pr-version: 3.113
X-GNATS-Notify:
>Submitter-Id:
current-users
>Originator:
Michael W Lucas
>Organization:

>Confidential:
no 
>Synopsis:

>Severity:
<[ non-critical | serious | critical ] (one line)>
>Priority:
<[ low | medium | high ] (one line)>
>Category:

>Class:
<[ sw-bug | doc-bug | change-request | update | maintainerupdate ] (one line)>
>Release:
FreeBSD 7.0-CURRENT i386
>Environment:
System: FreeBSD pesty.blackhelicopters.org 7.0-CURRENT FreeBSD 7.0-CURRENT #0:
Wed May 21 13:29:50 EDT 2008 mwlucas@pesty.blackhelicopters.org:/usr/obj/usr/
src/sys/GENERIC i386

>Description:

>How-To-Repeat:

>Fix:


Both the web interface and the text form have the same fields; the
difficulty lies in filling out the form correctly. The blank template also includes
several lines beginning with SEND-PR, which send-pr(1) deletes from the form
upon submission. Similarly, any text in angle brackets (< and >) is a comment,
and send-pr(1) deletes it. The template lists legitimate choices inside the
angle brackets. Let’s go over it, one chunk at a time.
Sy st em ( an d Sy s ad mi n) Pa n ics a n d C r as h es

649

The discussion below applies mostly for system panics, the most frustrating
type of problem report. FreeBSD also uses the PR database for submission
of new ports and new features, however. Fill out these sorts of submissions
appropriately for the submission, as per the documentation at the FreeBSD
website.
To:

This field is the email address of the FreeBSD PR database. Leave that
alone.
From, Reply-To:

Make sure these lines contain valid email addresses. GNATS will reply to
your email address with your PR number, and interested developers will
try to contact you here.
cc:

You might want to copy someone else on your PR. My boss feels better
when he is copied on support requests for issues that he thinks are critical, for example.
X-send-pr-version, X-GNATS-Notify, Submitter-Id:

Leave these fields unchanged.
Originator:

This is your name, generally pulled from the system environment. While
some folks use handles on the Internet, I recommend using your real
name. It’s difficult to treat a serious problem with the attention it deserves
if it comes from “Mr. Ticklebottoms.”
Organization:

This field is not used, so you can put here whatever you like.
Confidential:

FreeBSD’s GNATS is a public database, and you should never enter confidential information in a problem report. Changing this will result in
send-pr refusing to submit the PR. If you believe that you have discovered a bug with security implications, contact security-officer@FreeBSD.org.
Don’t contact the security officer just because you want to list your root
password in the PR, however.
Synopsis:

This is the most critical single line field in the problem report. Give a brief,
one-line description of the problem. Developers use this field to decide if
they are interested in a PR. A synopsis like, “FreeBSD sucks and blows
at the same time!” will get closed or ignored, while a subject like, “Panic
under heavy I/O load, backtrace attached” has a good chance of attracting skilled attention. If you have a patch to fix the problem, put the word
PATCH at the beginning of the synopsis and someone will look at that very
quickly. (They might not accept the patch, mind you, but they’ll look
at it.)

650

C h ap te r 2 1

Severity:

This field can be either non-critical, serious , or critical. Pick a reasonable one. Critical issues affect every user of FreeBSD. Serious problems
affect users of a particular device driver or in a common environment.
Everything else is non-critical. If you get a reputation listing trivial reports
as critical, you’ll find yourself being ignored fairly quickly. The FreeBSD
Project works on the honor system, and reputation counts for more
than you might think.
Priority:

You can enter either low, medium, or high. The priority field has been so
badly abused that it is now meaningless and ignored. I recommend either
low or medium.
Category:

You must enter one of the options listed in the comments area at the top
of the PR form. The categories include things such as ports, www, docs,
bin, kern, and misc. Pick a category where your problem fits in. I normally
use kern for kernel problems, bin for userland problems, ports for issues
with the Ports Collection, or doc for doc problems.
Class:

This is a type of problem. If you can crash a program or the system, it’s
a sw-bug.
Release:

send-pr(1) automatically puts your system type here. If you’re filling out
a PR on a system other than the one you’re reporting on, put the FreeBSD
release you’re reporting on here.
Environment:

Put the output of uname -a here. You can add additional information to
this field to describe other relevant parts of your machine. For example,
if the machine is a heavily loaded web server, mention that. If you have
a snippet of a configuration file that reproduces the panic, put it here.
The Environment section should be very brief. You might wish to obscure
the hostname for security reasons.
Description:
The Description field is a free-form, plaintext section for you to go into

detail about the issue. Don’t rant or rave; just describe what happens.
Include all your error messages. If it’s possible to get a crash dump, do
so and include the debugger output. Also, include your kernel configuration and the boot messages from a verbose boot.
How-To-Repeat:

Give brief instructions on how to replicate this error. For some PRs, this
can be very short: “Read FreeBSD-questions for a week and see how often
this is asked” is a perfectly legitimate How-To-Repeat for doc changes. More
technical problems require detailed information.

Sy st em ( an d Sy s ad mi n) Pa n ics a n d C r as h es

651

Fix:

The most important part of the PR goes under Fix. If you have a patch
that fixes the problem, put it here. If you have a way to work around the
problem, put it here. Anything you’ve discovered about how to solve
the problem goes here. Sometimes, the most unusual fix or condition
provides the vital clue for the solution. If you have no idea of how to
solve or work around this issue, leave Fix blank. Putting random thoughts
or speculation into the Fix field does not improve the quality of the PR.

A Sample PR
Given the above, let’s fill out a sample problem report about the crash I
debugged in the first half of this chapter. It’s a real crash that I encountered
while attempting to write Chapter 18, so I actually did need a solution. Immediately after updating my system to FreeBSD-current as of July 4, 2007, I started
getting crashes when accessing my external SCSI array. I had planned to use
this cage to demonstrate GEOM modules, and I needed it working right away.
The panic message seems to indicate a problem with the ahc0 driver. I fill
out my name and email address in the spaces at the top and then go to the
more interesting parts of the form:
>Submitter-Id:
>Originator:
>Organization:
>Confidential:
>Synopsis:
>Severity:
>Priority:
>Category:
>Class:
>Release:

current-users
Michael W Lucas
none
no
panic: _mtx_lock_sleep: in aic7xxx_osm.h (with backtrace)
serious
medium
kern
sw-bug
FreeBSD 7.0-CURRENT i386

The Submitter-Id remains unchanged. I am the originator, and anyone
who mentions the word organization in reference to me gets laughed at. This
PR is not confidential.
The Synopsis is difficult. I put in the first part of the panic string and the
location of the panic, and mention that I have a backtrace. That’s about all
that will fit on a single line.
As this error affects a single device, but causes a panic, it’s serious. I assign
it medium priority, just because I can’t bring myself to call it low and yet the
developers won’t think it’s critical.
This is a kernel panic, so its category is kern. I can crash the kernel, so it is
in the class of sw-bug.
The Environment gives me a little more flexibility. I start with the output of
uname -a and then talk about what makes my environment special:
This is a standard i386 system with an external SCSI array attached.
Verbose dmesg is attached.

652

C h ap te r 2 1

The Description is my opportunity to tell the FreeBSD team what the
problem is:
This system performs fine on an April 22 2007 kernel. I updated to -current on
July 4, 2007, and immediately started seeing these panics. Upon further
testing, I found that I could read data just fine from the SCSI drives, but
that attempting to write to the drives resulted in this panic. The panic
message and backtrace are attached.
How-To-Repeat is where I give step-by-step instructions on how to trigger

this bug:
/dev/da7 is my test drive. I can run:
# fdisk /dev/da7
and read the slice table as often as I wish without trouble. A simple write
operation such as
# fdisk -BI /dev/da7
panics the system. This does not happen every time, but if I run this command
a few times in a row, it regularly and reliably panics. My record is five
successful "fdisk -BI /dev/da7" runs, one after the other in quick succession,
before the sixth panics the system.

This succinctly describes my problem, without screaming that I have a
deadline and really need to get Chapter 18 written before my legion of
devoted fans 2 arrive at my door with pitchforks and torches demanding their
book. I have clear instructions on how to reproduce the problem. I also must
include the verbose boot messages and the kernel backtrace in this section.

Submitting the PR
Your system must have a working outbound email system to send a PR. If you
cannot send email from a machine, do not use send-pr(1) on it. Save your
problem report and related files on a machine that can send email and use
send-pr(1) there. Use send-pr(1)’s -f flag to specify a completed PR template:
# send-pr -f prreport.txt

Your PR is now submitted. Nothing to it, once you have all the information!

After Submitting the PR
A few hours after submitting the problem report, I received an email stating
that I was now the proud submitter of PR kern/114489. No matter how you
submit the PR, you’ll receive a confirmation email with a PR number within
several hours. Any response you make to that email will automatically attach
to that PR, so long as you don’t change the subject. You can submit patches
and responses from any computer with a working email system.

2
Three fans might not qualify as a legion, and Amanda is more of a sarcastic fan than a devoted
one, but I take what I can get.

Sy st em ( an d Sy s ad mi n) Pa n ics a n d C r as h es

653

Similarly, GNATS includes any response a developer makes to your PR in
its history of the problem report. Now that your PR is in the FreeBSD database, it’ll be tracked forever. That is not a guarantee that anyone will act
on your issue or that your problem will be solved; it’ll simply be recorded,
publicly, forever.
If it seems that your PR has been forgotten for several weeks, drop a
friendly note to the appropriate mailing list with your PR number and a brief
explanation of the issue and why it’s important. Since FreeBSD is a volunteer
effort, it’s quite possible that something happened to the person who would
normally handle that sort of PR. While many FreeBSD developers are professional programmers, for many of them this is still a hobby that must take
a backseat to sick kids or the big work deadline. If nothing else, you can contact one of the commercial support firms listed on the FreeBSD website.
Within a couple of days, I had responses from developers offering patches
to fix the problem. The first round of patches exposed bugs elsewhere, but
patches to resolve those followed quickly. As you can see by the presence of
Chapter 18, I was able to get a solution to my problem.
A surprising number of difficult PRs are closed quickly when they include
the proper information. Just remember that the FreeBSD folks do this out of
love for their work, not because they must. They want to produce quality code,
which is stronger motivation than a paycheck. If you can help them produce a quality product, they will happily work with you.
As of late 2007, I’ve submitted several dozen problem reports. Most have
been solved and/or committed, and then closed. The odd ones out were
mostly trivial goofs on documentation that lives under /usr/src/contrib, an area
where the FreeBSD Project specifically disavows responsibility for minor
fixes. If a doofus like me can get over 90 percent of his problem reports
successfully closed, anyone can. Be warned, however; if you submit enough
correct patches, you’ll find that the committers you work with will start to
talk about you behind your back. Eventually they will grow tired of acting as
the secretary for your high-quality work and will offer you commit access. If
you refuse, they will offer more insistently. Don’t worry, becoming a committer
is not that painful. The rumors that the FreeBSD Project initiation ritual
involves a group of Danes with axes behind a bikeshed are completely
untrue. Mostly.
Keep filing good problem reports anyway; that’s the only way FreeBSD
improves!

654

C h ap te r 2 1

AFTERWORD

If you’ve made it this far, you now know
how to manage and use FreeBSD as a
platform for just about any server task. You
might have to learn how new programs work, but
you know enough about the operating system to make
FreeBSD support just about anything. Congratulations!
FreeBSD is a wonderful, flexible platform, capable of assuming just about
any role in your network. To wrap things up, I’d like to talk briefly about
some other aspects of FreeBSD.
We’ve talked about the obvious parts of FreeBSD throughout this book:
the programs, the kernel, the features, and so on. One thing we haven’t
talked about much is the community that creates all of this.

The Community
The FreeBSD community includes computer scientists, experienced programmers, users, system administrators, documentation writers, and just about
anyone who is interested in the system. They come from all walks of life, from

all over the world, and have education levels ranging from high school to
post-doctoral. I personally have had dealings with FreeBSD users from every
continent and most large islands on the planet.1 Nationality simply isn’t
important, nor is race, color, or creed.
Some are computer scientists. Some work at ISPs or manufacturing firms.
Some are physicians, and some work as clerks in video rental shops. At one
point I worked closely with a brilliant developer who turned out to be too
young to drive. Oddly enough, time zone is important, but only because it
impacts the ability of developers to communicate with one another. Since
most of the community’s interaction is online, the only things that represent
you are your words and your work. These are the people who improve FreeBSD
and drive it forward, making it more than a collection of ones and zeroes and
more than just a way to serve websites.
One of the interesting things about the FreeBSD community is that it has
developed methods for coping with changes in its leadership. Many open
source projects have a single leader or a small static leadership team. When
those people decide to move on, their project is probably over. Someone else
might branch or fork that project, but the original community usually fragments. The people who created the FreeBSD Project have mostly moved on
to other things, but the community has grown other leaders. After three generations of leadership, FreeBSD as a project has demonstrated a resilience to
leadership changes that is almost unique in the open source world. Today’s
FreeBSD leaders take a very active interest in their own replacements, mentoring and coaching those junior members of the community who seem most
likely to become the leaders of the 2010s. What’s more, we always welcome
those who came before us, both in the CVS repository and at the bar.

Why Do We Do It?
Each person works on FreeBSD for his own reasons. A tiny portion of people
are paid to improve the code, either by corporations dependent on FreeBSD
or government agencies such as DARPA. Many developers work on FreeBSD
as a hobby, so they can program things more correctly than they are allowed
to at their day job. How many of you have had work projects completed less
successfully than you would like because of a deadline? FreeBSD’s deadlines
are announced months or years in advance, and developers set their own
work habits and their own levels of contribution.
Many of us are not software developers, but work on some other part of
FreeBSD instead. Some write documentation, some design the websites, some
just hang out on mailing lists and answer user questions. People spend hours
and hours working on FreeBSD-related matters. Why? I can assure you that
the royalties on this book will not come close to compensating me for the

1
Yes, this includes a man who took his FreeBSD laptop on a cruise to Antarctica. With his wife.
On their 20th anniversary. He did not mention whether his wife threw the laptop overboard or
not. If she did, I imagine he just bought a better one when they got back.

656

A ft erw ord

evenings I could have spent with my family. I’m writing this Afterword on a
porch on a cliff in Ontario, Canada, overlooking Lake Erie. Everybody else
has gone out to walk the trails and chase butterflies, hopefully not over that
same cliff. Why would I choose to do this instead?
We do it for the satisfaction of creating something useful to the rest of
mankind and to return some of what we’ve been given.
You’re free to simply take what FreeBSD offers and do whatever you
wish with it. After a while of doing this, I found that I wanted to return
something to the community. This is how the community grows, and a
growing community means that FreeBSD will prosper.
If you want some of that satisfaction yourself, there’s a place for you too.

What Can You Do?
If you’re interested in supporting the FreeBSD Project, for whatever reason,
there’s a space for you in every part of the Project. Ever since I started using
FreeBSD back in 1996, every so often someone posts on a mailing list, “I’d
like to help, but I can’t code.” (In fact, I think I’ve sent that message myself!)
The standard response to these messages is silence. If you’ve already decided
that you can’t help, you’re right; you can’t. Once you decide that you can
help, however, you can.
Nobody denies that some high-visibility programmers are the celebrities
of FreeBSD. Many of these people have impressive skills, and most of us could
never dream of being the next Robert Watson or Jordan Hubbard. However,
even if you can’t program your way out of a paper bag, you can still help. Turn
it around and ask a different question.
Don’t ask what FreeBSD needs. You can’t provide that, unless you have a
large bank balance begging for a charitable cause to belong to. (If you have
piles of cash, however, the FreeBSD Foundation would be happy to help you
out.) Don’t say, “Wouldn’t it be cool if FreeBSD did such-and-such” if you
can’t create that yourself. What do you do? Any large organization needs
many different people, and whatever skills you have today are useful to the
FreeBSD Project.
I know people who run FreeBSD-related websites and provide valuable
resources such as http://bsdforums.com and http://www.freshports.org. These
people are respected contributors.
I know FreeBSD users who are web designers and documentation writers.
They contribute content to the website. Do enough of that, and you’ll be a
committer.
I know FreeBSD users who contribute to user mailing lists, answering
questions. Do enough of that and you’ll want to update the FAQ. Submit
enough FAQ updates, and you’ll become a committer.

Af te rwo rd

657

I write copiously, and passably well. I wrote some updates for the FAQ,
and now this book. The FAQ updates made me a committer, although I
would say that this book is the greater contribution. The mere sight of code
I’ve written makes small children cry and old ladies make the sign to ward off
the evil eye, but the FreeBSD folks welcome me and treat me as a partner
simply because I do the work.
What is it that you do? What do you enjoy doing? Leverage that skill. It
will be appreciated.

If Nothing Else . . .
If you truly have no useful skills, and you have no other ideas, reread this book.
Read the documentation on the FreeBSD website. Subscribe to FreeBSDquestions@FreeBSD.org and help other users. Many people started this way.
I encourage you to direct people to existing information resources whenever possible. When someone asks a question in the FAQ, guide them there.
If the question has been asked before, suggest that they search the mailing
list archives. Teaching people to help themselves is the most effective use of
your time-—not just in FreeBSD, but out in the world as well. As the old saying
goes, teach a man to fish and you can sell him fishhooks. After answering
questions for a while, you’ll see the shape of the Project’s needs. One of
those needs will almost certainly match your skills.

Getting Things Done
Here’s the big secret of success in FreeBSD: Everything that it contains is there
because somebody saw a need that he could fill and did something about it.
NetBSD and FreeBSD started when a bunch of 386BSD patchkit users got sick
of waiting for the next patchkit release. I didn’t ask for permission to write
this book before starting. The fine folks over on bugbusters@FreeBSD.org don’t
wade through the PR database for fun; they do it because they think it’s
important enough to spend their time on. (Actually, if you’re a programmer,
going through the PR database and looking for problems you can solve is
one of the best contributions you can make.)
Once you have an idea, search the mailing list for discussions about it.
Many projects are suggested but never implemented. If someone’s previously
brought up your idea, read the archived discussion. If the idea met with general
approval, you can assume that you’ll get the same reaction. If nobody’s working
on the project any more, get to work! The FreeBSD team will be perfectly
happy if the first time they hear from you is an email message saying, “Hi, my
patches for implementing such-and-such feature in the installer are available
at this URL.”

658

A ft erw ord

Whatever you do, don’t post a question to the mailing list that says, “Why
doesn’t someone else do the work for X?” Most of these suggestions fall into
three categories: obvious (“Hey, wouldn’t it be cool if FreeBSD ran on mainframes?”), foolish (“Why isn’t there a kernel option BRINGMEACOLDBEER?”), or
both (“Why not support my Sinclair ZX80?”). In any of these cases, the person
asking is both unqualified to perform the work himself and claims to be helpless to support others who could do the work. All these suggestions do is annoy
people and waste bandwidth.
In short: Shut up and work. Do what you can do, and do it well, and people
will appreciate it. Programmers can help by picking an idea off the FreeBSD
Project Ideas List or jumping into the bugbusters@FreeBSD.org mailing list and
wading through the PR slurry. Nonprogrammers can help by picking a hole
and filling it in. You might become a leader in the Project, or you might be
known as “That dude who hangs out on -questions@ and helps people with
ACPI.” All are absolutely vital. Your help makes FreeBSD prosper and grow.
Stick around long enough, and you just might find yourself one of FreeBSD’s
new leaders.
I look forward to seeing you on the mailing lists.

Af te rwo rd

659

SOME INTERESTING
SYSCTL MIBS

This appendix is a dictionary of some useful
sysctl MIBs, as well as some sysctl MIBs that
are no longer useful but have long been discussed in FreeBSD forums. Chapter 5 discusses
tools for manipulating sysctl MIBs. When a MIB is discussed in detail elsewhere in the book, a reference is
made to the appropriate chapter. Your system certainly
has many more sysctls than these, but described here
are those that I’ve tripped over repeatedly.
Remember, thoughtless use of sysctls can easily damage or destroy a
working system. For example, if you set a limit on resource usage below the
value needed at that moment, you can crash a process or a system. Be sure
you understand the implications of what you’re doing before setting a sysctl.
If you don’t understand it, that’s what test systems are for.
For each sysctl, I list a sample value from my test system so you can see
what the value should look like.

Each sysctl description indicates when the sysctl can be changed. Some
sysctls can be changed at any time, others can be changed only at boot. A few
are read-only.
For each sysctl, I also mention if it is a variable, toggle, or tunable. Variables
have a wide variety of ranges. Toggles have only two legitimate values, 1 (the
described service is on) and 0 (the service is off). Tunables are set in the boot
loader. (Strictly speaking, a boot tunable is different from a sysctl, but you set
the sysctl by setting the tunable.)
kern.osrelease:

7.0-RELEASE

Read-only. This displays the running version of FreeBSD, such as
7.2-release, 8.0-current, and so on.
kern.maxvnodes:

100000

Run-time variable. The maximum number of vnodes (virtual filesystem
nodes) the system can have open simultaneously.
kern.maxproc:

6164

Boot-time tunable. The maximum number of processes the system can
run at any one time.
kern.maxfiles:

12328

Run-time variable and boot-time tunable. The maximum number of files
that the system can have open for reading or writing at any one time.
kern.argmax:

262144

Read-only. The maximum number of characters you can use in a single
command line. You might run up against this in some unusual circumstances. If you do, please see xargs(1).
kern.securelevel:

-1

Run-time variable. The current kernel security level. See Chapter 9.
kern.hostname:

bewilderbeast.blackhelicopters.org

Run-time tunable. The hostname, as set in /etc/rc.conf.
kern.posix1version:

200112

Read-only. This gives the version of POSIX the kernel complies with. If
you need to change this, please feel free to contribute code to support
other POSIX versions.
kern.ngroups: 16

Read-only. The maximum number of groups any one user may belong to.
kern.boottime:

{ sec = 1187744126, usec = 946476 } Tue Aug 21 20:55:26 2007

Read-only. This gives the time the system booted, in both epochal time
and as a human-friendly date.
kern.domainname:

Run-time variable. This offers the YP/NIS domain name, not the TCP/IP
domain name. If you are not using YP/NIS, it will be blank.

662

A pp en dix

kern.osreldate:

700052

Read-only. This is the FreeBSD version, not an actual date. Various ports
configure themselves with this sysctl value.
kern.bootfile:

/boot/kernel/kernel

Read-only. This is the kernel file that the system booted. While you could
technically change this, don’t. Reboot on a new kernel instead. The only
time this changes on a running system is when you tell FreeBSD to install
a new kernel and move the old kernel to /boot/kernel.old, but that’s a very
special case.
kern.maxfilesperproc:

11095

Run-time variable. This is the maximum number of files a single process
can open.
kern.maxprocperuid:

5547

Run-time variable. The maximum number of processes one user ID can
run simultaneously.
kern.ipc.somaxconn:

128

Run-time variable. The maximum number of new connections the system
will accept at any one time. If you’re running a heavily loaded server, you
might need to increase this to 256 or even 512.
kern.ipc.maxpipekva:

16777216

Boot-time tunable. This is the maximum amount of kernel memory that
can be used by pipes.
kern.ipc.pipekva:

212992

Read-only. This is the current amount of memory used by pipes.
kern.ipc.shmmax:

33554432

Run-time variable. This is the maximum size of a System V shared memory segment. You might have to tune this for programs that use large
amounts of shared memory, notably databases.
kern.ipc.shmseg:

128

Boot-time tunable. This is the maximum number of System V shared
memory segments any one process can open.
kern.ipc.shmall:

8192

Run-time variable. This is the maximum number of pages available for
System V shared memory.
kern.ipc.shm_use_phys:

0

Run-time toggle, boot-time tunable. Enabling this tells FreeBSD it can
never swap out shared memory segments. This can improve performance
if you are using large amounts of System V shared memory, but by making shared memory unpageable you can hurt the performance of every
other program on your system.

Som e I n te res t in g s ys ct l M I B s

663

kern.ipc.numopensockets:

70

Read-only. This is the number of open sockets of all sorts, including
network and local domain sockets.
kern.ipc.maxsockets:

12328

Run-time variable, boot-time tunable. This is the total number of sockets
available on the system. This includes Unix domain sockets as well as
network sockets.
kern.logsigexit:

1

Run-time toggle. A program that exits abnormally usually sends a signal.
When this toggle is set, the name of the program and the exiting signal
are logged to /var/log/messages.
kern.init_path:
sysinstall

/sbin/init:/sbin/oinit:/sbin/init.bak:/rescue/init:/stand/

Read-only. init(8) is the program that actually starts running the userland.
This sysctl lists the paths where the kernel looks for init(8). If you’ve
damaged your system (say, during an upgrade gone bad), this provides
a series of backup paths the system checks for a copy of init(8). This sysctl
is also available in the boot loader as init_path.
kern.init_shutdown_timeout:

120

Run-time variable. This is the number of seconds that userland processes
have to shut themselves off during the shutdown process. After this many
seconds, the kernel shuts itself off no matter what might be running.
kern.randompid:

0

Run-time variable. If set to 0, each newly assigned process ID is one greater
than the previous. If set to a larger number, the next PID is random. As
the value of this sysctl increases, the randomness of the next PID increases.
kern.openfiles:

246

Read-only. This is the number of files currently open on the system.
kern.module_path:

/boot/kernel;/boot/modules

Run-time variable. This lists the directories where kldload(8) checks for
kernel modules.
kern.maxusers:

384

Boot-time tunable. For many years, kern.maxusers was a general-purpose
hint the system administrator could set to tell BSD and FreeBSD how much
memory to allocate for certain types of system tasks. Modern FreeBSD
systems auto-tune kern.maxusers. Any documentation that suggests changing kern.maxusers is almost certainly obsolete.
kern.sync_on_panic:

0

Run-time toggle. When FreeBSD panics (see Chapter 21), it unmounts
all disks without synchronizing them first. This results in dirty disks, as
discussed in Chapter 8. If you set this sysctl, FreeBSD will attempt to sync
the disks before shutting down. This is extremely dangerous, as you have
no way to know the condition of a panicked system. You might actually
664

A pp en dix

damage data or the underlying filesystem by attempting to sync your disks
after a panic, resulting in a sysadmin panic as well as a system panic.
kern.coredump:

1

Run-time toggle. By default, when a program crashes, FreeBSD writes
a core dump. Setting this to 0 disables core dumps. It is most useful on
embedded and diskless systems.
kern.nodump_coredump:

0

Run-time toggle. Enabling this tells FreeBSD to set the nodump flag on
core files. See Chapter 4.
kern.timecounter.choice:

TSC(-100) i8254(0) dummy(-1000000)

Read-only. This lists the hardware clocks available on your system. Most
modern hardware has four common options, in order from best to worst:
ACPI, i8254, TSC, and dummy. The numbers give relative accuracy of the
various clocks.
kern.timecounter.hardware:

i8254

Run-time variable. You can change this to use a different hardware
clock. Your choices are provided in the kern.timecounter.choice sysctl.
By default, FreeBSD uses the best available clock.
kern.timecounter.smp_tsc:

0

Boot-time tunable. If this equals 0, do not use the TSC hardware clock on
this hardware. TSC does not work correctly on most SMP systems.
kern.sched.name:

4BSD

Read-only. This is the name of the scheduler in the running kernel.
kern.sched.quantum:

100000

Run-time variable. This gives the maximum number of microseconds a
process can run if other processes are waiting for CPU time when using
the 4BSD scheduler. If you’re considering changing this, you are almost
certainly doing something wrong. The ULE scheduler does not offer this
sysctl.
kern.sched.preemption:

1

Read-only. This shows if preemption is enabled in the kernel. Preemption allows a more urgent kernel thread to interrupt a less urgent kernel
thread.
kern.log_console_output:

1

Run-time toggle, boot-time tunable. By default, FreeBSD sends console
messages to syslogd. (Console messages do not include responses to
commands typed on the console, just messages that the system logs to
the console whether someone is logged in or not.) Changing this to 0
disables this feature.
kern.smp.disabled:

0

Boot-time toggle. Disable SMP in /boot/loader.conf by setting kern.smp
.disabled to 1.

Som e I n te res t in g s ys ct l M I B s

665

kern.smp.cpus:

2

Read-only. This is the number of CPUs running on the system.
kern.filedelay:

30

Run-time variable. This dictates how often FreeBSD synchronizes file
data between the vnode buffer cache and the disk. More frequent synchronization causes increased disk load, while decreasing the synchronization frequency increases risk. Only system administrators who
thoroughly understand the buffer cache should even consider touching this.
kern.dirdelay:

29

Run-time variable. This controls how often the system synchronizes
directory data between the buffer cache and the disk. It should be set
to slightly less than the kern.filedelay sysctl. Again, only highly experienced system administrators should consider changing this.
kern.metadelay:

28

Run-time variable. This controls how often FreeBSD synchronizes filesystem metadata between the buffer cache and the disk. It should be set to
slightly less than the kern.dirdelay sysctl. Again, for highly experienced
system administrators only! Mucking with synchronization is a great way
to get a deep education in data loss.
vm.v_free_min:

3258

Run-time variable. The minimum number of pages of cache and free
memory that must be available before a process waiting on memory will
be awakened and told to run.
vm.v_free_target:

13745

Run-time variable. The minimum number of pages of combined free
and cache memory that the virtual memory manager tries to maintain
or exceed.
vm.v_free_reserved:

713

Run-time variable. If the number of pages of free memory falls below this
value, the virtual memory manager will start swapping out processes.
vm.v_inactive_target:

20617

Run-time variable. FreeBSD tries to keep at least this much memory
inactive, by making active pages inactive.
vm.v_cache_min:

13745

vm.v_cache_max:

27490

Run-time variable. The minimum and maximum desired size of the
virtual memory cache queue.
vm.swap_enabled:

1

Run-time toggle. This controls the use of swap space. If set to 0, your
system will not swap. This is most useful for diskless systems.

666

A pp en dix

vm.swap_idle_enabled:

0

Run-time toggle.
vm.swap_idle_threshold1:

2

Run-time variable.
vm.swap_idle_threshold2:

10

Run-time variable. Setting the swap_idle_enabled sysctl tells the virtual
memory manager to pull idle processes into swap more quickly than
other processes. The threshold sysctls tell the system how many seconds
to wait before considering different sorts of processes idle. The defaults
are fine for most people; just enabling vm.swap_idle_enabled should do the
trick. This can help some systems stay afloat when they’re desperately
short on memory and are swapping out entire processes.
vm.exec_map_entries:

16

Boot-time toggle. This is the maximum number of processes that can be
started simultaneously.
vfs.nfs.diskless_valid:

0

Read-only. If 0, this system is not running diskless. If non-zero, the system
is running diskless.
vfs.nfs.diskless_rootpath:

Read-only. This gives the path to the root filesystem of a diskless machine.
vfs.nfs.diskless_rootaddr:

Read-only. This gives the IP address of the root filesystem of a diskless
machine.
vfs.vmiodirenable:

1

Run-time toggle. Allows UFS to use the virtual memory system to cache
directory lookups, increasing disk performance.
vfs.usermount:

0

Run-time toggle. If enabled, users may mount filesystems on mount
points that they own. This allows unprivileged users to mount floppy
disks and CDs. See Chapter 8.
vfs.ffs.doasyncfree:

1

Run-time variable. This tells the UFS filesystem to asynchronously update
inodes and indirect blocks after reorganizing disk space. See the sysctl
vfs.ffs.doreallocblks.
vfs.ffs.doreallocblks:

1

Run-time variable. By default, FreeBSD reorganizes its disks so that files
are contiguous at all times. In effect, UFS continuously defragments
itself. Set this to 0 to disable this behavior.
net.inet.ip.portrange.lowfirst: 1023
net.inet.ip.portrange.lowlast: 600

Run-time variables. Some programs or protocols require connections to
be made from low-numbered ports, indicating that they are started by
Som e I n te res t in g s ys ct l M I B s

667

privileged processes. Remember, an outgoing TCP/IP connection uses a
port on the local system. Also remember that ports from 0 through 1023
can only be opened by root. This means that if a connection comes from
a low-numbered port, it’s theoretically running as root. (This involves
trusting that the machine initiating the connection has not been broken
in to.) FreeBSD uses ports between those given by net.inet.ip.portrange
.lowfirst (default of 1023) and net.inet.ip.portrange.lowlast (default of
600). I only know of one instance where changing this makes sense. The
IPMI firmware on some Intel NICs always intercepts UDP packets bound
for ports 623 and 664. If you don’t have one of those cards, leave this
alone.
net.inet.ip.portrange.first: 49152
net.inet.ip.portrange.last: 65535

Run-time variable. When FreeBSD allocates a random port for an outgoing connection from unprivileged processes, it uses ports between those
given by net.inet.ip.portrange.first (default of 49152) and net.inet.ip
.portrange.last (default of 65535).
net.inet.ip.portrange.hifirst: 49152
net.inet.ip.portrange.hilast: 65535

Run-time variable. Some programs specifically request a high-numbered
port for an outgoing connection. By default, this range of ports overlaps
with the ports specified by net.inet.ip.portrange.first to net.inet.ip
.portrange.last, but you can separate these ranges by using the hifirst
and hilast sysctls.
net.inet.ip.portrange.reservedhigh: 1023
net.inet.ip.portrange.reservedlow: 0

Run-time variable. This specifies the range of ports that can only be
opened by root.
net.inet.ip.portrange.randomized:

1

Run-time variable. By default, FreeBSD will allocate random ports within
the ranges allocated for that type of connections.
net.inet.ip.portrange.randomcps:

10

Run-time variable. This is the maximum number of random port allocations within a single second. If more than this many ports are needed,
FreeBSD switches to sequential port allocation to avoid consuming all
system randomness.
net.inet.ip.portrange.randomtime:

45

Run-time variable. This is the number of seconds FreeBSD will use
sequential TCP/UDP port allocations before reverting to random port
allocation.
net.inet.ip.forwarding:

0

Run-time toggle. You might want FreeBSD to act as a router, gateway, or
firewall. With this set, the system forwards packets between interfaces.

668

A pp en dix

net.inet.ip.redirect:

1

Run-time toggle. With this set, a FreeBSD system acting as a router or
gateway will send ICMP redirect packets to other systems on the network.
It has no effect if the system is not routing.
net.inet.ip.ttl:

64

Run-time variable. This is the maximum number of hops any non-ICMP
protocol may take across the network.
net.inet.ip.sourceroute:

0

Run-time toggle. This enables forwarding of source-routed packets.
Source routing is generally considered a bad idea on the public Internet.
net.inet.ip.accept_sourceroute:

0

Run-time toggle. This enables acceptance of source-routed packets. If
you don’t know what source routing is, just accept my word that this is a
really bad idea in almost all cases.
net.inet.ip.fastforwarding:

0

Run-time toggle. This sysctl greatly accelerates packet throughput on
routers and gateways. It does so by eliminating most of the sanity and
integrity checks performed on packets and by completely eliminating
any packet-filtering rules.
net.inet.ip.check_interface:

0

Run-time toggle. This provides basic antispoofing protection for your
server. It’s only useful on a router, gateway, or firewall. Most of the time,
this is better done at the firewall level.
net.inet.icmp.icmplim:

200

Run-time variable. This is the maximum number of ICMP requests the
system will answer per second. Systems that have a huge amount of ICMP
requests (mostly those running security and network tools such as Nessus,
nmap, and Nagios) will need to increase this. Set this to 0 to disable
ICMP throttling altogether.
net.inet.icmp.drop_redirect:

0

Run-time toggle. This tells FreeBSD to ignore ICMP redirect packets.
Most of the time redirects improve performance, but I have had edge
cases where ignoring redirects was the proper course of action.
net.inet.icmp.log_redirect:

0

Run-time toggle. If set, FreeBSD will log all ICMP redirects to the console. (You didn’t really need that console for anything useful, did you?)
net.inet.icmp.bmcastecho:

0

Run-time toggle. This enables responses to requests to the broadcast
address of a network. While this facilitated troubleshooting and was
originally required for standards compliance, responding to broadcast
pings was such a security problem that it’s now disabled by default on
all modern operating systems.

Som e I n te res t in g s ys ct l M I B s

669

net.inet.tcp.rfc1323:
net.inet.tcp.rfc3042:
net.inet.tcp.rfc3390:

1
1
1

Run-time toggles. These enable various TCP methods described in the
corresponding RFCs and should be perfectly safe. Some very, very old
systems might have slow or problematic throughput when communicating with your FreeBSD system with these enabled.
net.inet.tcp.sendspace: 32768
net.inet.tcp.recvspace: 65536

Run-time variable. These give the default initial buffer size for a new
TCP/IP connection. While FreeBSD allocates these buffers dynamically
as the throughput to the remote host requires, these values provide a
good starting point. While you will find advice about changing these
values in much old documentation, today they should be left alone.
net.inet.tcp.log_in_vain:

0

Run-time toggle. This enables logging any attempt to connect to a TCP
port when no program is listening on that port. On the public Internet,
this can generate a huge amount of output. I suggest enabling this on a
test system just so you can see what sort of crap flies around the Internet
these days, and then turning it off so that you can use your console again.
net.inet.tcp.blackhole:

0

Run-time toggle. By default, TCP/IP returns an error code when you
attempt to connect to a closed port. This shows up as a “Connection
reset by peer” error. If you set this to 1, attempts to connect to closed
TCP ports are dropped without sending an error. This slows down port
scans and can add some semblance of security to your system. It is not a
replacement for packet filtering, however!
net.inet.tcp.delayed_ack:

1

Run-time toggle. This tells FreeBSD to attempt to include TCP ACK
information on a data packet instead of sending additional packets to
signal the end of a connection.
net.inet.tcp.recvbuf_auto:

1

Run-time toggle. This enables automatic resizing of receive buffers.
net.inet.tcp.recvbuf_inc:

16384

Run-time variable. This is the size of the increment by which receive
buffers are changed by automatic sizing.
net.inet.tcp.recvbuf_max: 262144

Run-time variable. This is the maximum size of any one receive buffer on
the system.
net.inet.tcp.slowstart_flightsize:

1

Run-time variable. This specifies the number of packets FreeBSD will
send during the slow-start portion of a TCP transaction across a wide
area network.
670

A pp en dix

net.inet.tcp.local_slowstart_flightsize:

4

Run-time variable. This specifies the number of packets FreeBSD will
send during the slow-start portion of a TCP transaction across the local
network.
net.inet.tcp.newreno:

1

Run-time toggles. This toggles RFC2582 connection recovery, also known
as the TCP NewReno Algorithm.
net.inet.tcp.tso:

1

Run-time toggle. Some network hardware can handle breaking network
requests into packets with proper sizes for the medium. For example, with
certain network cards, FreeBSD can just hand a lump of data to the card
and ask it to break that lump up into packets. This allows for more optimized communication with the network card and less work for the CPU.
This tells FreeBSD to use the hardware to perform segmentation when
supported.
net.inet.tcp.sendbuf_auto:

1

Run-time toggle. This enables automatic sizing of TCP connection send
buffers.
net.inet.tcp.sendbuf_inc:

8192

Run-time variable. This gives the amount to increase or decrease the TCP
send buffer each time a change is needed.
net.inet.tcp.sendbuf_max:

262144

Run-time variable. This is the maximum size of a single TCP send buffer.
net.inet.tcp.sack.enable: 1

Run-time toggle. FreeBSD uses selective ACK by default, but occasionally
very old hosts have trouble with this feature.
net.inet.tcp.do_tcpdrain:

1

Run-time toggle. This tells FreeBSD to flush packets from the reassembly
queue when it is low on mbufs. Now that FreeBSD automatically allocates
mbufs from kernel memory, this is much less useful than it once was.
net.inet.tcp.always_keepalive:

1

Run-time toggle. By default, FreeBSD checks all TCP connections to see
if they’re still alive. Normally, if a connection between two hosts drops,
these keepalives tell the systems to close down the connection. If you disable this behavior, FreeBSD will not check if the connection between the
hosts is up or down; rather, it will assume that the connection is up until
it cannot pass data.
net.inet.udp.log_in_vain:

0

Run-time toggle. This logs attempts to connect to any UDP port where
no program is listening.

Som e I n te res t in g s ys ct l M I B s

671

net.inet.udp.blackhole:

0

Run-time toggle. Like TCP, UDP sends an error message when someone
attempts to connect to a port that doesn’t have anything listening on it.
If you set this to 1, attempts to connect to closed UDP ports are dropped
without sending an error. This slows down port scans.
net.link.log_link_state_change:

1

Run-time toggle. If enabled, FreeBSD will log every time a network
interface goes up or down.
debug.debugger_on_panic:

1

Run-time toggle. If set, FreeBSD will enter the debugger on a panic.
debug.trace_on_panic:

0

Run-time toggle. If set, FreeBSD automatically prints a stack trace when
it enters the debugger after panicking.
debug.witness.watch:

1

Run-time variable, boot-time tunable. With WITNESS compiled into the
kernel, FreeBSD performs additional sanity checking of all kernel locks.
Setting this to 0 disables these checks. Once you turn WITNESS off, you
cannot turn it back on without rebooting. Note that you will not get the
benefits of WITNESS when you disable it; specifically, if your system
panics due to locking while under heavy load, you won’t get the lock
message that would have told a FreeBSD developer where to find the
problem.
debug.witness.skipspin:

1

Boot-time toggle. This tells WITNESS to not perform any checks on spin
locks. Checking spin locks with WITNESS makes almost any machine
unbearably slow. Skipping spin locks, but using WITNESS in all other
cases, makes WITNESS only slow instead of unbearably slow. This is the
same as the kernel option WITNESS_SKIPSPIN.
debug.minidump:

1

Run-time toggle, boot-time tunable. By default, FreeBSD only dumps
kernel memory on a panic. Set this to 0 to dump the entire contents of
physical memory on panic.
hw.ata.ata_dma:

1

Boot-time toggle. This enables DMA in IDE devices. Set this to 0 if your
hardware uses PIO instead of DMA. If your hardware is old enough to
require PIO, you probably know about it.
hw.ata.atapi_dma:

1

Boot-time toggle. This enables DMA in ATAPI devices. Set this to 0 if your
hardware uses PIO instead of DMA or if your hardware has a buggy DMA
implementation. Again, if your hardware is old enough to require PIO,
you probably know about it.

672

A pp en dix

hw.ata.wc:

1

Boot-time toggle. This enables IDE write caching, as supported by
some hard drives. Write caching uses a small buffer on the hard drive
to cache writes en route to the disk, improving performance at a cost in
reliability. Set this to 0 to disable write caching and enhance data safety.
See Chapter 8.
hw.syscons.kbd_reboot:

1

Run-time toggle. With this enabled, the old-fashioned CTRL-ALT-DELETE
sequence reboots the system. Set this to 0 to disable this behavior.
compat.linux.osrelease:

2.4.2

Run-time variable. This defines the version of Linux supported by Linux
mode. Some Linux programs behave differently when this is changed.
While you can change it on the fly, if any Linux programs are in use, the
system will panic. It’s best to let the Linux mode software do this for you.
security.jail.set_hostname_allowed:

1

Run-time variable. This dictates whether jail administrators can change
the hostname of their jail. See Chapter 9.
security.jail.socket_unixiproute_only:

1

Run-time variable. This controls whether jail owners can use protocols
other than TCP/IP. See Chapter 9.
security.jail.sysvipc_allowed:

0

Run-time variable. This determines whether jail owners can use System V
IPC calls. See Chapter 9.
security.jail.enforce_statfs:

2

Run-time variable. When set to 0, jailed users can see all mount points
and partitions on the system, both inside and outside the jail. When set
to 1, only mount points below the jail’s root are visible. When set to 2,
only the jail’s root mount point is visible.
security.jail.allow_raw_sockets:

0

Run-time variable. When set, root in a jail can create raw sockets. See
Chapter 9.
security.jail.chflags_allowed:

0

Run-time toggle. The jail administrator can use chflags(1). See Chapter 9.
security.jail.list:

Read-only. This lists all the active jails on the system. FreeBSD changes this
as you start and stop jails, but you cannot change this sysctl with sysctl(8)
to activate new jails. See Chapter 9.
security.jail.jailed:

0

Read-only. If set to 1, the program that queried the sysctl is running
inside a jail. See Chapter 9.

Som e I n te res t in g s ys ct l M I B s

673

security.bsd.see_other_uids:

1

Run-time toggle. By default, users can see processes belonging to
other users through ps(1) and related tools. Setting this to 0 disables
this visibility.
security.bsd.see_other_gids:

1

Run-time toggle. By default, users can see processes belonging to other
groups through ps(1) and related tools. Setting this to 0 disables this
visibility.
security.bsd.unprivileged_read_msgbuf:

1

Run-time toggle. By default, unprivileged users can read the system message buffer through dmesg(8). This includes users inside jails. Setting
this to 0 disables that visibility, which is usually desirable in a jail server.
security.bsd.hardlink_check_uid:

0

Run-time toggle. By default, users can create hard links—see ln(1)—to
any file. If you set this to 1, users cannot create hard links to files owned
by other users.
security.bsd.hardlink_check_gid:

0

Run-time toggle. By default, users can create hard links to files owned by
any group. If you set this to 1, users can only create hard links to files that
are owned by a group they belong to.
security.bsd.overworked_admin:

3.1415925

Read-only. If you’ve actually read this whole appendix, you’ve obviously
been working too hard. Put this book down. Go outside. Get some fresh
air for an hour or two. You’ll feel better. If you thought this was a real
sysctl MIB, however, stay outside and offline for at least two days.

674

A pp en dix

INDEX
Symbols and Numbers
(asterisks), in crontabs, 464
466
$ (dollar sign), for user name in environment fields, 200
$BLOCKSIZE environment variable, 216
$EDITOR environment variable, 185
$Header$ string, 113
$HOME/.nsmbrc file, 249
$Id$ string, 113
$Log$ string, 113
$TAPE variable, 91–92
. (dot) in names, for hidden files, 338
# (hash marks), for comments, 130
+IGNOREME file, 408
? (question mark), to display commands
for loader, 66
/ (root) partition, 38
// (slashes), for comments, 424
~ (tilde)
in environment fields for user home
directory, 200
for tip program, 75
3Ware, 542
4BSD scheduler, 349
32-bit computing, 369
64-bit computing, 369
110n.sh plug-in, for FreeSBIE, 635
386BSD, 4
*

@reboot,

A
command (FTP), 523
host record, 433
aaccli, 542
ABI (application binary interface)
reimplementation, 363
supported, 364
access control lists, for milter-greylist,
489–490
access-time stamp, 220
a
A

account expiration field, in
/etc/master.passwd file, 187
ACPI (Advanced Configuration and
Power Interface), 142
disabled when booting, 62–63
Active Directory, 513
active memory, 577
addr keyword, in /etc/nsmb.conf file, 250
Address Resolution Protocol (ARP),
146, 160
table, 160
adduser command, 181
adduser plug-in, for FreeSBIE, 634
administrative group, creating, 192–193
Adobe Acrobat Reader, 365
Advanced Configuration and Power
Interface (ACPI), 142
disabled when booting, 62–63
aggregation protocols, 175–176
AHC_REG_PRETTY_PRINT option, for kernel, 133
AHD_REG_PRETTY_PRINT option, for kernel, 133
AIX, 9
alert level for syslog protocol, 588
Alias statement, for Apache, 511
aliases
assigning to network card, 84
and outgoing connections, 164
ALL EXCEPT option, in login.access, 197
ALL option
for Apache, 509
in login.access, 197
in TCP wrappers, 267, 268
AllowGroups variable, for SSH, 444–445
AllowOverrride keyword
for Apache, 510
for virtual host, 519
AllowTcpForwarding, for SSH, 443
AllowUsers variable, for SSH, 444–445
allscreens_flags variable, 86
Alpha servers, 7
amd64 platform, 34

anonymous FTP server, 526
antispam blacklists, 473, 485–486
a.out format, 357
Apache web server, 500–506
configuration files, 501
controlling, 521
core configuration, 501–503
directories and permissions, 507–515
HTTPS websites, 520
including other configuration files,
515–516
logs, 503–505
modules, 505–506
password protection and, 512–515
virtual hosting, 517–519
apachectl program, 521
application binary interface (ABI)
reimplementation, 363
supported, 364
applications, 148
apropos, for man page searches, 23
architectures, FreeBSD build for
difference in, 399
archives
checking contents, 101–102
listing file in, 95
restoring files from, 102–103
restoring filesystem, 103
Areca, 542
ARM platform, 10
ARP (Address Resolution Protocol),
146, 160
table, 160
arp -a command, 160
ASCII file transfer, 523
asterisks (*), in crontabs, 464
asynchronous mounts in FFS, 219
Asynchronous Transfer Mode (ATM), 147
AT&T, and BSD, 2, 4
AT&T UNIX, 364
ATA disk drives, numbering, 212
atapicam kernel module, 230
atime, 220
ATM (Asynchronous Transfer Mode), 147
atomic clock, 448
attach rules, for devd, 257
audit group, 194
auth facility, 587
AuthConfig override, in .htaccess file, 510
authentication
PAM (Pluggable Authentication
Modules), 310
Radius user authentication, 513–515
with SASL, 491–493

676

I ND EX

authoritative nameserver, 412
chain of, 415
authpf group, 194
authpriv facility, 587
AuthXRadiusCache option, for Apache, 514
AuthXRadiusCacheTimeout option, for Apache,
514
autoboot_delay variable, 70
autologin plug-in, for FreeSBIE, 634
automatic reboots, after system panic,
638, 642
avail memory, listing in startup
messages, 77

B
background fsck, 81, 223–224
backtrace, 643–645
backup server, setup for mirroring, 565
backups
of disklabels, 538
before drive install, 240
extracting files from, 95
of kernel, 128
multiple on one tape, 105–106
of old version of software, 407
programs
dump, 98–101
tar, 94–97
restoring, from dump, 101–104
rewinding tapes, 91
slice table, 533
specifying tape drive for, 96
of system, 90
tapes for, 90–93
bandwidth, 571
PF control of, 276
Banner setting, for SSH, 444
base 2, 150
base 10, 151
BASEDIR option, for FreeSBIE, 632
Bash shell, installing, 57
Basic Input/Output System (BIOS), 62
setting to boot from CD, 45
beastie_disable variable, 70
Beck, Bob, 487
Begemot SNMPD (bsnmpd), 598,
600–602
Berkeley Internet Name Daemon (BIND)
chroot environment for, 437
configuration files, 423–424
masters and slaves, 423
security, 436–437
Berkeley Software Distribution (BSD), 2–3
license, 3

best practices, in revision control, 106
Big Giant Lock (BGL), 345
bin command (FTP), 523
/bin directory, commands in, 64–65
bin group, 194
/bin/sh shell, 58, 64
/bin/tcsh shell, 58, 64
binaries, formats, 357
binary
branding, 364
identifying and setting, 367
file transfer, 523
math, 150–151
updates, 378–380
values, as sysctl value, 121
BIND (Berkeley Internet Name Daemon).
See Berkeley Internet Name Daemon
(BIND)
bind group, 194
binding network server to ports, 157
BIOS (Basic Input/Output System), 62
setting to boot from CD, 45
bits, 150
blacklists, 473, 485–486
blank screen, for idle system, 85
blanktime variable, 85
bleeding-edge version of
FreeBSD-current, 373–374
block size, 41, 236–237
blocking on disk, 574
blocks in FFS, 217
$BLOCKSIZE environment variable, 216
Bluetooth, 302
boot blocks, 62
boot CDs, preparing, 47–48
boot disks, mirroring, 549–550
boot floppies, preparing, 47
boot loader
for diskless client, on network, 607
and tftpd server, 609
boot media, 45–46
boot process
with alternate kernel, 137–138
default files, 68–69
loader, 62–63
configuration, 69–70
loader prompt, 67–68
loading kernel modules, 125–126
multi-user startup, 79–88
serial consoles, 70–75
single-user mode, 63–66
startup messages, 76–79
bootable CDs, FreeSBIE to build, 630
bootable disk, marking, 229

/boot/defaults/loader.conf file, 68–69
boot.flp file, 47
/boot/kernel directory, 119, 124
/boot/kernel/kernel directory, 67
/boot/kernel.good directory, 128
/boot/kernel.old directory, 128
/boot/loader.conf file, 68, 69–70
controlling consoles from, 72
to disable write caching, 221
to load geom_mirror kernel module, 549
for setting NanoBSD serial console
speed, 629
BOOTP (Internet Bootstrap
Protocol), 607
boot-time tunable sysctls, 123
testing, 124
boot-time tunables, 69
boot_verbose variable, 70
botnets, 179
bottleneck, 586
analysis with vmstat, 571–574
disk speed as, 574
input/output as, 570
locating, 570
bounce message, 469
branch ports, 405
brandelf command, 367
branding software binaries, identifying
and setting, 367
bridge module for bsnmpd, 602
broadcast address, 155
broadcast protocol, Ethernet as, 159
BSD (Berkeley Software Distribution), 2–3
license, 3
BSD 4.4-Lite, 4
BSDstats Project, 7
bsdtar, 94
bsnmpd (Begemot SNMPD), 598
configuring, 600–602
loading modules, 602
buffer overflows, 178
buffers, for incoming connections,
174–175
bug tracking system (GNATS), 647
build status, tracking, 338
building FreeBSD, 388–396
GENERIC kernel, 389
installing userland, 393–395
make buildworld command, 388–389
optimizing with parallel builds, 390
preparing for userland install, 390–393
building nameserver, 422–424
burnCD command, 48
bytes, 152
bzip compression, 97
I N D EX

677

C
CA (Certificate Authorities), 282
cache memory, 577, 578
caching
and name service switching, 450–453
zeroing, 452–453
cash symbol ($), for user name in environment fields, 200
CAT5 cable, 159
console connections over, 73
Category field, in problem report, 651
CD images, viewing contents, 235
CD packages, 322–323
CD9660 option, for kernel, 133
cdrtools, 226
CDs
booting from, 45, 46
as fixit disk, 115
installing FreeBSD from, 46–47
ISO 9660 filesystem for, 230
preparing boot, 47–48
purchasing with FreeBSD, 43
central processing unit (CPU)
FreeBSD support for, 131
information display, in startup messages,
76–77
load on system, 576
multiple, 143, 344–349
requirements, 36
and SMP, 347–348
usage and performance, 583–584
vmstat information on, 573
certification as Unix, 9
CERT directory, 44
Certificate Authorities (CA), 282
certificates, 282–285
creating request, 283–284
obtaining signed, 284–285
self-signed, 285
for SSL web servers, 520
CFLAGS option, in /etc/make.conf, 307
challenge password, for certificate, 284
CHANGES file, for ports infrastructure, 318
check-ins, multiple, 112–113
checksum, 333
chflags command, 203
chmod command, 65
chown command, 193
chpass() command, for changing accounts,
185–186
chroot command, 286, 525, 613
for tftpd server, 463
chroot environment, for BIND, 437
ci command, 107

678

I ND EX

CIFS (Common Internet File Sharing).
See Common Internet File Sharing
(CIFS)
ciphertext, 280
Cisco, Fast EtherChannel (FEC), 175
Class A network, 153
Class B network, 153
Class C network, 153
class environment, for users, 199
Class field, in problem report, 651
cleaning ports, 338–339
cleartext, 280
clients
FTP, 522–523
for geom_gate, 563
for NFS, restricting, 247
for SSH, 445–447
cloned interfaces statement, 176
CLONEDIR option, for FreeSBIE, 632
CNAME host records, 433
co command, 107
for old versions of file, 111
coaxial cable, 159
code freeze, 374
cold backup, 96
collections, for csup, 384
collision domain, 159
combined log format, for Apache, 504
combinedio log format, for Apache, 504
comconsole plug-in, for FreeSBIE, 634
COMCONSOLE_SPEED variable, 628
command line, 14, 15
command prompt
as intruder’s goal, 262
root, in single-user mode, 605
shell for, 188–189
in single-user mode, 63
SNMP tools, 600
commands
displaying for loader, 66
man pages for documenting, 21–24
comments
hash marks (#) for, 130
to remove entry from kernel
configuration, 131
slashes (//) for, 424
commercial operating system, 12
committers, 5–6, 654
becoming, 657, 658
Common Internet File Sharing (CIFS),
243, 248–252
configuring, 249
file ownership, 252
kernel support for, 249
mounting share, 251

name resolution, 250
prerequisites, 248
serving shares, 252
common log format, for Apache, 503
Common Unix Printing System
(CUPS), 459
communication channels, 14–15
communities in SNMP, 600
comparison operators, in syslog.conf
rules, 590
COMPAT_IA32 option, for kernel, 369
compatibility options, for kernel, 133
compat.linux.osrelease sysctl, 673
compiling programs, 316–317
compressed tarballs, 97
compression
of files, 97
of log file, 596
computer resources, performance
and, 570
Computer Science Research Group
(CSRG), 2
copyright, 4
Concurrent Versions System (CVS), 382
/conf/base directory, 611–612
/conf/default directory, 612–613
Confidential field, in problem report, 650
confidentiality, from cryptosystems, 280
config program, in kernel build
process, 136
CONF_INSTALL variable, for NanoBSD, 623
confirmation email, of problem report
submission, 653–654
CONF_WORLD variable, for NanoBSD, 623
connected protocol, 156
connectionless protocol, 156
console facility, 587
consoles, 604
hardware serial, 71
insecure, 605–606
options in /etc/rc.conf file, 85–86
serial, 70
serial port on Soekris as default, 617
software serial, 71–72
consumer in GEOM, 530
context switches
by CPU, 344
voluntary or involuntary, 579
contributors, 6
COPTFLAGS option, in /etc/make.conf, 307
copy of panic message, obtaining, 639
copycenter, 3
copying
default configuration, risk from, 69

files
NanoBSD disk image to flash drive,
617–618
over SSH, 446
copyright, for FreeBSD Project, in startup
messages, 76
COPYRIGHT file, for Ports Collection, 318
Core, 6
coredumpsize variable, in login.conf file, 199
corrupt filesystem, tar and, 94
corrupted data, from mounting dirty
partition as read-write, 224
CPU (central processing unit). See central
processing unit (CPU)
cpu criteria, in kernel configuration file,
128–129
cputime variable, in login.conf file, 199
CPUTYPE option, in /etc/make.conf, 307–308
crash. See panics
crash boxes, 35
crash dump, 640
configuring, 640–641
manual, 642–643
system preparation to capture, 638
using, 643–646
create mode for tar, 95
crit level for syslog protocol, 588
cron facility, 587
cron (job scheduler), 463–466
and environment, 464
for scheduling regular portsnap, 404
crontabs, 463
in boot process, 466
format, 464–466
cross-building, FreeBSD, 399
cryptographic key, generating and using
in GELI, 559–560
CSRG (Computer Science Reserach
Group), 2
copyright, 4
csup tool, 382, 387
to get source tree, 387
CUPS (Common Unix Printing
System), 459
current resource limits, for user
account, 199
cust_allow_ssh_root script, 628
cust_comconsole script, 628
cust_install_files script, 628
Custom distribution, 42
customized kernel, 126–130
CustomLog statements, for Apache, 504
customroot plug-in, for FreeSBIE, 634
customscripts plug-in, for FreeSBIE,
634–635
I N D EX

679

cust_pkg script, 628
CVS (Concurrent Versions System), 382
CVSup server
building local, 399–402
controlling access, 402
CVSup tool, 382
CXXFLAGS option, in /etc/make.conf, 307
cylinders in disk drives, 530
Cyrus IMAP, 494

D
daemon facility, 587
daemon group, 194
Daemon News, 26
DAEMON provider, for rc scripts, 352
daemons, unprivileged user
accounts for, 262
daily status check, of mirrored disks, 550
daily_local variable for periodic, 311
daily_output variable for periodic, 310
daily_show_badconfig variable for
periodic, 311
daily_show_info variable for periodic, 311
daily_show_success variable for
periodic, 310
daily_status_gstripe_enable, 547
DARPA (Defense Advanced Research
Projects Agency), 2
Darwin, 8
data command, 471
data corruption, from mounting dirty
partition as read-write, 224
data, separating from operating system, 41
databases
for Sendmail files, 480
shutdown, 88
three-way mirror for, 550
datalink layer, in OSI, 146–147, 149, 150
datasize variable, in login.conf file, 199
date, converting epochal seconds to, 298
db> prompt, 642
DB9-to-RJ45 converters, 73
dd command, 47, 236–237
deactivating encrypted disks, 560–561
deadly embrace, 346
debug level for syslog protocol, 589
debug.debugger_on_panic sysctl, 672
debugging
enabling, 80
kernels, 641–642
Linux mode, 368–369
preparing system to provide information after panic, 639–640
rc scripts, 353–354

680

I ND EX

debug.minidump sysctl, 672
debug.witness.skipspin sysctl,
debug.witness.watch sysctl,

672
672

decimal math, 151
default accept, vs. default deny, 264–265
in packet filtering, 273–274
default BSD pager (more), 23
default class, 197
default configuration, copying,
risk from, 69
default files, 68–69
default groups
in FreeBSD, 194
for user account, 181, 183
default host, 518
default route, 163
defaultclass setting, in adduser.conf file, 183
defaultLgroup setting, in adduser.conf file, 183
DEFAULTS file, for kernel
configuration, 129
defaultshell setting, in adduser.conf file, 183
Defense Advanced Research Projects
Agency (DARPA), 2
deleting
shared libraries, 394
user accounts, 188
DenyGroups variable, for SSH, 444–445
DenyUsers variable, for SSH, 444–445
dependencies
changing, 408
portmaster to check, 408
for software packages, 325
uninstalling package and, 328
Description field, in problem report,
651, 653
detach rules, for devd, 257
/dev/console device, 604
devd command, 256–260
configuration, 257
Developer distribution, 42
devfs filesystem, 238, 253–260
global rules, 255–256
Device busy message, 93
device drivers
development, 35
entry in startup messages, 77
kernel options, 134
man pages for, 27
sysctl flags for, 123–124
device filesystem, 238
device names, 27
of floppy drives, 47
for root partition, 64
device nodes, 79, 210–212
devfs filesystem for managing, 253–260

for dynamic devices, 256
and removable media, 230
for tape drive, 91
Device not configured message, 92
devices criteria, in kernel configuration
file, 129
/dev/ttyd device, 604
/dev/ttyp device, 604
/dev/ttyv device, 604
/dev/zero device, 236
df command, 215–217, 245
DHCP (Dynamic Host Configuration
Protocol), 49, 161, 165, 456–459
configuration, 55
server setup for diskless system, 607–609
_dhcp group, 194
dialer group, 194
dial-up terminal, 604
dictionary attacks, 482
diff mode, 96
dig command, 414–418
axfr keyword, 436
disabling recursion, 417–418
finding hostnames with, 416–417
on specific nameserver, 417
digital certificates, 282
directories
for Apache, 507–515
options, 508–510
Apache server return of formatted
list, 509
displaying disk usage, 216
exporting multiple, 246–247
directory services, authentication against
third-party, 513
DirectoryIndex statement, for Apache, 511
dirty disks in FFS, 222–224
forcing read-write mounts on, 224
disk device, network exports, 561–565
disk drives, 209–210. See also filesystems
basics, 530–531
deactivating encrypted, 560–561
device nodes for, 211
hard disks and partitions, 211–212
mirroring, 547–550
mounting and unmounting, 214–215
striping, 545–547
vmstat information on, 573
disk images
mounting, 235
for NanoBSD
copying to flash drive, 617–618
updating, 630
disk management system. See GEOM

disklabel, 536–537
backup and restore, 538
editing, 538–539
missing, 540
reading, 537–538
disklabel command, 229–230
disklabel editor, 241
diskless clients, TFTP for downloading
operating system, 461
diskless farm, configuring, 611–612
diskless FreeBSD, 606–610
activating remounting, 611
clients, 607
DHCP setup for, 607–609
distributing /etc/rc.conf to hosts, 613
installing, 615
installing packages on, 613–615
NFS server and client userland, 609–610
diskless workstations, NanoBSD for userland, 630
disks, in single-user mode, 64
distfiles directory, 44
for Ports Collection, 319
distfiles, portmaster offer to delete, 408
distribution of FreeBSD, choosing, 42, 54
djbdns, 422
dmsg.boot, to determine CPU type, 131
DNS (Domain Name System). See Domain
Name System (DNS)
dnswalk command, 436
doadump function, 644
doc directory, 44
documentation
from FreeBSD.org, 25–26
Frequently Asked Questions (FAQ),
25, 648, 659
Handbook, 25, 27
mailing list archives, 26, 28
man pages, 21–24, 27–28
DocumentRoot setting, for Apache, 503
dollar sign ($), for username in environment fields, 200
domain lists, milter-greylist, 489
Domain Name System (DNS), 82, 411
basic tools, 413–419
dig command, 414–418
host command, 413–414
in-addr.arpa, 418–419
nslookup, 414
checking, 436
configuring resolver, 419–421
how it works, 412
for jail, 292
master and slave file storage, 427

I N D EX

681

Domain Name System (DNS), continued
reverse DNS zones, 433–434
Soekris for server, 628
swap meets, 423
zone files, 428–434
domains
finding mail servers for, 468–469
setting local names, 420–421
virtual, 481–483
dot (.) in names, for hidden files, 338
dotted-quad notation, for configuring
interface, 162
Dovecot IMAP server
configuring, 494–495
installing, 494
running, 496
SSL certificate creation, 495
downloading. See also FTP sites
FreeBSD updates, 379
DSA key files, 441
du command, 216
dual console, 72
dump, 640. See also crash dump
swap space for, 39
dump program, 94, 98–101, 643–646
levels in, 98–99
restoring files from, 101–104
running, 100–101
snapshot for, 222
duplex setting for Ethernet, 160
DVDs. See also CDs
filesystems for, 230
installing FreeBSD from, 46–47
Dvorak keyboard layout, 85
dynamic devices, devd command for
managing, 256
Dynamic Host Configuration Protocol
(DHCP), 49, 161, 165, 456–459
server setup for diskless system, 607–609
dynamic nameservers, 422

E
$EDITOR environment variable, 185
ejecting
removable media, 231
tapes, 91
ELF binaries, 357
email
greylisting, 473, 487–491
for help, 29–31
IMAP and POP3, 493–498
mailwrapper program, 474
managing, 86
overview, 468–473

682

I ND EX

permanence of, 31
responding to, 31
sending, 30–31
Sendmail Mail Transfer Agent (MTA),
473–476
attaching milter-greylist to, 490–491
authentication with SASL, 491–493
configuration options, 476–481
database files, 480
submission vs. reception, 474–475
SMTP protocol, 470–472
spam
mail server access for, 472
rejecting sources, 485–486
text wrap in, 30
undeliverable, 469
virtual domains, 481–483
whitelist, 487
email address of administrator, for
Apache, 502–503
emerg level for syslog protocol, 588
employees, as security risk, 179
empty quotes, in loader.conf file, 70
emulator, 362, 363
encapsulation, 149
encrypted disks, deactivating, 560–561
encryption
of filesystems, 558–561
of partitions, 80
password field, in /etc/master.passwd
file, 187
public key, 280–286
certificates, 282–285
connecting to SSL-protected ports,
285–286
OpenSSL configuration, 281–282
encryption keys, for SSH, 441–442
Environment field, in problem report,
651, 652
environment variables
for commands run from cron, 464
specifying for crontab, 465
epochal seconds, converting to normal
dates, 298
erasing
hosts cache, 452–453
tape, 93
err level for syslog protocol, 588
errata branch, 372–373
ERRATA directory, 44
error log, for Apache, 503–504
error messages, for NanoBSD build
failure, 625–626
ErrorDocument keyword, for Apache, 511
errors, in DNS configuration, 436

/etc/adduser.conf file, 182–183
/etc/amd.map, 302
/etc/bluetooth, 302
/etc/bluetooth.device.conf, 302
/etc/crontab
for maintenance jobs, 586
vs. user crontabs, 463
/etc/csh.* files, 303
/etc/defaults/bluetooth.device.conf, 302
/etc/defaults/devfs.rules file, 255, 303
/etc/defaults/periodic.conf, 310–311
/etc/defaults/rc.conf file, 79–87, 356
/etc/devd.conf file, 303
for flash drives, 259–260
for laptop wireless networks, 258–259
/etc/devfs.conf file, 253–254, 303
/etc/devfs.rules file, 255, 303
/etc/dhclient.conf file, 303
/etc directory, 301
in different Unix-like systems, 302
/etc/disktab file, 303
/etc/dumpdates file, 100
/etc/exports file, 244–245
/etc/freebsd-update.conf, 378–379
/etc/fstab file, 64, 212–213
configuring for new disks, 241
for diskless systems, 614
to enable linprocfs, 367
file-backed filesystems and, 238
for jails, 292
memory disks and, 234
for mirrored boot drives, 549
removable media and, 231
/etc/ftpchroot file, 525
groups list in, 525
/etc/ftpmotd file, 526
/etc/ftpusers file, 525
/etc/ftpwelcome message, 526
/etc/gg.exports file, 562
/etc/group file, 190–191
/etc/hosts file, 419–420
local DNS overrides with, 422
/etc/hosts.allow file, 265, 304
/etc/hosts.equiv file, 304
/etc/hosts.lpd file, 305
/etc/inetd.conf, 305, 453–454, 524
/etc/libmap.conf file, 360
/etc/localtime file, 305
/etc/locate.rc, 305
/etc/login.* file, 306
/etc/login.access file, 195
/etc/login.conf file, 197
environment settings in, 199–200
/etc/login.conf.db file, 198
/etc/mail/access file, 476–478

/etc/mail/aliases file, 476, 478–479
/etc/mail/local-host-names file, 481
/etc/mail/mailer.conf file, 306, 474
/etc/mail/mailertable file, 476, 479
/etc/mail/relay-domains file, 476, 480
/etc/mail/sendmail.cf file, 473
changing, 483–486
/etc/make.conf, 306–308, 396
/etc/master.password file, 184
vipw to edit, 186
etcmfs plug-in, for FreeSBIE, 635
/etc/motd file, 308
/etc/mtree directory, 308
/etc/namedb directory, 427
/etc/netstart script, 65, 309
/etc/network.subr script, 309
/etc/newsyslog.conf file, 309, 594–597
/etc/nscd.conf, 309
/etc/nsmb.conf file, 249
sample entries, 252
/etc/nsswitch.conf file, 420, 450–451
/etc/opie*, 309
/etc/pam.d/*, 310
/etc/passwd file, 184
/etc/pccard_ether script, 310
/etc/periodic.conf, 310–311
daily_status_gmirror_enable, 550
daily_status_gstripe_enable, 547
/etc/pf.conf file, 275–277
/etc/pf.os, 311
/etc/phones, 311
/etc/portsnap.conf file, 403–404
/etc/ppp, 311
/etc/printcap file, 312, 460–461
/etc/profile file, 312
/etc/protocols file, 148, 312
/etc/pwd.db file, 184
/etc/rc script, 79, 87–88, 312
/etc/rc.conf file, 79–87
and clock synchronization, 449
console options, 85–86
defaultrouter entry, 163
for diskless system, 614
distributing to diskless hosts, 613
dumdev variable, 641
editing to disable program, 65
to enable inetd at boot, 455–456
to enable mail daemons, 475
to enable sshd at boot, 440
enabling devfs rules, 256
enabling dhcpd in, 457
filesystem options, 80–81
ifconfig statement, 164
ifconfig_interface_name option, 165
for jails, 292, 293
I N D EX

683

/etc/rc.conf file, continued
network daemons, 81–83, 207
network options, 83–84
network routing options, 84–85
nfsclient option, 245
other options, 86–87
securelevels and, 206
to start lpd at boot, 460
startup options, 80
to turn on NFS server support, 244
/etc/rc.d directory, 87–88
/etc/rc.d/jail file, 293
/etc/rc.initdiskless script, 611
/etc/rc.shutdown script, 88
/etc/remote file, 74, 312
/etc/resolv.conf
default local server in, 420
nameserver list in, 421
/etc/rpc, 313
/etc/security/ directory, 313
/etc/services file, 157, 167, 313
/etc/shells file, 188–189, 313
/etc/snmpd.config file, 313, 600
/etc/spwd.db file, 184
/etc/src.conf file, 313, 396
/etc/ssh/sshd_config file, 442
/etc/ssl/openssl.cnf file, 281
/etc/sysctl.conf file, 122
kern.ipc.nmbclusters sysctl value, 172
/etc/syslog.conf file, 313, 589
/etc/termcap file, 314
/etc/ttys file, 314, 604–606
ether device, 135
Ethernet, 147, 158–160
configuring connection, 161–166
as device, 135
speed and duplex, 160
Ethernet address, 160
Eudora, FreeBSD support for, 493
ExecCGI option, for Apache, 509
exit command, for script command, 114
expire value, for slave nameserver,
430, 431
exporting
disk device across network, 561–565
multiple directories, 246–247
NFS, 243
configuring, 244–245
ext2fs Linux filesystem, 227
ext3fs Linux filesystem, 227
extra directory, for Apache, 516
EXTRA option, for FreeSBIE, 633
extract command, 104
extracting files, from backup, 95

684

I ND EX

F
failed attempts to connect to system, 83
failover, 176
FAQ (Frequently Asked Questions), 25,
648, 659
Fast EtherChannel (FEC), 175
Fast File System (FFS), 2, 217–225
creating filesystem, 229–230
diagram of kernel internals, 224
dirty disks, 222–224
kernel options, 132
mount options, 220
mount types, 218–219
snapshots, 222
soft updates and journaling with,
220–221
syncer at shutdown, 224
write caching, 221
FAT filesystem, 226
for removable media, 228–230
fault tolerance, mirrored disks for, 547
faults, vmstat information on, 573
fdescfs filesystem, 238
fdformat command, 229
fdimage.exe utility, 47
fdisk

to backup slice table, 533
instructions, in FreeBSD install, 50
interactive mode, 534
for NanoBSD disk image, 626
for splitting hard drive into slices,
534–536
viewing slice table with, 532–533
FEC (Fast EtherChannel), 175
fetch dependencies, 320
fetch extract command, 404
fetch update command, 404
FFS. See Fast File System (FFS)
file descriptor filesystem, 238
file flags, 201–203
limitations, 206
removing, 203
setting and viewing, 203
file ownership
in TFTP server, 462
viewing with ls, 193
file size, and log rotation, 595
file transfer, 522–526
binary and ASCII, 523
FTP client, 522–523
FTP server, 524
anonymous, 526
messages, 525–526
FTP user control, 524–525
scp program, 527

security, 522
sftp program, 527
file-backed filesystems, 238
FileInfo override, in .htaccess file, 510
FILE_LIST option, for FreeSBIE, 633
files, 16
adding to NanoBSD, 628–629
changing owner and group, 193
forwarding email to, 478–479
filesize variable, in login.conf file, 199
filesystem integrity checking tool (fsck).
See fsck
filesystem table, 212–213. See also
/etc/fstab file
filesystems, 11
attempt for automatic fix, 81
building, 540–541
corruption from nextboot, 142
creating FFS, 229–230
displaying currently mounted, 214
dump program and, 98, 99–100
encryption, 558–561
in files, 235–238
creating empty filesystem file, 236–237
foreign, 225–228
journaling with gjournal, 554–557
making accessible, 64
mounting, 214
network, 243–248
options in /etc/rc.conf file, 80–81
populating and trimming
remounted, 612
removable media, 228–231
restoring, 103
special purpose, 231–238
memory filesystems, 232–234
FILESYSTEMS provider, for rc scripts, 352
finding
mail servers for domain, 468–469
man pages, 23
software, 320–322
by keyword, 321–322
by name, 321
fine-grained locking, 346
fingerprint for public key, 441–442
from client, 445
firewall, 275
displaying rules currently
running on, 279
for TFTP, 461
FireWire, kernel support for, 136
Fix field, in problem report, 652
fixit disk, 114–115
flags, for inetd, 456
flash drives
copy NanoBSD disk image to, 617–618

devd example with, 259–260
managing, 256
Soekris with, 616
FlashDevice.sub, expanding, 618–619
floppy disk
booting from, 45
formatting, 228–230
preparing boot, 47
using FFS on, 229–230
.flp file extension, 47
FollowSymLinks option, for Apache, 509
fonts, on console, 85
forcing read-write mounts, on dirty disks
in FFS, 224
foreign
filesystems, 225–228
software libraries, 365
forget command (gstripe), 548
Forward DNS, 413
forwarding email, 478
4BSD scheduler, 349
FQDN (fully qualified domain name), 83
fragments in FFS, 217
frame, 149
free memory, 577
FreeBSD. See also building FreeBSD;
installing FreeBSD
attitude, 20–21
birth of, 4–5
CD set, compiled packages, 323
Configuration Menu, 58–59
CVSWeb history, 406
desktop, 13–14
development, 5–7
Documentation Project, 6
FAQ, 25, 648, 659
forcing use of serial console, 72
FTP site, 43–45
information resources, 19
management of, 6
manual, sections, 22
master source code repository,
read-and-write access to, 5
minimal install, 315–316
partition requirements, 536
potential users, 11
Ports Collection, 317
projects built on, 8
security announcements, 180
shrinking, 396–397
-specific time, 596
strengths, 10–11
website, contributing to, 657
what it is, 2–5

I N D EX

685

FreeBSD community, 655–656
leadership, 656
reasons for involvement, 656–657
FreeBSD Mall, 26, 46
FreeBSD Project
email servers, 467
submitting improvements, 646
supporting, 657
FreeBSD-current, 373–374
FreeBSD.org, 25–26
FreeBSD-questions, 30–31
FreeBSD-stable, 374–375
freebsd-update command, 379
FreeSBIE, 8
building image, 636
choosing packages, 635
configuring, 631–634
installing toolkit, 631
live media with, 630–636
plug-ins, 634–635
rebuilding, 636
freesbie.defaults.conf file, 631
Frequently Asked Questions (FAQ), 25
fsck program, 64, 222–224
background, 223–224
FreeBSD use of variations, 225
turning off prompt, 223
fsck_y_enable variable, 81
FTP client, sftp program as, 447
ftp facility, 587
FTP packages, 324–325
FTP sites
for FreeBSD, 43–45
FreeBSD ISO images on, 46
media setup for install from, 48–49
ftp-chroot variable, in login.conf, 200
ftpd daemon, 524
full-duplex setting for Ethernet, 160, 162
fully qualified domain name (FQDN), 83

G
games, decision to install, 42
games group, 194
gateway_enable variable, 85
GBDE (Geom Based Disk Encryption),
80–81, 558
GELI, 80–81, 558
to encrypt partitions, 558–560
to encrypt swap space, 561
GENERIC file
for kernel configuration, 129
and NFS, 243

686

I ND EX

GENERIC kernel, 126
building, 389
GENERIC.hints file, for kernel
configuration, 130
GEOM, 529
commands, 544–545
essentials, 529–530
RAID and disk size, 542
Geom Based Disk Encryption (GBDE),
80–81, 558
GEOM classes, 529
geom_eli.ko kernel module, 559
geom_gate application, 561
client setup, 563
identifying devices, 564
security for, 562
server setup, 562
shutdown, 564
geom_mirror kernel module, 547
/boot/loader.conf file to load, 549
geom_stripe kernel module, loading, 545
gerbil
filesystem, 40
on keyboard, 473
Liberation Front, 526
get command, for downloading files
with FTP, 523
Getif, 600
ggatec command, 563
list command, 564
rescue command, 564–565
ggated command, 562
GID (group ID). See group ID (GID)
GIDs file, for Ports Collection, 319
gjournal command, 554–557
configuring, 556–557
stop command, 557
gmirror command, 547–550
clear command, 550
stop command, 550
GNATS (bug tracking system), 647
including database on CVSup server, 401
GNU tar, 94
Google, BSD-specific search, 26
graceful restart, of Apache, 521
graid3 load command, 551
graphical interface, X Window
System as, 42
grep command, 324
greylisting, 473, 487–491
group ID (GID), 191
for administrative users, 192–193
in /etc/master.passwd file, 187
Group option, for Apache, 502

groups of users, 190–191
to avoid root user, 191–194
changing for file, 193
changing memberships, 191
creating, 191
default, 181, 194
listing in /etc/ftpchroot, 525
gstat program, to check disk activity,
574–575
gstripe clear command, 546
gstripe command, 545–547
destruction, 546
gtk-send-pr, 649
guest group, 194
gzip compression, 97

H
hacked system, 300
hackers, 180
half-duplex setting for Ethernet, 160, 162
Handbook, 25, 27
hard drives
adding, 240–243
creating partitions, 241
creating slices, 240
fdisk for splitting into slices, 534–536
installing existing files on new, 241–242
multiple, 40–41
requirements, 37
selecting for FreeBSD install, 50–51
hard-coding disk numbering, 238–239
hardware
checking on support for, 36
device name display, 78–79
optimizing network performance with,
170–171
and protocols, 159–160
requirements, 10, 34–37
hardware clock, identifier in startup
messages, 76
hardware serial consoles, 71
hash marks (#), for comments, 130
HDLC (High Level Data Link
Control), 147
$Header$ string, 113
headless systems, 606
heads in disk drives, 210, 530, 532
heartbeat, 567
helo command, 470–471
help. See also documentation
emailing for, 29–31
mailing list, for FreeBSD, 20–21
problem-solving resources, 26–28
from websites, 26

Hewlett-Packard, HP/UX, 9
hexadecimal numbers, 151, 162
hidden files, . (dot) in names for, 338
High Level Data Link Control
(HDLC), 147
hm.pagesize sysctl, 583
home directories
changing for user account, 186
locking FTP users into, 525
for unprivileged users, 262
for users, 40, 58, 182
/home partition, 40
$HOME/.nsmbrc file, 249
homeprefix setting, in adduser.conf file, 183
host addresses, and login access, 196
host command, 413–414
host key, for certificate, 283
host.allow option, 201
host.deny option, 201
hostname, 83
finding with dig, 416–417
and login access, 196
Hostres module for bsnmpd, 602
hosts cache, erasing, 452–453
How-To-Repeat field, in problem report,
651, 653
HP RILOE serial console, 71
HP/UX (Hewlett-Packard), 9
.htaccess file, 510
configuring authentication in, 512
groups and, 515
to require Radius authentication, 514
require-group statement, 515
HTML, avoiding for email, 30
HTML index, for browsing Ports
Collection, 322
htpasswd command, 512
HTTP (HyperText Transfer Protocol), 500
httpd.conf file, 501
configuring Radius
authentication in, 514
including other configuration files,
515–516
permissions and settings in, 507
HTTPS websites, 520
hub for Ethernet, 159
hushlogin variable, in login.conf, 200
hw.ata.ata_dma sysctl, 672
hw.ata.atapi_dma sysctl, 672
hw.ata.wc sysctl, 673
hw.syscons.kbd_reboot sysctl, 673
HyperTerm, 74
HyperText Transfer Protocol (HTTP), 500
HyperThreading, 348, 358

I N D EX

687

I
i386 platform, 34
memory usage, 39
IBM, 5, 9
ICMP (Internet Control Message
Protocol), 147, 155
redirects, 83
icmp_drop_redirect variable, 83
icmp_log_redirect variable, 84
$Id$ string, 113
IDE drives, 37
tape, 90
device nodes, 91
write caching, 221
ident criteria, in kernel configuration
file, 129
ident strings, Revision Control
System and, 113
identd protocol, 267
ifconfig command, 84, 161–162
alias keyword, 163–164
for assigning IP address to network card,
162–163
name keyword, 165
ifconfig statement, in /etc/rc.conf file, 164
ifconfig_em0 variable, 84
+IGNOREME file, 408
ignorenologin variable, in login.conf, 200
image files, copying onto disk, 47
IMAP (Internet Message Access
Protocol), 493
testing with SSL, 497–498
imap-wu, 494
IMGPATH option, for FreeSBIE, 632
inactive memory, 577
in-addr.arpa, 418–419
inbound traffic, on network, 166
Includes directory
for Apache, 516
virtual hosting configuration in, 517
Includes option, for Apache, 509
IncludesNOEXEC option, for Apache, 509
incoming connections, buffers for,
174–175
incremental backups, in dump, 98–99
index, for port directory, 320
Indexes option, for Apache, 509
Indexes override, in .htaccess file, 510
INET option, for kernel, 132
INET6 option, for kernel, 132
inetd daemon, 82, 453–456
changing behavior, 456
configuring servers, 454–455
/etc/inetd.conf, 453–454

688

I ND EX

on jail host server, 288
option to configure, 56
starting, 455–456
inetd_enable variable, 82
info level for syslog protocol, 589
info.0 file, 643
inodes, 217
input, 14
input/output
as bottleneck, 570
top tool and, 579
insecure console, 605–606
INSTALL option, in /etc/make.conf, 308
install path, changing, 339
installation media, 46–47
installed
packages, listing with description, 329
ports, updating, 404–409
installed World Wide Web data, including
on CVSup server, 401
installing
Dovecot IMAP server, 494
ports, 332–334, 337–338
software packages, 325
userland, 393–395
installing FreeBSD, 49
adding software packages, 56–57
adding users, 57–58
boot floppies preparation, 47
on diskless system, 615
hardware
proprietary, 35–36
requirements, 34–37
sample hardware, 35
planning, 33
post-installation setup, 58–59
preinstall decisions, 37–42
distribution choices, 42
games, 42
multiple hard drives, 40–41
partition block size, 41
partitioning, 37–40
process, 45–47, 49–59
INSTKERNAME variable, for kernel, 141
integers, as sysctl values, 121
integrity of cryptosystems, 280
Intel network cards, 171
interactive restores, 104
Internet Authentication Service, 513
Internet Bootstrap Protocol
(BOOTP), 607
Internet Control Message Protocol
(ICMP), 147, 155
redirects, 83

Internet Message Access Protocol
(IMAP), 493
testing with SSL, 497–498
Internet Protocol (IP), 147
Internet Software Consortium, 422
Internetwork Packet Exchange (IPX), 147
interruptible NFS mount, 248
interrupts, vmstat information on, 573
intrusion
preparation with mtree, 296–299
reacting to, 299
involuntary context switches, 579
iostat, 571
IP (Internet Protocol), 147
IP addresses, 152–155
adding to interface, 162–163
Apache binding to, 502
assigning, 155
for interface, 84
controlling Apache access by, 507–508
determining for hostnames, 413
finding hostnames for, 416–417
information for FTP install, 49
for jails, 287
multiple on interface, 163–164
unusable, 155
IP Filter, 273
IP-based virtual hosts, 517–518
IPFW, 273
IPv6 addresses, 418
configuration, 55
IPX (Internetwork Packet Exchange), 147
IRIX (Silicon Graphics), 9
ISA cards, 36
ISO 8601 time format, for logs, 595–596
ISO 9660 filesystem, for CDs, 230
ISO image, name of, 48
ISO-IMAGES directory, 44
ISOPATH option, for FreeSBIE, 632
iX Systems, 46

J
jail command, 291
jail ID ( JID), 294
jails, 286–296
client setup, 290–291
for diskless clients, 610
and /etc/rc.conf, 293
host server setup, 287–289
inetd daemon, 288
NFS programs, 289
sshd, 289
syslogd daemon, 288
in-jail setup, 291–292
and kernel, 289–290

limitations, 295–296
managing, 294–295
NanoBSD for userland, 630
reducing disk space, 291
for running named, 437
shutdown, 293–294, 295–296
startup, 293–294
Java license, 322
jexec program, 294–295
JID (jail ID), 294
jls program, 294
job control, 2
job scheduler (cron), 463–466
and environment, 464
for scheduling regular portsnap, 404
journaling
in FFS, 220–221
with gjournal, 554–557
Juniper, 5, 616
junk email. See also spam
blacklists to reduce, 485–486
preventing, 472–473

K
KDB_UNATTENDED option,

642
Kerberos, 246
kern facility, 587
kern.argmax sysctl, 662
kern.bootfile sysctl, 663
kern.boottime sysctl, 662
KERNCONF variable, 136, 389
kern.coredump sysctl, 665
Kern-Developer distribution, 42
kern.dirdelay sysctl, 666
kern.domainname sysctl, 662
kernel, 117–143
actions in reverse order in backtrace,
644–645
assumptions, 344–345
backups, 128
booting alternate, 137–138
building, 126–130, 136–138
buses and attachments, 127
configuration files, 128–130
preparations, 126–127
troubleshooting, 137
debugging, 641–642
defined, 118–119
determining currently booted, 121
and jails, 289–290
network capacity in, 169–170
no options and inclusions, 139
nodevice keyword, 139
NOTES file for hardware-specific
features list, 138
I N D EX

689

kernel, continued
options for problem solving, 139–140
sharing, 140
sysctl program, 119–124
testing remotely, 141–142
trimming, 131–136
basic options, 131–134
CPU types, 131
device drivers, 134
multiple processors, 134
pseudodevices, 135–136
removable hardware, 136
viewing, 67
WITNESS option, 347
kernel minidump, 39
kernel modules, 124–126
vs. Apache modules, 506
loading and unloading, 125
loading at boot, 125–126
kernel thread, 359
KERNELCONF option, for FreeSBIE, 632
kern.filedelay sysctl, 666
kern.hostname sysctl, 662
kern.init_path sysctl, 664
kern.init_shutdown_timeout sysctl, 664
kern.ipc.maxpipekva sysctl, 663
kern.ipc.maxsockets sysctl, 664
kern.ipc.nmbclusters sysctl value, 171–172
kern.ipc.numopensockets sysctl, 664
kern.ipc.pipekva sysctl, 663
kern.ipc.shmall sysctl, 663
kern.ipc.shmmax sysctl, 663
kern.ipc.shmseg sysctl, 663
kern.ipc.shm_use_phys sysctl, 663
kern.ipc.somaxconn sysctl, 174, 663
kern.log_console_output sysctl, 665
kern.logsigexit sysctl, 664
kern.maxfiles sysctl, 662
kern.maxfilesperproc sysctl, 663
kern.maxproc sysctl, 662
kern.maxprocperuid sysctl, 663
kern.maxusers sysctl, 171–172, 664
kern.maxvnodes sysctl, 662
kern.metadelay sysctl, 666
kern.module_path sysctl, 664
kern.ngroups sysctl, 662
kern.nodump_coredump sysctl, 665
kern.openfiles sysctl, 664
kern.osreldate sysctl, 663
kern.osrelease sysctl, 662
kern.posix1version sysctl, 662
kern.randompid sysctl, 664
kern.sched.name sysctl, 665
kern.sched.preemption sysctl, 665
kern.sched.quantum sysctl, 665

690

I ND EX

kern.securelevel sysctl, 662
kern_securelevel variable, 87
kern_securelevel_enable variable,

87
666
kern.smp.disabled sysctl, 665
kern.sync_on_panic sysctl, 664
kern.timecounter.choice sysctl, 665
kern.timecounter.hardware sysctl, 665
kern.timecounter.smp_tsc sysctl, 665
kernX.flp files, 47
key fingerprint, for SSH, 441
keyboard
autodetection, 72
choosing layout during install, 49
redirecting input to serial port, 71
keyboard map, 85
keywords, in TCP wrappers, 267
kgdb program, 644
kldstat command, 124–125
kldunload command, 125
kmem group, 194
knobs, 79
KNOBS file, for Ports Collection, 319
KNOWN keyword, in TCP wrappers, 267
kqueue program, 362
kern.smp.cpus sysctl,

L
LACP (Link Aggregation Control
Protocol), 176
lagg module, 175
configuring interface, 176
laptop computers, devd example with,
258–259
layers in protocol stack, 146–148
LDAP (Lightweight Directory Access
Protocol), 513
ldconfig program, 355–356
entries in /etc/defaults/rc.conf, 356
ldconfig32_paths option, 369
ldconfig_local32_dirs option, 369
ldconfig_paths variable, 87
ldd command, 358, 360–361
LD_LIBRARY_PATH environment variable,
87, 357
and security, 358
ldp_enable variable, 86
leaf ports, 406
lease in DHCP, 457
LEGAL file, for Ports Collection, 319
less command (FTP), 523
levels in dump, 98–99
libarchive, 94
autodetection of compression type, 97

libc_r library, 359
libiconv.ko module, 249
libkse threading library, 359
libmchain.ko module, 249
libpthread threading library, 359
libraries
foreign, 365
program requirements for, 358
shared, 87, 354–358
versions, 354
threading, 359–360
library directories
adding to search list, 356
list, 355–356
libthr library, 359
license, for BSD, 3
Lightweight Directory Access Protocol
(LDAP), 513
Limit override, in .htaccess file, 510
Link Aggregation Control Protocol
(LACP), 176
linprocfs filesystem, 238
Linux, 9
filesystems, 225
network interface names in, 165
recompiling software for, 362
Linux mode, 364, 365–369
and commercial software, 369
debugging, 368–369
option to enable, 56
testing, 366–367
Linux process filesystem, 238
linux_base, 365
Linuxulator, 364
userland, 366
list command (GEOM), 544–545
LIST command, for IMAP, 497
list mode for tar, 95
Listen setting, for Apache, 502
ListenAddress, for SSH, 443
live files, moving, 242
live media, with FreeSBIE, 630–636
load average, 576
load command (GEOM), 544
loader, 62–63
configuration, 69–70
prompt, 67–68
escape, 63
when booting alternate kernel, 137
loader.conf file, 123
loader_logo variable, 70
loading kernel modules, 125
LoadModule statement, for Apache, 505–506

local distfile respository, 340
LOCAL keyword

in login.access, 196
in TCP wrappers, 267
local package repositories, 339
local programs directory, 14
localhost zones, in named.conf file,
425–426
localhost-forward.db file, 424
localhost-reverse.db file, 424
localn facility, 588
lock order
message, 347
reversals, 143, 347
locked files
breaking locks, 112–113
in revision control, 108
locking, fine-grained, 346
locking accounts, pw command for, 188
locks, automated searches, 112
$Log$ string, 113
logging host, 591
logical packet tunnel, 135
logical port, 157
login
controlling, 604–606
disallowing for unprivileged user, 262
restricting, 195–197
through serial port, 606
login classes, 181, 197
LOGIN command, for IMAP, 497
login group, for user account, 183
LoginGraceTime, for SSH, 443
log_in_vain variable, 83
LogLevel INFO, for SSH, 443
logs
for Apache, 503–505
rotating, 504–505
file format and compression, 596
file management, 593–597
filling drive with, 38
for ftpd, 524
for NanoBSD build failure, 625–626
overlap, 592
by program name, 590–591
of screen display, 114
from Sendmail, 476
with syslogd, 587–593
verbose, 63, 593
booting with, 63
viewing in revision control, 109–110
lookups, supporting name service
switching, 450

I N D EX

691

loopback
device, 135
interface, 161
IP address, 421
lost+Found directory, 222
low-level formatting, of FAT32 media, 229
lpd protocol, 460
lpr facility, 587
ls command, for viewing file ownership
and permissions, 193
lsdev command, 66
lsmod command, 67

M
MAC (Media Access Control) addresses,
146, 160
for DHCP client, 607–608
MAC file, for kernel configuration, 130
Mac OS X, FreeBSD in, 8
MAC table, 160
machdep.hyperthreading_allowed sysctl
value, 348
macros, in /etc/pf.conf file, 276
magnetic media, 210
mail exchanger (MX) record,
432–433, 468
mail facility, 587
mail from: command, 471
mail group, 194
mail servers for domain
finding, 468–469
relay control in, 472
mailer.conf, 492
mailing list
alias for, 478
archives, 26, 28
including on CVSup server, 401
for FreeBSD development, 6
for FreeBSD help, 20–21
retention of postings to, 31
mailnull group, 194
mailq program, 474
mailwrapper program, 474
make build command, 334
make buildkernel command, 389
make buildworld command, 388–389
make check-old command, 393–394
make checksum command, 333
make clean command, 408
make config command, 333
make configure command, 334
make delete-old command, 394
make depends command, 334

692

I ND EX

make distclean command, 339
make extract command, 334
make fetch command,

333
334, 337
make installkernel command, 389
make installworld command, 393–395
make kernel command, 389
make package command, 339
make packageselect command, 635
make patch command, 334
make print-index command, 320
make search command, 321
make tool, 316
for NanoBSD, 623
PREFIX variable for, 339
setting options permanently, 340
and SMP, 349
for upgrading FreeBSD, 398
MAKE_CONF option, for FreeSBIE, 633
Makefile, 316, 319, 320, 336–337
for OpenOffice.org, 331–332
for Sendmail, 484–485
MAKEJ_KERNEL option, for FreeSBIE, 632
MAKEJ_WORLD option, for FreeSBIE, 632
MAKEOBJDIRPREFIX option, for FreeSBIE, 634
MAKEOPT option, for FreeSBIE, 632
makeoptions criteria, in kernel configuration file, 129
malloc-backed disks, 232
creating, 233
man command, 22, 24
man group, 194
man pages, 21–24, 27–28
Management Information Base (MIB), for
sysctls, 120
manpath variable, in login.conf, 200
maproot statement, 246
mark facility, 588
master boot record (MBR), 531
installing, 52
master domain, configuring, 427
master zone files, organizing, 427
MaxAuthTries, for SSH, 443
maximum resource limits, for user
account, 199
maxproc variable, in login.conf file, 199
MaxStartups, for SSH, 444
MBR (master boot record), 531
installing, 52
mbuf clusters, 170, 172
mbufs, 169
memory allocated for, 171
md device, 136
mdconfig command, 234, 235
make install command,

mdmfs command, 232–233
Media Access Control (MAC) addresses,
146, 160
for DHCP client, 607–608
media keyword, for ifconfig, 162
mediaopt keyword, for ifconfig, 162
MegaCli, 542
megarc, 542
member groups, 58
memory
buffer, size of, 577
disks, 136
filesystems, 232–234
constraints, 233–234
shutdown, 234
and network performance optimization,
171–173
program use of, 578
requirements, 37
top for information on, 577–578
usage, 583
vmstat information on, 572
memoryuse variable, in login.conf file, 199
merged from -current (MFC), 375
mergemaster program, 390–393, 395
metadata, 217
MFC (merged from -current), 375
mget command (FTP), 523
MIB (Management Information Base), for
sysctls, 120. See also Simple Network
Management Protocol (SNMP)
Microsoft
FAT32 filesystem, 225
.NET for FreeBSD, 499
Print Services for Unix, 460
Windows, 315
utility for copying disk images, 47
Xenix, 9
migrating from Windows to FreeBSD, 227
milter-greylist, 487
access control lists for, 489–490
attaching to Sendmail, 490–491
configuring, 488–490
retry requirement, 490
mime_magic file, for Apache, 501
mime.types file, for Apache, 501
Minimal distribution, 42
MINIMAL option, for FreeSBIE, 633
minimum time-to-live, for slave
nameserver, 430
minpasswordlength option, 200
mirror of striped disks (RAID 0+1), 544
mirror sites for FreeBSD
FTP site, 43
website, 25

mirroring
disks across network, 565–567
backup server, 565
primary server, 566
(RAID-1), 543, 547–550
boot disks, 549–550
daily status check of, 550
failover and recovery, 567
repairing, 548
mixpasswordcase option, 201
mknod command, 253
mod_bandwidth Apache module, 506
mod_dtcl Apache module, 506
modem, storing phone numbers for
remote, 311
mod_fastcgi Apache module, 506
mod_gzip Apache module, 506
mod_mp3 Apache module, 506
mod_perl2 Apache module, 506
mod_python Apache module, 506
mod_ruby Apache module, 506
modular kernel, 118
mod_webapp-apache2 Apache
module, 506
monitor, displaying, 86
monitoring system security, 299–300
more (default BSD pager), 23
Mosaic web browser, 499
mount command, 64, 214
for foreign filesystems, 225
mount options
in FFS, 220
for journaled filesystems, 557
mount point, for removable media, 230
mountd command, 245
mountdisks plug-in, for FreeSBIE, 635
mounts, stackable, 242–243
mount_smbfs command, 251, 252
mouse, 86
moused_enable variable, 86
moused_type variable, 86
MOVED file, for Ports Collection, 319
moving files to new drives, 242
mput command (FTP), 523
msdosfs mount type, 226
MSDOSFS option, for kernel, 133
msgfile setting, in adduser.conf file, 183
mt command
fsf command, 105
for multiple backups on one tape,
105–106
for tape drive status, 92
mt erase command, 93
mt offline command, 93
mt retension command, 93
I N D EX

693

mt rewind command, 93
mtree program, 296–299
multicore processors, 348
multiple
hard drives, 40–41
processors, 344–349
multitasking, 344
multi-user mode, upgrade problems
from, 396
multi-user startup, 79–88
MultiViews option, for Apache, 510
mutt, 30
/mvar/log/messages, for problem solving, 29
MX (mail exchanger) record,
432–433, 468

N
name service, 411. See also Domain Name
System (DNS)
switching, 420
and caching, 450–453
name-based virtual hosts, 517–518
named daemon, 82
managing, 434–435
security for, 437
named.conf file, 424–427
zones in, 425–426
named.root file, 423–424
names
for groups, 191
of ISO images, 48
for network interfaces, changing,
164–165
nameserver, 412
building, 422–424
disabling recursion, 417–418
dynamic, 422
security, 436–437
update of secondary, zone file serial
number and, 430
NanoBSD
building, 624–627
building appliances, 615–630
configuration options, 619–621
sample entries, 621–624
copying disk image to flash drive,
617–618
customizing, 627–629
defined, 616–617
disk image construction failure, 626
and diskless systems, 618
minor updates, 629
toolkit, 618
using, 629–630

694

I ND EX

nanobsd.sh script, 624
NAT (Network Address Translation), 147n
PF and, 276
navigating man pages, 23
nbns keyword, in /etc/nsmb.conf file, 250
negative niceness, 585
negative time-to-live option, 452
nested RAID, 553, 555
NetApp, 5, 616
NetBSD, 7, 11–12, 87
Netcraft, 7
Netgraph module for bsnmpd, 602
net.inet.icmp.bmcastecho sysctl, 669
net.inet.icmp.drop_redirect sysctl, 669
net.inet.icmp.icmplim sysctl, 669
net.inet.icmp.log_redirect sysctl, 669
net.inet.ip.accept_sourceroute sysctl, 669
net.inet.ip.check_interface sysctl, 669
net.inet.ip.fastforwarding sysctl, 669
net.inet.ip.forwarding sysctl, 668
net.inet.ip.portrange.first sysctl, 668
net.inet.ip.portrange.hifirst sysctl, 668
net.inet.ip.portrange.hilast sysctl, 668
net.inet.ip.portrange.last sysctl, 668
net.inet.ip.portrange.lowfirst sysctl, 667
net.inet.ip.portrange.lowlast sysctl, 667
net.inet.ip.portrange.randomcps sysctl, 668
net.inet.ip.portrange.randomized sysctl, 668
net.inet.ip.portrange.randomtime sysctl, 668
net.inet.ip.portrange.reservedhigh sysctl, 668
net.inet.ip.portrange.reservedlow sysctl, 668
net.inet.ip.redirect sysctl, 669
net.inet.ip.sourceroute sysctl, 669
net.inet.ip.ttl sysctl, 669
net.inet.tcp.always_keepalive sysctl, 671
net.inet.tcp.blackhole sysctl, 670
net.inet.tcp.delayed_ack sysctl, 670
net.inet.tcp.do_tcpdrain sysctl, 671
net.inet.tcp.local_slowstart_flightsize

sysctl, 671
net.inet.tcp.log_in_vain sysctl, 670
net.inet.tcp.newreno sysctl, 671

sysctl, 670
670
net.inet.tcp.recvbuf_max sysctl, 670
net.inet.tcp.recvspace sysctl, 173, 174, 670
net.inet.tcp.rfc1323 sysctl, 670
net.inet.tcp.rfc3042 sysctl, 670
net.inet.tcp.rfc3390 sysctl, 670
net.inet.tcp.sack.enable sysctl, 671
net.inet.tcp.sendbuf_auto sysctl, 671
net.inet.tcp.sendbuf_inc sysctl, 671
net.inet.tcp.sendbuf_max sysctl, 671
net.inet.tcp.sendspace sysctl, 173, 174, 670
net.inet.tcp.slowstart_flightsize sysctl, 670
net.inet.tcp.recvbuf_auto

net.inet.tcp.recvbuf_inc sysctl,

net.inet.tcp.tso sysctl, 671
net.inet.udp.blackhole sysctl,

672
net.inet.udp.log_in_vain sysctl, 671
net.inet.udp.recvspace sysctl value, 173, 174
net.link.log_link_state_change sysctl, 672
netmasks, 152–155, 162
computing in decimal, 154
Netscape Corporation, 499
net-snmp, 600
netstat program, 166, 168–169
for viewing kernel memory used for
networking, 169
network
adapter teaming, 175–176
capacity in kernel, 169–170
capacity planning, 173
cards
assigning IP address to, 162–163
device drivers in kernel, 134
changing window size, 174–175
collisions, 166
current activity, 166–167
daemons, in /etc/rc.conf file, 81–83
device, configuring, 54–56
exports, of disk device, 561–565
filesystems, 243–248
gateway, 56
group, 194
identifying programs listening to, 207
layers, 146–148
maximum incoming connections,
173–174
mirroring disks across, 565–567
backup server, 565
primary server, 566
once-in-a-lifetime vs. standard load, 173
optimizing performance, 170–175
of hardware, 170–171
memory usage, 171–173
options in /etc/rc.conf file, 83–85
performance of, 571
polling, 174
in practice, 148–150
secure mode, 205
server, binding to ports, 157
services, FreeBSD as platform, 11
in single-user mode, 65
starting daemons, 207
time, 447–450
synchronization, 449–450
traffic control, 263–264
Network Address Translation (NAT), 147n
PF and, 276
Network Configuration screen, 55

Network File System (NFS), 243
clients, restricting, 247
enabling client, 245–246
enabling server, 244
exporting, 243
configuring, 244–245
multiple directories, 246–247
kernel options, 133
performance and options, 247–248
server
and diskless client userland, 609–610
for diskless system, 607
and users, 246
network interfaces
adding IP to, 162–163
displaying list, 161–162
multiple IP addresses on, 163–164
renaming, 164–165
testing, 163
testing configuration, 166
Network Neighborhood (Windows), 248
Network Time Protocol (NTP), 448–450
NETWORKING provider, for rc
scripts, 352
newfs command, 103, 229
and encrypted devices, 560
for filesystem creation on file, 237
journaling and, 556–557
for soft updates, 546
for UFS filesystem, 540–541
newfs_msdos command, 229
news facility, 588
news group, 194
newsyslog program, 594
and Apache logs, 504
newsyslog.conf, sample entry, 597
next generation RC scripts (rcNG), 87
nextboot command, 141, 142
NFS (Network File System). See Network
File System (NFS)
NFSCLIENT option, for kernel, 132
nfsd program, 289
NFSSERVER option, for kernel, 132
nice command, in tcsh shell, 585
nice processes, 577
niceness, for reprioritizing, 584–586
Nintendo GameCube emulator, 363
nmbclusers, 172
“no such user” error, returning message
to email sender, 482
noasync mounts, 219
noatime mount option, 220
noauto filesystem mount option, 212
nobody account, 263
nobody group, 194
I N D EX

695

NO_BUILDKERNEL option, for FreeSBIE, 633
NO_BUILDWORLD option, for FreeSBIE, 633

option, for FreeSBIE, 634
98, 101
NOERROR, in dig response, 415
noexec mount option, 220
nogroup group, 194
Nokia, 5
nologin shell, 192
nologin variable, in login.conf, 200
nomatch rules, for devd, 257
none level for syslog protocol, 589, 590
None option, for Apache, 508
None override, in .htaccess file, 511
nonrepudiation, from cryptosystems, 280
normal performance, defining, 571
nosuid mount option, 220
nosymfollow mount option, 220
NOTES file, 130, 138
notice level for syslog protocol, 588
notify rules, for devd, 257
NS record, 431
nscd program, 450
name query caching with, 451–453
and timing, 452
nslookup, 414
NTFS, 226–227
permissions, 228
NTP (Network Time Protocol), 448–450
ntp facility, 588
ntpd program, 447
configuring, 448
time correction, instant, 449
null modem cable
for serial console, 73
for Soekris, 617
Nvidia, 35
NO_COMPRESSDFS
nodump,

O
OK (loader prompt), 66
One-time Passwords In Everything
(OPIE), 310
opaques, as sysctl values, 121
open source software, support
mechanism, 20–21
OpenBSD, 7, 11–12
openfiles variable, in login.conf file, 199
OpenOffice.org suite, 13, 325, 331
dependencies, 405
OpenRADIUS, 513
OpenSolaris, 8, 227
OpenSSH, 442, 445, 447
OpenSSL, configuration, 281–282
openssl command, 496, 497

696

I ND EX

openssl s-client command, 285
operating system
BIOS location of, 62
determining version and platform, 29
for diskless client
download with TFTP, 461
on network, 607
running software from other, 86–87,
361–365
separating data from, 41
operator group, 194
OPIE (One-time Passwords In
Everything), 310
optimizing performance
with parallel builds, 390
partitioning and, 38
Options, for virtual host, 519
options criteria, in kernel configuration
file, 129
O’Reilly Network BSD Developer
Center, 26
OSI network protocol stack, 146
outbound traffic, on network, 166
outgoing connections, and aliases, 164
output, 14
overlap, in ABI implementation, 363
ownership
of device, changing, 254
of file, changing, 193
of log file, 594
of non-Unix filesystem, 228

P
environment variable, for
pkg_add, 326
packages. See software packages
PACKAGESITE environment variable, 331, 339
for pkg_add, 326
packet filtering, 264, 272–280
activating rules, 279–280
configuring, 275–277
default accept and default deny,
273–274
rule sample, 278–279
with SNMP, 600
and stateful inspection, 274–275
packet normalization, in /etc/pf.conf
file, 276
packet sniffer, and telnet, 440
packets, 149
PAE (Physical Address Extensions), 139,
142–143
“Page not Found” error, 512
pages of memory, 572
PACKAGEROOT

paging, 581–582
vmstat information on, 572–573
PAM (Pluggable Authentication
Modules), 310
panic function, 644–645
panics
causes, 637–638
configuring computer to process, 639
manual crash dumps, 642–643
recognizing, 638–639
responding to, 639–642
and serial consoles, 641
triggering in single-user mode, 645–646
parallel builds, optimization with, 390
PARANOID keyword, in TCP wrappers,
267, 269
parent-child relationship, of processes, 580
parity, 542–543
partitions, 211–212, 531. See also root
partition
block size for, 41
creating, 241
during install process, 52–53
slices, 536–537
de-journaling, 557
encrypted, 80
/etc/fstab file entry for, 241
GELI to encrypt, 558–560
for jails, 290–291
mounting onto filesystem, 214–215
replication of, 539–540
restoring filesystem on empty, 103
space remaining, 215–217
for swap space, 537
tar for backups, 95
unmounting, 215
pass command, 496
passphrase, 284, 559, 560
passwd command, 65, 184–185
passwd_format option, 201
passwdtype setting, in adduser.conf file, 183
password expiration field, in
/etc/master.passwd file, 187
password keyword, in /etc/nsmb.conf file, 250
passwords
for Apache, 512–515
files on diskless system, 615
for FTP, 522
for groups, 191
for single-user mode, 605
for user accounts, 182
changing, 184–185
expiration dates, 186
file for, 184

patch of source code files, 318
patching system, 180
path, to loaded kernel, 67
path variable, in login.conf, 200
PC Weasel, 71
pc98, 34, 36
pciconf command, 127
per-client directory, for diskless hosts,
612–613
performance
computer resources and, 570
disk input/output, 574–575
following processes, 580–581
of network, 571
optimization, partitioning and, 38
problem solving, 570
of software, 586
status mail, 586–587
swap and paging, 581–582
system status information, 575–579
tuning, 582–586
CPU usage, 583–584
memory usage, 583
swap space, 583
periodic command, 586
Perl, 323
PermiRootLogin, for SSH, 443
permissions
for Apache, 507–515
of device node, 254
and foreign filesystems, 228
for log file, 594
NFS and, 246
restoring for extracted files, 97
Revision Control System and, 108
viewing with ls, 193
personal data field, in /etc/master.passwd
file, 187
per-subnet directory, for diskless hosts,
612–613
PF (packet filter), 273
activating rules, 279–280
configuring, 275–277
enabling, 273
rule sample, 278–279
PF module for bsnmpd, 602
pf plug-in, for FreeSBIE, 635
pfctl program, 279–280
pf_enable variable, 83
_pflogd group, 194
PGID (process group ID), 580
pgrep command, 585
phpt Apache module, 506
Physical Address Extensions (PAE), 139,
142–143
I N D EX

697

physical layer, in OSI, 146, 149
PID (process ID), 576, 578
pidfile, 597
pinging IP addresses, 163
pkg_add program, 325, 613
environment settings in, 326–327
pkg_delete program, 328–329, 337
PKGDIR environment variable, for
pkg_add, 327
PKGFILE option, for FreeSBIE, 633
pkg_info tool, 329
PKG-TMPDIR environment variable, for
pkg_add, 326
pkill command, 578
plaintext, for email, 30
platters, 210, 530
Pluggable Authentication Modules
(PAM), 310
plug-ins, for FreeSBIE, 634–635
PocketPC devices, 7
Point to Point Protocol (PPP), 135, 147
politeness, in asking for help, 29–30
polling, in network, 174
POP (Post Office Protocol), 493
testing with SSL, 496–497
portability of FreeBSD, 10
portaudit tool, 340
porting, 361
portmaster script
checking all dependencies, 408
initial setup, 405–406
ports, 317–320
build options for, 337
cleaning up, 338–339
directory, 44
ignoring, 408–409
installing, 332–334
integrated customizations, 334–336
names, 74–75
redirecting, PF and, 276
security, 340–341
tracking build status, 338
for transport protocols, 157–158
closing, 207
connecting to SSL-protected, 285–286
information on, 167–168
netstat for information on, 168–169
reserved, 158
reviewing open, 207
for SSH, 442
for virtual host, 519
tree
contents, 318–320
installing, 318
reducing size of, 409

698

I ND EX

uninstalling and reinstalling, 337–338
updating installed, 404–409
Ports Collection, 10
for Apache modules, 506
browsing, 322
installing, 54
software from, 331–340
legal restrictions, 322
upgrading, 403–404
software to version in, 404
portsnap program, 318, 403–404
portupgrade tool, 405
POSIX, 120
standards, 362
threads, 359
Post Office Protocol (POP), 493
testing with SSL, 496–497
Postfix, 16, 473
postmaster account, 470
forwarding email for, 478
power for FreeBSD, 10
powerpc, 34, 36
PPP (Point to Point Protocol), 135, 147
PPP over Ethernet (PPPoE), 147
Preboot Execution Environment
(PXE), 607
installation, 45
preemption options, 132
preemptive multitasking, 345
PREFIX variable, for make, 339
primary server, setup for mirroring, 566
print servers, 459–461
printers, 86
/etc/printcap file for settings, 460–461
printing, 459–461
priorities, niceness and, 584–586
Priority field, in problem report, 651
priority variable, in login.conf, 200
private key, 280
problem report (PR)
actions before filing, 647–648
after submission, 653–654
bad versions, 648–649
good versions, 649–652
response to, 639
sample, 652–653
submitting, 646–654
problem-solving resources, 26–28
process filesystem, 238
process ID (PID), 576, 578
process group ID (PGID), 580
process state, 580
processes, 358
counts, 576
displaying all on system, 294

following, 580–581
parent-child relationship of, 580
priorities, 584–586
top list of, 578–579
vmstat statistics on, 572
processor. See central processing
unit (CPU)
procfs filesystem, 238
and jails, 295
for Linux, 367
PROCFS option, for kernel, 133
production system, changing clock on, 449
programs
attaching shared libraries to, 355–357
compiling, 316–317
/etc/rc.conf to disable, 65
forwarding email to, 479
identifying those listening to
network, 207
identifying those requiring libraries, 394
library requirements, 358
logging by name, 590–591
scripts to manage running, 353
sending log messages to, 591
Project Evil, 36
prompt command (FTP), 523
proprietary hardware, 35
proprietary operating system, 12
protocols
fitting together, 157
and hardware, 159–160
in network layers, 146–148
provider in GEOM, 530
proxy group, 194
PRUNE_LIST option, for FreeSBIE, 633
PS/2 mouse, setup, 56
ps command, 294, 576
pseudodevices, 129
kernel options, 135–136
PSEUDOFS option, for kernel, 133
pseudorandom number generation, random
device for, 135
pseudoterminal, 135, 604, 605
pty device, 135
.pub file extension, 441
public key encryption, 280–286
certificates, 282–285
connecting to SSL-protected ports,
285–286
OpenSSL configuration, 281–282
for SSH, 441
put command (FTP), 523
PuTTY, 445
pw command, scripting with, 188

PXE (Preboot Execution
Environment), 607
installation, 45
pxeboot file, for diskless clients, 609

Q
Qmail, 473
qotd (Quote of the Day), 158
implementing, 454–455
question mark (?), to display commands
for loader, 66
quit command, for restore, 104
quotas, for disk space, 40

R
rackmount servers, booting, 45
Radius user authentication, 513–515
RAID (Redundant Array of Independent
Disks), 541–544
controllers, 541
hardware vs. software, 541–542
nested, 553, 555
parity and stripe size, 542–543
types, 543–544
RAID-0 (striping), 543
configuring, 545–547
RAID-1 (mirroring), 543, 547–550
boot disks, 549–550
daily status check of, 550
repairing, 548
RAID-3 (striping with dedicated parity
disk), 543, 550–553
destroying, 553
RAID-5 (striping with parity shared across
all drives), 543
RAID-10 (stripe of mirrored disks), 544,
553–554
RAM (random access memory),
requirements, 37
Rambler search engine, 26
random device, for pseudorandom number
generation, 135
random password generator, 182
rc scripts
debugging, 353–354
example, 351–352
self-ordering, 350
special providers, 352–353
rc_debug variable, 80
rc_info variable, 80
rcNG (next generation RC scripts), 87
rcorder program, 351
rcs command, breaking locks with, 112
I N D EX

699

rcsdiff command, 110–111
README file, for Ports Collection, 319
README.TXT directories, 44
read-only
mode, 81
for FTP server, 524
mounts, in FFS, 218–219
read-write mounts in FFS, forcing on
dirty disks, 224
real memory, listing in startup
messages, 77
real-time streaming data, UDP for, 156
@reboot, 466
reboot command, 88
automatic, after system panic, 638, 642
rebuild of software, forcing, 407–408
recpt to: command, 471
recursive query, 412
Red Hat Linux, 315
redirections for email, 478
redundancy
in network servers, 175
restoring to RAID-3, 551–552
Redundant Array of Independent Disks
(RAID). See RAID (Redundant Array
of Independent Disks)
Reed, Darren, 273
refresh time, for slave nameserver,
430, 431
refreshing zone, 435
refuse file, 386–387
regular expressions, 597
regular user account, creating, 57
reinstalling ports, 337–338
ReiserFS, 227
rejecting logins, 195
relay control, in mail server, 472
Release Engineering team, 372
Release field, in problem report, 651
release versions of FreeBSD, 372–373
releases directory, 44
reloading nameserver, 435
Remote Name Daemon Control (rndc),
434–435
command, 193, 434–435
remote printers, 459
Remote Procedure Calls (RPC), 313
remote reset of computer, serial console
for, 70–75
removable hardware
ejecting media, 231
kernel support for, 136
removable media
/etc/fstab file and, 231
filesystems, 228–231

700

I ND EX

replication, of slices and partitions,
539–540
require-group statement, in .htaccess file, 515
requirehome variable, in login.conf file, 198
rescheduling, for CPU performance, 584
/rescue directory, 65
research, before sending problem
report, 647
reserved ports, 158
resolver, 412
configuring, 419–421
nameserver list for, 421
resource limits, 198–199
response to hacking, 300
restore program for dump, 98, 101–104
and further backups, 103
interactive, 104
restoring
disklabels, 538
slice table, 533
retensioning tape, 93
retry requirement, for milter-greylist, 490
retry value, for slave nameserver, 430, 431
reverse DNS, 413
zones, 433–434
reversed IP addresses, 418
revision control, 106–113
breaking locks, 112–113
checking back in, 108–109
editing files, 108
initializing, 107–108
reviewing files revision history, 110–111
viewing logs, 109–110
revision history, reviewing, 110–111
rewinding backup tapes, 91, 93
RIP (Routing Information Protocol), 85
rkhunter, 300
rlog command, 109
rmuser program, 188
rndc (Remote Name Daemon Control),
434–435
command, 193, 434–435
rndc-confgen script, 434
ro filesystem mount option, 212
roaming users, email service to, 472
root
directory, for tftpd server, 461
partition, 38, 214, 537
device names for, 64
in /etc/fstab, 213
running mtree across, 297–298
in single-user mode, 64
password, 189–190
for jails, 292
resetting, 65

posts, in portmaster, 405
server, 412
user
for changing accounts, 186
minimizing use of, 57
NFS server and, 246
ownership of non-Unix filesystem, 228
status mail to, 586
using groups to avoid, 191–194
zone, in named.conf file, 425
rootmfs plug-in, for FreeSBIE, 635
root_rw_mount variable, 81
rotatelogs program, 504–505
rotating logs, 593
file size as basis, 595
flags for actions, 596
time as basis for, 595–596
route command, 163
router_enable knob, 85
Routing Information Protocol (RIP), 85
RPC (Remote Procedure Calls), 313
rpcbind program, 289
RSA key files, 441
rtld program, 355–356
running processes, 576
run-time tunable sysctl, 122
rw filesystem mount option, 212

S
safe mode, booting in, 63
Samba, 248
SAS drives, 37
SASL (Simple Authentication and Security
Layer), 491
testing, 493
saslauthd daemon, 492
SATA drives, 37
savecore program, 640, 641
/sbin directory, commands in, 64–65
sbsize variable, in login.conf file, 199
schedulers, 349–350
scheduling
adjusting for CPU performance, 584
binary updates, 380
tasks, 463–466
schg file flag, 202
scp program, 446, 527
screen, blank, for idle system, 85
script command, 114
script kiddies, 178
and twist for system overload, 270
scripts
for customizing NanoBSD, 627–628
to manage running programs, 353

shutdown, 350–354
from vendor, 353
SCSI drives, 37
numbering, 238–239
tape, 90
device nodes, 91
wiring down, 238–239
SCSI_DELAY option, for kernel, 133
SCTP (Stream Control Transmission
Protocol), 157
sdiff command, 392
search keyword, in /etc/resolv.conf, 421
sections, specifying for man search, 24
sectors in disk drives, 210, 531
Secure Shell (SSH), 82, 282. See also SSH
(Secure Shell) daemon
Secure Sockets Layer (SSL). See SSL
(Secure Sockets Layer)
secure websites, 520, 521
securelevels, 204–207
definitions, 204–205
and file flags, 201
limitations, 206
security
default accept vs. default deny, 264–265
for diskless NFS, 610
Ethernet and, 159
file flags, 201–203
limitations, 206
for file transfer, 522
FreeBSD announcements, 180
for geom_gate, 562
groups of users, 190–191
to avoid root user, 191–194
default, 194
HyperThreading and, 348
for inetd, 453
intrusion preparation with mtree,
296–299
jails, 286–296
and /etc/rc.conf, 293
client setup, 290–291
host server setup, 287–289
in-jail setup, 291–292
and kernel, 289–290
limitations, 295–296
managing, 294–295
shutdown, 293–294, 295–296
startup, 293–294
and LD_LIBRARY_PATH environment
variable, 358
for nameserver, 436–437
network targets, 206
network traffic control, 263–264

I N D EX

701

security, continued
packet filtering, 272–280
activating rules, 279–280
configuring, 275–277
default accept and default deny,
273–274
rule sample, 278–279
and stateful inspection, 274–275
for ports and packages, 340–341
potential attackers, 178–180
public key encryption, 280–286
certificates, 282–285
connecting to SSL-protected ports,
285–286
OpenSSL configuration, 281–282
response to hacking, 300
restricting login ability, 195–197
restricting system usage, 197–201
risks, 177–178
root password, 189–190
securelevels, 204–207
definitions, 204–205
limitations, 206
shells, 188–189
for SNMP, 600
system monitoring, 299–300
TCP wrappers, 265–272
configuring, 265–271
example, 271–272
unprivileged users, 261–263
user accounts, 181–188
creating, 181–183
deleting, 188
editing, 183–188
and vmcore file, 645–646
workstation vs. server, 207
security facility, 588
security.bsd.hardlink_check_gid sysctl, 674
security.bsd.overworked_admin sysctl, 674
security.bsd.see_other_gids sysctl, 674
security.bsd.see_other_uids sysctl, 674
security.bsd.unprivileged_read_msgbuf

sysctl, 674
security.jail.allow_raw_sockets
security.jail.allow_raw_sockets

sysctl, 673
sysctl

value, 290
security.jail.chflags_allowed sysctl,

673

security.jail.chflags_allowed sysctl

value, 290
security.jail.enforce_statfs

sysctl, 673

security.jail.enforce_statfs sysctl value,
security.jail.jailed sysctl,

673
security.jail.jailed sysctl value, 290
security.jail.list sysctl, 673
security.jail.list sysctl value, 290

702

I ND EX

290

security.jail.set_hostname_allowed
security.jail.set_hostname_allowed

sysctl, 673
sysctl

value, 289
security.jail.socket_unixiproute_only

sysctl, 673
security.jail.socket_unixiproute_only sysctl

value, 289
security.jail.sysvipc_allowed sysctl,
security.jail.sysvipc_allowed sysctl

673
value,

289–290
SELECT command,

for IMAP, 497
selecting options from sysinstall menus,
spacebar for, 50
Sendmail Mail Transfer Agent, 86,
473–476
attaching milter-greylist to, 490–491
authentication with SASL, 491–493
configuration options, 475, 476–481
Makefile for, 484–485
submission vs. reception, 474–475
sendmail.cf file, 483
building, 492–493
sendmail_enable variable, 86
sendmail_outbound_enable variable, 86
send-pr, 649
serial consoles, 70–75
disconnection, 75
hardware, 71
and panics, 641
physical setup, 73
software, 71–72
speeds for Soekris, 626–627, 628
use, 73–75
Serial Line Internet Protocol (SLIP), 135
serial number, of zone file, 429–430
serial port
logging in through, 606
on Soekris, as default console, 617
server farm, diskless system for, 606
Server Message Block (SMB), 248
ServerName, for Apache, 503
ServerRoot setting, for Apache, 502
servers
device node management on, 253
security, vs. workstation, 207
SERVERS provider, for rc scripts, 352
server-side includes, 509
set command, to change variables, 67
setenv command, 92
setenv variable, in login.conf, 200
setuid programs, preventing running, 220
Severity field, in problem report, 651
sftp program, 447, 527
shared libraries, 87, 354–358
attaching to programs, 355–357

remapping, 360–361
versions, 354
shell variable, in login.conf, 200
shells, 188–189
changing for user account, 185
kernel option to choose, 64
for user account, 182
for users, 58
show command, 67
shrinking FreeBSD, 396–397
shutdown, 88
of Apache, 521
jails, 293–294, 295–296
and memory disk erasure, 233
scripts, 350–354
from vendor, 353
syncer at, 224
signatures, in email, 30
Silicon Graphics, IRIX, 9
Simple Authentication and Security Layer
(SASL), 491
testing, 493
simple device, 616
Simple Mail Transfer Protocol (SMTP),
470–472
Simple Network Management Protocol
(SNMP), 598–601
client (agent), 598
Management Information Base (MIB),
598–600
definitions and browsers, 599–600
security for, 600
Single Unix Specification, 9
single-user mode
for boot process, 63–66
disks in, 64
network in, 65
programs available, 64–65
fixit disk for, 114–115
root command prompt in, 605
triggering panic in, 645–646
upgrades and, 395–396
64-bit computing, 369
skilled attackers, 179–180
sl device, 135
slash notation for IP addresses, 153
for configuring interface, 162
slashes (//), for comments, 424
slave domain, configuring, 426–427
sleeping processes, 576
slice table
backups, 533
changing, 533–536
viewing with fdisk, 532–533

slices, 211, 531
creating, 240
fdisk for splitting hard drive into,
534–536
partitioning, 536–537
replication of, 539–540
SLIP (Serial Line Internet Protocol), 135
smart hosts, for email, 484–485
SMB (Server Message Block), 248
smbfs.ko module, 249
smbutil crypt command, 250
smbutil login command, 250
smmsp group, 194
SMP (symmetric multiprocessing). See
symmetric multiprocessing (SMP)
SMTP (Simple Mail Transfer Protocol),
470–472
snappnd file flag, 202
snapshots
in FFS, 222
of FreeBSD, 375–376
snapshots directory, 44
SNMP (Simple Network Management
Protocol). See Simple Network
Management Protocol (SNMP)
Snort, 334, 335
SOA (Start of Authority) record, 428–429
sockstat program, 167–168, 207
Soekris, 138, 616
serial console speeds, 626–627, 628
serial port as default console, 617
soft updates, 219, 220–221
vs. journaling, 555
kernel options, 132
software
finding, 320–322
by keyword, 321–322
by name, 321
forcing rebuild, 407–408
identifying and upgrading, 406–407
identifying unneeded, 406
make for, 316
management, 10
performance tuning, 586
recompilation, 362
running from other operating system,
86–87, 361–365
and source code, 316–317
software packages, 317–320, 322–331. See
also libraries
adding, 56–57
to NanoBSD, 628
building, 339
on CDs, 322–323
from FTP, 324–325
I N D EX

703

software packages, continued
information, 329–330
installing, 325
on diskless systems, 613–615
listing installed with description, 329
problems, 330–331
result of installing, 327–328
security, 340–341
uninstalling, 328–329
software serial consoles, 71–72
Solaris, 8
recompiling software for, 362
sound plug-in, for FreeSBIE, 635
source code, 377
building FreeBSD, 388–396
GENERIC kernel, 389
installing userland, 393–395
make buildworld command, 388–389
optimizing with parallel builds, 390
preparing for userland install,
390–393
and software, 316–317
updating, 387
for upgrading FreeBSD, 382–387
spacebar, for selecting options from
sysinstall menus, 50
spam. See also junk email
mail server access for, 472
rejecting sources, 485–486
spambots, retransmission times, 490
spamd program, 487
Spamhaus, 486
spamming tools, and backup MXs, 469
SPARC servers, 7
sparc64, 34
spawn option, for TCP wrappers, 270–271
specification for mtree, 297–298
saving, 298–299
spool directory, for printer, 461
SRC_CONF option, for FreeSBIE, 633
SRCDIR option, for FreeSBIE, 632
SSH (Secure Shell) daemon, 82, 167–168,
439–447
clients, 445–447
copying files, 446–447
server (sshd), 289, 440–442
configuring, 442–444
stopping, 440
user management, 444–445
SSH keys, for diskless system, 614
SSH login, option to configure, 56
sshd group, 194
sshd script, 88
ssh-keygen command, 441–442

704

I ND EX

SSL (Secure Sockets Layer), 282
connecting to protected ports, 285–286
host key, 283
for web traffic, 520
ssl_cert_file variable, for Dovecot,
494–495
ssl_key_file variable, 494–495
stackable mounts, 242–243
stacksize variable, in login.conf file, 199
staff group, 194
standard error, 14–15
standard input, 14
standard output, 14
Start of Authority (SOA) record, 428–429
startup. See also boot process
options in /etc/rc.conf file, 80
messages, 76–79
/var/run/dmesg.boot for storing, 78
scripts, 350–354
from vendor, 353
state of process, 579
stateful inspection, and packet filtering,
274–275
stateful protocol, 156
stateless protocol, 156
status command (GEOM), 544
stop command, for service, 88
storage devices, device nodes for, 211
Stream Control Transmission Protocol
(SCTP), 157
streaming protocol, 156
strings, as sysctl value, 121
stripe of mirrored disks (RAID-10), 544,
553–554
striped provider, creating, 546
striping
(RAID-0), 543
configuring, 545–547
with dedicated parity disk (RAID-3),
543, 550–553
with parity shared across all drives
(RAID-5), 543
su command (switch user), 189
subdirectories, displaying size in
blocks, 216
submit.cf file, 483
subnet statement, for DHCP clients,
458–459
Subsystem sftp setting, for SSH, 444
Sun Microsystems, 3, 8
sunlnk file flag, 202
superuser, password changes by, 184–185
supfiles, 383–384
examples, 386

modifying, 384–385
for upgrading FreeBSD, 398
SVR4 (System V Release 4), 364
swap space, 212
analysis, 581–582
encrypting with GELI, 561
in /etc/fstab, 213
/etc/fstab file entry for, 241
/etc/rc.conf file setting for, 80
and panic preparation, 640
partition for, 38–39, 537
and performance, 583
splitting among multiple drives, 41
swap-backed disks, 232
creating, 233
swapfind plug-in, for FreeSBIE, 635
switch user command (su), 189
switches for Ethernet, 159
failure, 159–160
quality of, 171
symbol versioning, 354
symbols files, 641, 644
symlinks, 509
disabling, 220
for library, 354
SymLinksIfOwnerMatch option, for
Apache, 509
symmetric multiprocessing (SMP), 143,
344–349
current implementation, 346–347
first implementation, 345–346
kernel options, 134
processors and, 347–348
using, 348–349
SYN-ACK packets, forged, 274
syncer at shutdown in FFS, 224
synchronization of time, 448
synchronous mounts in FFS, 219
Synopsis field, in problem report, 650, 652
sys group, 194
/sys/conf/NOTES file, 138
sysctl command, 121–122
sysctl values, 69, 121, 661–674
changing, 122–124
for device drivers, 123–124
setting automatically, 122
viewing, 121–122
sysinstall
for adding hard disks, 240–243
for upgrades, 377
screen, 49–50
for reconfiguring system, 59
syslog facility, 588

syslogd daemon, 82, 587–593
customization, 592–593
facilities for log entry, 587–588
on jail host server, 288
levels for log message, 588–589
processing messages with, 589–592
syslogd.conf, for diskless system, 615
syslogd_enable variable, 82
syslogd_flags variable, 82
SyslogFacility AUTH, for SSH, 443
systat, 571
system
accounts, 192
backups, 90
binaries, preventing replacement with
trojan versions, 203
console, physical protection of, 189
keeping up to date, 180
memory, Windows loading of, 315
monitoring, 299–300
tar to backup, 95
traps, vmstat information on, 573
System-V-style shared memory, options to
enable, 133

T
tables, in /etc/pf.conf file, 276
tail command, 476
tape drives
and dump, 99
moving backwards on, 105
$TAPE variable, 91–92
tapes for backups, 90–93
erasing, 93
multiple backups on one tape, 105–106
retensioning, 93
rewinding, 91, 93
tar (tape archiver) program, 94–97
modes, 94–96
for moving files to new drives, 242
tarball, 94
creating, 96
.tar.gz file extension, 97
.tar.Z file extension, 97
.taz file extension, 97
tclhttpd, 500
TCP (Transmission Control Protocol). See
Transmission Control Protocol (TCP)
TCP wrappers, 265–272, 456
and access to network daemons, 264
configuring, 265–271
ALL keyword, 268
client list, 266–267
daemon name, 266
I N D EX

705

TCP wrappers; configuring, continued
logging, 269
options, 269
spawn option, 270–271
twist option, 269–270
example, 271–272
tcp_extensions variable, 83
TCP/IP, 2, 145
configuring, 54–55
kernel options, 132
network layers, 146–148
variables for, 83
tcsh shell, nice command, 585
telnet, 285–286, 440
to connect to SMTP port, 470
to connect to SSH TCP port, 440–441
/temp directory, for memory filesystem, 232
Templates directory, for Ports
Collection, 319
term variable, in login.conf, 200
terminals, 604
emulators, 74
servers, 73
testing
boot-time tunable sysctls, 124
for FreeBSD, 376
IMAPS, 497–498
kernel remotely, 141–142
Linux mode, 366–367
network interface configuration, 166
POP3S, 496–497
SASL (Simple Authentication and
Security Layer), 493
text editors
for problem report, 649
for user management, 185
XEmacs, 13
text wrap, in email, 30
TFTP (Trivial File Transfer Protocol),
461–463
tftpd server, 461
and boot loader, 609
configuring, 462–463
file ownership, 462
and read-write files, 462
.tgz file extension, 97
third-party directory services, authentication against, 513
32-bit computing, 369
threaded programs, and multiple
processors, 348–349
threading libraries, 359–360
threads, 346, 358–359
waiting for CPU time, 576
three-way handshake, 156

706

I ND EX

three-way mirror, for database, 550
Thunderbird, FreeBSD support for, 493
Tier 1 NTP servers, 448
Tier 2 NTP servers, 448
tilde (~)
in environment fields for user home
directory, 200
for tip program, 75
time
for log rotation, 595–596
network, 447–450
specifying for crontab, 465
time servers, 448
time slice, 345
time zone setting, 56, 447
timecounter, identifier in startup
messages, 76
timeout, for FTP session, 524
time-to-live (TTL)
in dig answer, 415
for zones, 428
times.allow option, 201
times.deny option, 201
timestamps, and dump, 100
timezone variable, in login.conf, 200
tip program, 74–75, 617
disconnecting serial console, 75
/tmp directory, 39
tmpfs memory filesystem, 233–234
tmpmfs variable, 80
tmpmfs_flags variable, 80
tmpsize variable, 80
toggles, 662
tools directory, 44, 319
top tool, 575
and I/O, 579
torrents directory, 44
touch command, 462, 512
tracking changes, revision control for,
106–113
tracks in disk drives, 210, 530
Transmission Control Protocol (TCP),
147, 156–157
handshake, 156
status of, 169
port, Apache binding to, 502
transport layer, in OSI, 147–148, 149, 150
trap function, 644–645
trimming kernel, 131–136
basic options, 131–134
CPU types, 131
device drivers, 134
multiple processors, 134
pseudodevices, 135–136
removable hardware, 136

Trivial File Transfer Protocol (TFTP),
461–463
trojan versions of binaries, preventing
replacement with, 203
troubleshooting
kernel builds, 137
NanoBSD build, 625–626
result of refuse files, 387
truncate command, 565
trunk ports, 405
truss program, 368–369
TTL (time-to-live). See time-to-live (TTL)
tty group, 194
tun device, 135
tunables, 662
low-level kernel vs. run-time, 123
turnables, 79
tutorials, 25
twist option for TCP wrappers, 269–270
variables, 271
tzsetup program, 447

U
uappnd file flag, 202
uchg file flag, 202

UDF (Universal Data Format), 226
setting, in adduser.conf file, 183
UDP (User Datagram Protocol), 147,
155–156
wrappers and, 265
ufs (Unix Fast File System), 212
UFS (Unix File System), 217
access control lists, kernel options, 132
UFS2, snapshot facility, 99
UFS_DIRHASH option, for kernel, 132
UID (user ID). See user ID (UID)
UIDs file, for Ports Collection, 319
ULE scheduler, 349–350
umask variable, in login.conf, 200
umount command, 215, 231
for foreign filesystems, 226
uname command, -a option, 29
undeliverable email, 469
uninstalling
ports, 337–338
software packages, 327, 328–329
Universal Data Format (UDF), 226
Universal Time Clock (UTC), 447
University of California, 4
Unix, 2
Unix Fast File System (ufs), 212
Unix Systems Laboratories (USL), 4
Unix-like systems, vs. Unix, 9
UNKNOWN keyword, in TCP wrappers, 267
udotdir

unload command,

68
in GEOM, 544
unloading kernel modules, 125
unmounting
disks, for configuring mirroring, 547
and ejecting removable media, 231
partitions, 215
unprivileged users, 261–263
untarring, 95
Update server, building, 380
updating
blocking, 386–387
installed ports, 404–409
UPDATING file, for Ports Collection, 319
upgrading
Ports Collection, 403–404
software, 406–407
upgrading FreeBSD, 371–409
binary updates, 378–380
cross-building, 399
methods, 377
procedure for, 11
and single-user mode, 395–396
via source, 382–387
supfiles and make for, 398
via sysinstall, 380–381
versions, 372–377
decision on use, 376–377
FreeBSD-current, 373–374
FreeBSD-stable, 374–375
releases, 372–373
snapshots, 375–376
testing, 376
uptime, statistics on, 576
USB hardware
FAT filesystem for, 228–230
filesystems for, 230
kernel support for, 136
tape drives, 90
user accounts, 181–188. See also root, user
changing shell, 185
creating, 181–183
deleting, 188
editing, 183–188
for jail, 292
user command, 496
user crontabs, vs. /etc/crontab, 463
User Datagram Protocol (UDP), 147,
155–156
wrappers and, 265
User distribution, 42
user facility, 588
user ID (UID), 181
for administrative users, 192–193
in /etc/master.passwd file, 187
I N D EX

707

user mapping, 481–483
User option, for Apache, 502
user processes, 577
user sessions, logging to, 591
userland
for diskless clients, NFS server and,
609–610
installing, 390, 393–395
usernames, 57
in /etc/master.passwd file, 187
for FTP, 522
users, 7
adding, 57–58
Apache configuration by, 510–511
home directories, 40
unprivileged, 261–263
user’s class field, in /etc/master.passwd
file, 187
user’s home directory field, in
/etc/master.passwd file, 188
user’s shell field, in /etc/master.passwd
file, 188
USL (Unix Systems Laboratories), 4
/usr partition, 40
/usr/bin/sendmail file, 474
/usr/compat/linux, 366
/usr/local/bin directory, 14
/usr/local directory, 339
/usr/local/etc/dhcpd.conf file, 457–459
/usr/local/etc/dovecot.conf file, 494
/usr/local/etc/mail/greylist.conf file, 488
/usr/local/etc/rc.d script, 207, 350
/usr/local/lib directory, 356
/usr/local/share/dovecot directory, 495
/usr/local/share/freesbie directory, 631
/usr/local/share/freesbie/conf, 631
/usr/ports directory, 318
/usr/ports/emulators directory, 363
/usr/ports/games directory, 42
/usr/ports/INDEX-7 file, 320
/usr/ports/LEGAL file, 322
/usr/ports/mail/dovecot directory, 494
/usr/ports/mail/mutt, 30
/usr/ports/mail/sendmail, 491
/usr/ports/net/cvsup-mirror port, 400
/usr/ports/security/nessus utility, 300
/usr/ports/sysutils/lsof utility, 300
/usr/ports/www directory, 500
/usr/share/syscons/fonts directory, 85
/usr/share/syscons/keymaps directory, 85
/usr/src/tools/tools/nanobsd directory, 618
/usr/src/UPDATING file, 388
UTC (Universal Time Clock), 447
uucp facility, 588

708

I ND EX

uucp group, 194
uunlnk file flag, 203

V
,v file extension, 107
/var partition, 39–40
/var/crash directory, 643
/var/db/pkg directory, 327
/var/db/sup/refuse file, 386
variables, 662
for loader, 67
in loader.conf file, 70
set command to change, 67
in sysctl.out file, 119
/var/log/maillog file, 476
for Dovecot errors, 496
/var/log/messages, 84, 347
for DNS configuration errors, 436
varmfs plug-in, for FreeSBIE, 635
/var/run/dmesg.boot file, 63, 78
for problem solving, 29
and tape drive recognition by system, 90
/var/tmp directory, 39
VAXes, 7
verbose logging, 593
booting with, 63
verbose mode for tar, 96
verbose_loading variable, 69
VeriSign, 282
version numbers, in revision control, 107
version of FreeBSD, in startup
messages, 76
VersionAddendum, 442
vfs.ffs.doasyncfree sysctl, 667
vfs.ffs.doreallocblks sysctl, 667
vfs.nfs.diskless_rootaddr sysctl, 667
vfs.nfs.diskless_rootpath sysctl, 667
vfs.nfs.diskless_valid sysctl, 611, 667
vfs.usermount sysctl, 667
vfs.usermount sysctl variable, 228
vfs.vmiodirenable sysctl, 667
vi, Revision Control System and, 108
video card, with serial port, 71
video, redirecting to serial port, 71
view command (smbutil), 251
Vigor, 185
vipw utility, 186–188
virtual domains, 481–483
virtual hosting
in Apache web server, 517–519
SSL configuration, 520
virtual memory
disk space for, 38–39
statistics, 571

virtual nodes (vnodes), 218
writing to disk at shutdown, 224
virtual terminal, 604
viruses, 179
vmcore file, 643
and security, 645–646
vm.exec_map_entries sysctl, 667
vmstat program, 571–574
continuous, 574
using, 573–574
vm.swap_enabled sysctl, 666
vm.swap_idle_enabled sysctl, 667
vm.swap_idle_threshold1 sysctl, 667
vm.swap_idle_threshold2 sysctl, 667
vm.v_cache_max sysctl, 666
vm.v_cache_min sysctl, 666
vm.v_free_min sysctl, 666
vm.v_free_reserved sysctl, 666
vm.v_free_target sysctl, 583, 666
vm.v_inactive_target sysctl, 666
vnode-backed disks, 232
creating, 233
vnodes (virtual nodes), 218
writing to disk as shutdown, 224
voluntary context switches, 579

W
warning level for syslog protocol, 588
warnings, from self-signed certificates, 285
web server. See also Apache web server
estimating needs, 173
functioning of, 500
websites, HTTPS, 520
welcome variable, in login.conf, 200
whatis, for man page searches, 23
wheel group, 189, 190, 194
whereis command, 406
whitelist, 487
wildcards, for logging, 589
window size, for network incoming
connections, 174–175
Windows (Microsoft)
background, and Unix
administration, 13
NT/200/XP filesystem, 226–227
WINS server, 459
wired memory, 578
wiring down SCSI devices, 238–239

WITHOUT_ options,

for customizing
FreeBSD, 397
WITNESS kernel option, 347
wlan_wep.ko module, 125
workgroup keyword, in /etc/nsmb.conf
file, 249
workstation, security, vs. server, 207
worms, 179
wrappers. See TCP wrappers
wrapping text, in email, 30
write caching in FFS, 221
write-only mode, for FTP server, 524
www group, 194

X
X Window System, 42, 323
dependencies required, 328
X11Forwarding, for SSH, 443–444
xautostart plug-in, for FreeSBIE, 635
xbox, 34
xconfig plug-in, for FreeSBIE, 635
xconfigure-probe plug-in, for
FreeSBIE, 635
X-Developer distribution, 42
XEmacs, 13
Xenix (Microsoft), 9
XFS partitions, 227
X-Kern-Developer distribution, 42
xorg package, 328
X-User distribution, 42

Y
Yahoo!, 5, 499

Z
ZEN blacklist, 486
ZFS, 227
zone files, 428–434
dots and termination in, 433
example, 432
reloading, 435
secondary nameservers update, serial
number and, 430
zone transfers, controlling, 436–437
zones in named.conf, 424

I N D EX

709

5IFSFT
NPSF
UP
LFFQJOH
'SFF#4%
GSFF
UIBO
NFFUT
UIF
FZF
4HEY WORK HARD BEHIND THE SCENES AND YOU HARDLY
EVER SEE THEM 4HEYRE 4HE &REE"3$ &OUNDATION
AND THEY QUIETLY FUND AND MANAGE PROJECTS SPONSOR
&REE"3$ EVENTS $EVELOPER 3UMMITS AND PROVIDE
TRAVEL GRANTS TO &REE"3$ DEVELOPERS 4HE &REE"3$
&OUNDATION REPRESENTS THE 0ROJECT IN EXECUTING
CONTRACTS LICENSE AGREEMENTS COPYRIGHTS TRADE
MARKS AND OTHER LEGAL ARRANGEMENTS THAT REQUIRE
A RECOGNIZED LEGAL ENTITY 4HE &OUNDATIONS FUNDING
AND MANAGEMENT EXPERTISE IS ESSENTIAL TO KEEP
&REE"3$ FREE !ND KEEPING IT FREE IS GETTING MORE
COSTLY EVERY YEAR
4HATS WHY THEY NEED YOUR HELP 4HE WORK OF 4HE
&REE"3$ &OUNDATION IS ENTIRELY SUPPORTED BY YOUR
GENEROUS DONATIONS
4O MAKE A DONATION
VISIT OUR WEB SITE AT

WWWFREEBSDFOUNDATIONORG
0LEASE HELP US TODAY (ELP KEEP &REE"3$ FREE

5IF
'SFF#4%
'PVOEBUJPO
10
#PY
 
#PVMEFS 
$0
 
64"
1IPOF
 

'BY
 

8FC
XXXGSFFCTEGPVOEBUJPOPSH

UPDATES

Visit http://www.nostarch.com/abs_bsd2.htm for updates, errata, and other
information.

COLOPHON

Absolute FreeBSD, 2nd Edition was laid out in Adobe FrameMaker. The font families
used are New Baskerville for body text, Futura for headings and tables, and
Dogma for titles.
The book was printed and bound at Malloy Incorporated in Ann Arbor,
Michigan. The paper is Glatfelter Thor 50# Smooth, which is made from
15 percent postconsumer content. The book uses a RepKover binding, which
allows it to lay flat when open.

President of

F R E E BSD:
NOT JUST FOR
ALPHA GEEKS

the FreeBSD Foundation

ANYMORE!

With a foreword by

ROBERT WATSON,

• Use advanced security features like packet filtering,
virtual machines, and host-based intrusion detection
• Build custom live FreeBSD CDs and bootable flash
• Manage network services and filesystems
• Use DNS and set up email, IMAP, web, and FTP
services for both servers and clients
• Monitor your system with performance-testing and
troubleshooting tools
• Run diskless systems

“ I L AY F L AT .”
This book uses RepKover — a durable binding that won’t snap shut.
Printed on recycled paper

• Integrate FreeBSD-specific SNMP into your network
management system
Whether you’re just getting started with FreeBSD or
you’ve been using it for years, you’ll find this book to
be the definitive guide to FreeBSD that you’ve been
waiting for.
ABOUT THE AUTHOR

Michael W. Lucas is a network engineer and system
administrator responsible for a network that stretches
across the Western Hemisphere. He is the author of the
critically acclaimed Absolute OpenBSD, Cisco Routers
for the Desperate, and PGP & GPG, all from No Starch
Press. Despite being from Detroit, Michigan, he knows
almost nothing about automobiles. He has been using
Unix systems for over 20 years and FreeBSD since 1995.
Fortunately for the rest of us, his writing keeps him too
busy to implement his plans for world domination.

$59.95 ($65.95 CDN)
SHELVE IN:
OPERATING SYSTEMS/UNIX

w w w.nostarch.com

• Implement redundant disks, without special hardware

LUCAS

T H E F I N E ST I N G E E K E N T E RTA I N M E N T ™

• Build custom network appliances with embedded
FreeBSD

2
2N
ND
D EE D
D II TT II O
ON
N

Absolute FreeBSD, 2nd Edition is your complete guide
to FreeBSD, written by FreeBSD committer Michael
W. Lucas. Lucas considers this completely revised and
rewritten second edition of his landmark work to be his
best work ever; a true product of his love for FreeBSD
and the support of the FreeBSD community. Absolute
FreeBSD, 2nd Edition covers installation, networking,
security, network services, system performance, kernel
tweaking, filesystems, SMP, upgrading, crash debugging,
and much more, including coverage of how to:

• Manage schedulers, remap shared libraries, and
optimize your system for your hardware and your
workload

ABSOLUTE
FREEBSD

FreeBSD—the powerful, flexible, and free Unix-like
operating system—is the preferred server for many
enterprises. But it can be even trickier to use than either
Unix or Linux, and harder still to master.

ABSOLUTE

F R2 NED E
B
S
D
EDITION
T H E

COMPLETE GUIDE TO FREEBSD
MICH A EL W. LUC A S

®



File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.6
Linearized                      : No
Page Mode                       : UseNone
XMP Toolkit                     : 3.1-702
Producer                        : Acrobat Distiller 7.0.5 (Windows)
Create Date                     : 2007:11:19 17:51:15Z
Creator Tool                    : FrameMaker 7.2
Modify Date                     : 2008:01:20 18:09:07+03:00
Metadata Date                   : 2008:01:20 18:09:07+03:00
Format                          : application/pdf
Creator                         : .
Title                           : 
Document ID                     : uuid:6095f6cf-92d1-41d0-9b3d-749f45e7b3b1
Instance ID                     : uuid:dfc401d8-82b8-49fe-9b19-5685d0c0f193
Has XFA                         : No
Page Count                      : 745
Page Layout                     : SinglePage