Security Engineering: A Guide To Building Dependable Distributed Systems Engineering

User Manual:

Open the PDF directly: View PDF .
Page Count: 1083 [warning: Documents this large are best viewed by clicking the View PDF Link!]

Cover
Title Page
Copyright
Contents
Preface to the Second Edition
Foreword by Bruce Schneier
Preface
Acknowledgments
Part I
Part II
Part III
Bibliography
Index
EULA

Security Engineering

A Guide to Building

Dependable Distributed

Systems

Second Edition

Ross J. Anderson

Wiley Publishing, Inc.

Security Engineering: A Guide to Building Dependable Distributed Systems,

Second Edition

Published by

Wiley Publishing, Inc.

10475 Crosspoint Boulevard

Indianapolis, IN 46256

Published by Wiley Publishing, Inc., Indianapolis, Indiana

Published simultaneously in Canada

ISBN: 978-0-470-06852-6

Manufactured in the United States of America

10987654321

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any

means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections

107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or

authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood

Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be

addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317)

572-3447, fax (317) 572-4355, or online at http://www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with

respect to the accuracy or completeness of the contents of this work and speciﬁcally disclaim all warranties, including

without limitation warranties of ﬁtness for a particular purpose. No warranty may be created or extended by sales or

promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work

is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional

services. If professional assistance is required, the services of a competent professional person should be sought.

Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or

Website is referred to in this work as a citation and/or a potential source of further information does not mean that

the author or the publisher endorses the information the organization or Website may provide or recommendations

it may make. Further, readers should be aware that Internet Websites listed in this work may have changed or

disappeared between when this work was written and when it is read.

For general information on our other products and services or to obtain technical support, please contact our Customer

Care Department within the U.S. at (800) 762-2974, outside the U.S. at (317) 572-3993 or fax (317) 572-4002.

Library of Congress Cataloging-in-Publication Data

Anderson, Ross, 1956-

Security engineering : a guide to building dependable distributed systems / Ross J Anderson. — 2nd ed.

p. cm.

Includes bibliographical references and index.

ISBN 978-0-470-06852-6 (cloth)

1. Computer security. 2. Electronic data processing–Distributed processing. I. Title.

QA76.9.A25A54 2008

005.1–dc22 2008006392

Trademarks: Wiley, the Wiley logo, and related trade dress are trademarks or registered trademarks of John Wiley

& Sons, Inc. and/or its afﬁliates, in the United States and other countries, and may not be used without written

permission. All other trademarks are the property of their respective owners. Wiley Publishing, Inc. is not associated

with any product or vendor mentioned in this book.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be

available in electronic books.

To Shireen

Credits

Executive Editor

Carol Long

Senior Development

Editor

Tom Dinse

Production Editor

Tim Tate

Editorial Manager

Mary Beth Wakeﬁeld

Production Manager

Tim Tate

Vice President

and Executive Group

Publisher

Richard Swadley

Vice President

and Executive

Publisher

Joseph B. Wikert

Project Coordinator,

Cover

Lynsey Stanford

Proofreader

Nancy Bell

Indexer

Jack Lewis

Cover Image

©Digital Vision/Getty Images

Cover Design

Michael E. Trent

Contents at a Glance

Preface to the Second Edition xxv

Foreword by Bruce Schneier xxvii

Preface xxix

Acknowledgments xxxv

Part I

Chapter 1 What Is Security Engineering? 3

Chapter 2 Usability and Psychology 17

Chapter 3 Protocols 63

Chapter 4 Access Control 93

Chapter 5 Cryptography 129

Chapter 6 Distributed Systems 185

Chapter 7 Economics 215

Part II

Chapter 8 Multilevel Security 239

Chapter 9 Multilateral Security 275

Chapter 10 Banking and Bookkeeping 313

Chapter 11 Physical Protection 365

Chapter 12 Monitoring and Metering 389

Chapter 13 Nuclear Command and Control 415

vii

viii Contents at a Glance

Chapter 14 Security Printing and Seals 433

Chapter 15 Biometrics 457

Chapter 16 Physical Tamper Resistance 483

Chapter 17 Emission Security 523

Chapter 18 API Attacks 547

Chapter 19 Electronic and Information Warfare 559

Chapter 20 Telecom System Security 595

Chapter 21 Network Attack and Defense 633

Chapter 22 Copyright and DRM 679

Chapter 23 The Bleeding Edge 727

Part III

Chapter 24 Terror, Justice and Freedom 769

Chapter 25 Managing the Development of Secure Systems 815

Chapter 26 System Evaluation and Assurance 857

Chapter 27 Conclusions 889

Bibliography 893

Index 997

Contents

Preface to the Second Edition xxv

Foreword by Bruce Schneier xxvii

Preface xxix

Acknowledgments xxxv

Part I

Chapter 1 What Is Security Engineering? 3

Introduction 3

A Framework 4

Example 1–A Bank 6

Example 2–A Military Base 7

Example 3–A Hospital 9

Example 4–The Home 10

Deﬁnitions 11

Summary 15

Chapter 2 Usability and Psychology 17

Introduction 17

Attacks Based on Psychology 18

Pretexting 19

Phishing 21

Insights from Psychology Research 22

What the Brain Does Worse Than the Computer 23

Perceptual Bias and Behavioural Economics 24

Different Aspects of Mental Processing 26

Differences Between People 27

Social Psychology 28

What the Brain Does Better Than Computer 30

x Contents

Passwords 31

Difﬁculties with Reliable Password Entry 32

Difﬁculties with Remembering the Password 33

Naive Password Choice 34

User Abilities and Training 35

Design Errors 37

Operational Issues 39

Social-Engineering Attacks 40

Trusted Path 42

Phishing Countermeasures 43

Password Manglers 43

Client Certs or Specialist Apps 44

Using the Browser’s Password Database 44

Soft Keyboards 45

Customer Education 45

Microsoft Passport 46

Phishing Alert Toolbars 47

Two-Factor Authentication 47

Trusted Computing 48

Fortiﬁed Password Protocols 49

Two-Channel Authentication 49

The Future of Phishing 50

System Issues 52

Can You Deny Service? 53

Protecting Oneself or Others? 53

Attacks on Password Entry 54

Interface Design 54

Eavesdropping 55

Technical Defeats of Password Retry Counters 55

Attacks on Password Storage 56

One-Way Encryption 56

Password Cracking 57

Absolute Limits 57

CAPTCHAs 59

Summary 60

Research Problems 61

Further Reading

When I wrote the ﬁrst edition of this book, there was only a small handful

of notable papers on passwords, including classic papers by Morris and

Thompson [906], Grampp and Morris [550], and Klein [720], and some DoD

guidelines [377]. Since then there has arisen a large research literature on

phishing, with a compendium of papers published as [659]. Perhaps the

greatest gains will come when security engineers start paying attention to

standard HCI texts such as [1039], and researchers start reading widely in the

psychology literature.

A text I’ve found helpful is James Reason’s ‘Human Error’, which essentially

tells us what the safety-critical systems community has learned from many

years studying the cognate problems in their ﬁeld [1060]. Recently, we’ve

seen the ﬁrst book on security usability — a collection of the early research

papers [333]. There is also an annual workshop, the Symposium On Usable

Privacy and Security (SOUPS) [1240].

I’m loth to provide much of a guide to the psychology literature, as I don’t

know it as well as I ought to, and we’ve only just started on the project of

building ‘security psychology’ as a discipline. It will take some years for us

to ﬁnd which psychological theories and experimental results provide us with

62 Chapter 2 ■Usability and Psychology

useful insights. But here are some pointers. Tom Gilovich, Dale Grifﬁn and

Danny Kahneman put together a volume of papers summarising the state of

play in the heuristics and biases tradition in 2002 [529]; while a more gentle

introduction might be a book chapter by Richard Samuels, Steven Stich and Luc

Faucher discussing the tensions between that tradition and the evolutionary

psychologists [1106]. It may also be of interest that a number of psychologists

and primatologists (such as Nicholas Humphrey, Richard Byrne and Andy

Whiten) have argued that we evolved intelligence because people who were

better at deception, or at detecting deception in others, had more surviving

offspring — the so-called ‘Machiavellian Brain’ hypothesis [250]. This might

lead us to wonder whether security engineering is the culmination of millions

of years of evolution! (Other psychologists, such as Simon Baron-Cohen,

would deﬂate any such hubris by arguing that nurturing the young was at

least as important.) Further fascinating analogies with evolutionary biology

have been collected by Raphael Sagarin and Terence Taylor in their book

‘Natural Security’.

Finally, if you’re interested in the dark side, ‘The Manipulation of Human

Behavior’ by Albert Biderman and Herb Zimmer reports experiments on inter-

rogation carried out after the Korean War with US Government funding [162].

It’s also known as the Torturer’s Bible, and describes the relative effective-

ness of sensory deprivation, drugs, hypnosis, social pressure and so on when

interrogating and brainwashing prisoners.

CHAPTER

Protocols

It is impossible to foresee the consequences of being clever.

— Christopher Strachey

Every thing secret degenerates, even the administration of justice; nothing is safe

that does not show how it can bear discussion and publicity.

— Lord Acton

3.1 Introduction

If security engineering has a deep unifying theme, it is the study of security

protocols. We’ve come across a few protocols informally already — I’ve men-

tioned challenge-response authentication and Kerberos. In this chapter, I’ll

dig down into the details. Rather than starting off with a formal deﬁnition of

a security protocol, I will give a rough indication and then reﬁne it using a

number of examples. As this is an engineering book, I will also give many

examples of how protocols fail.

A typical security system consists of a number of principals such as people,

companies, computers and magnetic card readers, which communicate using

a variety of channels including phones, email, radio, infrared, and by carrying

data on physical devices such as bank cards and transport tickets. The security

protocols are the rules that govern these communications. They are typically

designed so that the system will survive malicious acts such as people telling

lies on the phone, hostile governments jamming radio, or forgers altering

the data on train tickets. Protection against all possible attacks is often too

expensive, so protocols are typically designed under certain assumptions

about the threats. For example, the logon protocol that consists of a user

64 Chapter 3 ■Protocols

entering a password into a machine assumes that she can enter it into the right

machine. In the old days of hard-wired terminals in the workplace, this was

reasonable; now that people log on to websites over the Internet, it is much

less so. Evaluating a protocol thus involves answering two questions: ﬁrst, is

the threat model realistic? Second, does the protocol deal with it?

Protocols may be extremely simple, such as swiping a badge through a

reader in order to enter a building. They often involve interaction, and do

not necessarily involve technical measures like cryptography. For example,

when we order a bottle of ﬁne wine in a restaurant, the standard wine-waiter

protocol provides some privacy (the other diners at our table don’t learn the

price), some integrity (we can be sure we got the right bottle and that it wasn’t

switched for, or reﬁlled with, cheap plonk) and non-repudiation (it’s hard for

the diner to complain afterwards that the wine was off). Blaze gives other

examples from applications as diverse as ticket inspection, aviation security

and voting in [185].

At the technical end of things, protocols can be much more complex. The

world’s bank card payment system has dozens of protocols specifying how

customers interact with cash machines and retail terminals, how a cash machine

or terminal talks to the bank that operates it, how the bank communicates with

the network operator, how money gets settled between banks, how encryption

keys are set up between the various cards and machines, and what sort of

alarm messages may be transmitted (such as instructions to capture a card).

All these protocols have to work together in a large and complex system.

Often a seemingly innocuous design feature opens up a serious ﬂaw. For

example, a number of banks encrypted the customer’s PIN using a key known

only to their central computers and cash machines, and wrote it to the card

magnetic strip. The idea was to let the cash machine verify PINs locally, which

saved on communications and even allowed a limited service to be provided

when the cash machine was ofﬂine. After this system had been used for many

years without incident, a programmer (who was playing around with a card

reader used in a building access control system) discovered that he could

alter the magnetic strip of his own bank card by substituting his wife’s bank

account number for his own. He could then take money out of her account

using the modiﬁed card and his own PIN. He realised that this enabled him

to loot any other customer’s account too, and went on to steal hundreds of

thousands over a period of years. The affected banks had to spend millions

on changing their systems. And some security upgrades can take years; at

the time of writing, much of Europe has moved from magnetic-strip cards to

smartcards, while America has not. Old and new systems have to work side

by side so that European cardholders can buy from American stores and vice

versa. This also opens up opportunities for the crooks; clones of European

cards are often used in magnetic-strip cash machines in other countries, as the

two systems’ protection mechanisms don’t quite mesh.

3.2 Password Eavesdropping Risks 65

So we need to look systematically at security protocols and how they fail. As

they are widely deployed and often very badly designed, I will give a number

of examples from different applications.

3.2 Password Eavesdropping Risks

Passwords and PINs are still the foundation on which much of computer

security rests, as they are the main mechanism used to authenticate humans

to machines. I discussed their usability and ‘human interface’ problems of

passwords in the last chapter. Now let us consider some more technical

attacks, of the kind that we have to consider when designing more general

protocols that operate between one machine and another. A good case study

comes from simple embedded systems, such as the remote control used to open

your garage or to unlock the doors of cars manufactured up to the mid-1990’s.

These primitive remote controls just broadcast their serial number, which also

acts as the password.

An attack that became common was to use a ‘grabber’, a device that would

record a code broadcast locally and replay it later. These devices, seemingly

from Taiwan, arrived on the market in about 1995; they enabled thieves lurking

in parking lots to record the signal used to lock a car door and then replay it

to unlock the car once the owner had left1.

One countermeasure was to use separate codes for lock and unlock. But

this is still not ideal. First, the thief can lurk outside your house and record

the unlock code before you drive away in the morning; he can then come

back at night and help himself. Second, sixteen-bit passwords are too short.

It occasionally happened that people found they could unlock the wrong car

by mistake (or even set the alarm on a car whose owner didn’t know he

had one [217]). And by the mid-1990’s, devices appeared which could try all

possible codes one after the other. A code will be found on average after about

215 tries, which at ten per second takes under an hour. A thief operating in a

parking lot with a hundred vehicles within range would be rewarded in less

than a minute with a car helpfully ﬂashing its lights.

So another countermeasure was to double the length of the password from

16 to 32 bits. The manufacturers proudly advertised ‘over 4 billion codes’. But

this only showed they hadn’t really understood the problem. There was still

1With garage doors it’s even worse. A common chip is the Princeton PT2262, which uses 12

tri-state pins to encode 312 or 531,441 address codes. However implementers often don’t read

the data sheet carefully enough to understand tri-state inputs and treat them as binary instead,

getting 212. Many of them only use eight inputs, as the other four are on the other side of the

chip. And as the chip has no retry-lockout logic, an attacker can cycle through the combinations

quickly and open your garage door after 27attempts on average.

66 Chapter 3 ■Protocols

only one code (or two codes) for each car, and although guessing was now

impractical, grabbers still worked ﬁne.

Using a serial number as a password has a further vulnerability: there may

be many people with access to it. In the case of a car, this might mean all the

dealer staff, and perhaps the state motor vehicle registration agency. Some

burglar alarms have also used serial numbers as master passwords, and here

it’s even worse: the serial number may appear on the order, the delivery note,

the invoice and all the other standard commercial paperwork.

Simple passwords are sometimes the appropriate technology, even when

they double as serial numbers. For example, my monthly season ticket for

the swimming pool simply has a barcode. I’m sure I could make a passable

forgery with our photocopier and laminating machine, but as the turnstile is

attended and the attendants get to know the ‘regulars’, there is no need for

anything more expensive. My card keys for getting into the laboratory where

I work are slightly harder to forge: the one for student areas uses an infrared

barcode, while the card for staff areas has an RFID chip that states its serial

number when interrogated over short-range radio. Again, these are probably

quite adequate — our more expensive equipment is in rooms with fairly good

mechanical door locks. But for things that lots of people want to steal, like cars,

a better technology is needed. This brings us to cryptographic authentication

protocols.

3.3 Who Goes There? — Simple Authentication

A simple example of an authentication device is an infrared token used in some

multistorey parking garages to enable subscribers to raise the barrier. This ﬁrst

transmits its serial number and then sends an authentication block consisting

of the same serial number, followed by a random number, all encrypted using

a key which is unique to the device. We will postpone discussion of how to

encrypt data and what properties the cipher should have; we will simply use

the notation {X}Kfor the message Xencrypted under the key K.

Then the protocol between the access token in the car and the parking garage

can be written as:

T−→ G:T,{T,N}KT

This is the standard protocol engineering notation, and can be a bit confusing

at ﬁrst, so we’ll take it slowly.

The in-car token sends its name Tfollowed by the encrypted value of

Tconcatenated with N,whereNstands for ‘number used once’, or nonce.

Everything within the braces is encrypted, and the encryption binds Tand

Ntogether as well as obscuring their values. The purpose of the nonce is

to assure the recipient that the message is fresh, that is, it is not a replay of

3.3 Who Goes There? — Simple Authentication 67

an old message that an attacker observed. Veriﬁcation is simple: the parking

garage server reads T, gets the corresponding key KT, deciphers the rest of the

message, checks that the nonce Nhas not been seen before, and ﬁnally that

the plaintext contains T(which stops a thief in a car park from attacking all

the cars in parallel with successive guessed ciphertexts).

One reason many people get confused is that to the left of the colon, T

identiﬁes one of the principals (the token which represents the subscriber)

whereas to the right it means the name (that is, the serial number) of the token.

Another is that once we start discussing attacks on protocols, we can suddenly

start ﬁnding that the token T’s message intended for the parking garage Gwas

actually intercepted by the freeloader Fand played back at some later time. So

the notation is unfortunate, but it’s too well entrenched now to change easily.

Professionals often think of the T−→ Gto the left of the colon is simply a hint

as to what the protocol designer had in mind.

The term nonce can mean anything that guarantees the freshness of a

message. A nonce can, according to the context, be a random number, a serial

number, a random challenge received from a third party, or even a timestamp.

There are subtle differences between these approaches, such as in the level

of resistance they offer to various kinds of replay attack, and they increase

system complexity in different ways. But in very low-cost systems, the ﬁrst

two predominate as it tends to be cheaper to have a communication channel

in one direction only, and cheap devices usually don’t have clocks.

Key management in such devices can be very simple. In a typical garage

token product, each token’s key is simply its serial number encrypted under a

global master key KM known to the central server:

KT ={T}KM

This is known as key diversiﬁcation. It’s a common way of implementing

access tokens, and is very widely used in smartcard-based systems as well.

But there is still plenty of room for error. One old failure mode that seems

to have returned is for the serial numbers not to be long enough, so that

someone occasionally ﬁnds that their remote control works for another car in

the car park as well. Having 128-bit keys doesn’t help if the key is derived by

encrypting a 16-bit serial number.

Weak ciphers also turn up. One token technology used by a number of car

makers in their door locks and immobilisers employs a block cipher known as

Keeloq, which was designed in the late 1980s to use the minimum number of

gates; it consists of a large number of iterations of a simple round function.

However in recent years an attack has been found on ciphers of this type, and

it works against Keeloq; it takes about an hour’s access to your key to collect

enough data for the attack, and then about a day on a PC to process it and

recover the embedded cryptographic key [172]. You might not think this a

practical attack, as someone who gets access to your key can just drive off with

68 Chapter 3 ■Protocols

your car. However, in some implementations, there is also a terrible protocol

vulnerability, in that the key diversiﬁcation is not done using the block cipher

itself, but using exclusive-or: KT =T⊕KM. So once you have broken a single

vehicle key for that type of car, you can immediately work out the key for any

other car of that type. The researchers who found this attack suggested ‘Soon,

cryptographers will drive expensive cars.’

Indeed protocol vulnerabilities usually give rise to more, and simpler,

attacks than cryptographic weaknesses do. At least two manufacturers have

made the mistake of only checking that the nonce is different from last time,

so that given two valid codes Aand B, the series ABABAB... was interpreted

as a series of independently valid codes. A thief could open a car by replaying

the last-but-one code. A further example comes from the world of prepayment

utility meters. Over a million households in the UK, plus many millions in

developing countries, have an electricity or gas meter that accepts encrypted

tokens; the householder buys a token, takes it home and inserts it into the

meter, which then dispenses the purchased quantity of energy. One electricity

meter widely used in South Africa checked only that the nonce in the decrypted

command was different from last time. So the customer could charge the meter

up to the limit by buying two low-value power tickets and then repeatedly

feeding them in one after the other [59].

So the question of whether to use a random number or a counter is not as easy

as it might seem [316]. If you use random numbers, the lock has to remember

a reasonable number of past codes. You might want to remember enough of

them to defeat the valet attack. Here, someone who has temporary access to the

token — such as a valet parking attendant — can record a number of access

codes and replay them later to steal your car. Providing enough nonvolatile

memory to remember hundreds or even thousands of old codes might push

you to a more expensive microcontroller, and add a few cents to the cost of

your lock.

If you opt for counters, the problem is synchronization. The key may be

used for more than one lock; it may also be activated repeatedly by jostling

against something in your pocket (I once took an experimental token home

where it was gnawed by my dogs). So there has to be a way to recover after the

counter has been incremented hundreds or possibly even thousands of times.

This can be turned to advantage by allowing the lock to ‘learn’, or synchronise

on, a key under certain conditions; but the details are not always designed

thoughtfully. One common product uses a sixteen bit counter, and allows

access when the deciphered counter value is the last valid code incremented

by no more than sixteen. To cope with cases where the token has been used

more than sixteen times elsewhere (or gnawed by a family pet), the lock will

open on a second press provided that the counter value has been incremented

3.3 Who Goes There? — Simple Authentication 69

between 17 and 32,767 times since a valid code was entered (the counter rolls

over so that 0 is the successor of 65,535). This is ﬁne in many applications, but

a thief who can get six well-chosen access codes — say for values 0, 1, 20,000,

20,001, 40,000 and 40,001 — can break the system completely. So you would

have to think hard about whether your threat model includes a valet able to

get access codes corresponding to chosen counter values, either by patience or

by hardware hacking.

A recent example of design failure comes from TinyOS, an operating system

used in sensor networks based on the IEEE 802.15.4 ad-hoc networking

standard. The TinySec library commonly used for security protocols contains

not one, but three counters. The ﬁrst is lost as the radio chip driver overwrites

it, the second isn’t remembered by the receiver, and although the third is

functional, it’s used for reliability rather than security. So if someone monkeys

with the trafﬁc, the outcome is ‘error’ rather than ‘alarm’, and the network will

resynchronise itself on a bad counter [340].

So designing even a simple token authentication mechanism is not at all

straightforward. There are many attacks that do not involve ‘breaking’ the

encryption. Such attacks are likely to become more common as cryptographic

authentication mechanisms proliferate, many of them designed by program-

mers who thought the problem was easy and never bothered to read a book

like this one. And there are capable agencies trying to ﬁnd ways to defeat

these remote key entry systems; in Thailand, for example, Muslim insurgents

use them to detonate bombs, and the army has responded by deploying

jammers [1000].

Another important example of authentication, and one that’s politically con-

tentious for different reasons, is ‘accessory control’. Many printer companies

embed authentication mechanisms in printers to ensure that genuine toner

cartridges are used. If a competitor’s product is loaded instead, the printer

may quietly downgrade from 1200 dpi to 300 dpi, or simply refuse to work at

all. Mobile phone vendors make a lot of money from replacement batteries,

and now use authentication protocols to spot competitors’ products so they

can be blocked or even drained more quickly. All sorts of other industries are

getting in on the act; there’s talk in the motor trade of cars that authenticate

their major spare parts. I’ll discuss this in more detail in Chapter 22 along

with copyright and rights management generally. Sufﬁce it to say here that

security mechanisms are used more and more to support business models,

by accessory control, rights management, product tying and bundling. It is

wrong to assume blindly that security protocols exist to keep ‘bad’ guys ‘out’.

They are increasingly used to constrain the lawful owner of the equipment in

which they are built; their purpose may be of questionable legality or contrary

to public policy.

70 Chapter 3 ■Protocols

3.3.1 Challenge and Response

Most cars nowadays have remote-controlled door unlocking, though most

also have a fallback metal key to ensure that you can still get into your car

even if the RF environment is noisy. Many also use a more sophisticated two-

pass protocol, called challenge-response, to actually authorise engine start. As

the car key is inserted into the steering lock, the engine controller sends a

challenge consisting of a random n-bit number to the key using short-range

radio. The car key computes a response by encrypting the challenge. So,

writing Efor the engine controller, Tfor the transponder in the car key, K

for the cryptographic key shared between the transponder and the engine

controller, and Nfor the random challenge, the protocol may look something

like:

E−→ T:N

T−→ E:{T,N}K

This is still not bulletproof.

In one system, the random numbers generated by the engine management

unit turned out to be predictable, so it was possible for a thief to interrogate the

key in the car owner’s pocket, as he passed, with the anticipated next challenge.

In fact, many products that incorporate encryption have been broken at some

time or another because their random number generators weren’t random

enough [533, 395]. The ﬁx varies from one application to another. It’s possible

to build hardware random number generators using radioactive decay, but

this isn’t common because of health and safety concerns. There are various

sources of usable randomness in large systems such as PCs, such as the small

variations in the rotation speed of the hard disk caused by air turbulence [358].

PC software products often mix together the randomness from a number of

environmental sources such as network trafﬁc and keystroke timing and from

internal system sources [567]; and the way these sources are combined is often

critical [703]. But in a typical embedded system such as a car lock, the random

challenge is generated by encrypting a counter using a special key which is

kept inside the device and not used for any other purpose.

Locks are not the only application of challenge-response protocols. In HTTP

Digest Authentication, a web server challenges a client or proxy, with whom

it shares a password, by sending it a nonce. The response consists of the

hash of the nonce, the password, and the requested URI [493]. This provides a

mechanism that’s not vulnerable to password snooping. It’s used, for example,

to authenticate clients and servers in SIP, the protocol for Voice-Over-IP

(VOIP) telephony. It is much better than sending a password in the clear,

but suffers from various weaknesses — the most serious being middleperson

attacks, which I’ll discuss shortly.

3.3 Who Goes There? — Simple Authentication 71

A much more visible use of challenge-response is in two-factor authentication.

Many organizations issue their staff with password generators to let them

log on to corporate computer systems [1354]. These may look like calculators

(and some even function as calculators) but their main function is as follows.

When you want to log in to a machine on the network, you call up a logon

screen and are presented with a random challenge of maybe seven digits. You

key this into your password generator, together with a PIN of maybe four

digits. The device encrypts these eleven digits using a secret key shared with

the corporate security server, and displays the ﬁrst seven digits of the result.

You enter these seven digits as your password. This protocol is illustrated in

Figure 3.1. If you had a password generator with the right secret key, and you

entered the PIN right, and you typed in the result correctly, then the corporate

computer system lets you in. But if you do not have a genuine password

generator for which you know the PIN, your chance of logging on is small.

Formally, with Sfor the server, Pfor the password generator, PIN for the

user’s Personal Identiﬁcation Number that bootstraps the password generator,

Ufor the user and Nfor the random nonce:

S−→ U:N

U−→ P:N,PIN

P−→ U:{N,PIN}K

U−→ S:{N,PIN}K

. . . . .

N, PIN K

Figure 3.1: Password generator use

72 Chapter 3 ■Protocols

These devices appeared from the early 1980s and caught on ﬁrst with phone

companies, then in the 1990s with banks for use by staff. There are simpliﬁed

versions that don’t have a keyboard, but just generate a new access code every

minute or so by encrypting a counter: the RSA SecurID is the best known.

One sector after another has been adopting authentication tokens of one kind

or another to replace or supplement passwords; the US Defense Department

announced in 2007 that the introduction of an authentication system based

on the DoD Common Access Card had cut network intrusions by 46% in the

previous year [225].

The technology is now starting to spread to the customer side of things. By

2001, password generators were used by some exclusive private banks, such

as Coutts, to authenticate their online customers. These banks never suffered

any phishing fraud. By 2006, some banks in the Netherlands and Scandinavia

had rolled out the technology to all their millions of customers; then the frauds

started. The phishermen typically use real-time man-in-the-middle attacks

(which I’ll describe in the next section) to take over a session once the user

has authenticated herself to the bank. As of late 2007, some banks in the

UK and elsewhere in Europe have been introducing the Chip Authentication

Program (CAP), which is implemented by giving bank customers a calculator

that uses their bank card to do crypto2. This calculator, when loaded with a

bank card, will ask for the customer’s PIN and, if it’s entered correctly, will

compute a response code based on either a counter (as a one-off authentication

code for a card transaction, or a one-step logon to a banking website) or a

challenge (for a two-step logon). There is also a third mode of operation: if

session takeover becomes a problem, the CAP calculator can also be used to

authenticate transaction data. In this case, it’s planned to have the customer

enter the amount and the last eight digits of the payee account number into

her CAP calculator.

But the result might not be as good in banking as it has been in the armed

forces. First, when your wallet is stolen the thief might be able to read your

PIN digits from the calculator — they will be the dirty and worn keys. If you

just use one bank card, then the thief’s chance of guessing your PIN in 3 tries

has just come down from about 1 in 3000 to about 1 in 10. Second, when you

use your card in a Maﬁa-owned shop (or in a shop whose terminals have been

quietly reprogrammed without the owner’s knowledge), the bad guys have

everything they need to loot your account. Not only that — they can compute

a series of CAP codes to give them access in the future, and use your account

for wicked purposes such as money laundering. Third, someone who takes

your bank card from you at knifepoint can now verify that you’ve told them

2Bank cards in many European countries have an EMV smartcard chip on them, and new UK

bank cards have software to compute authentication codes as well as to operate ATMs and shop

terminals.

3.3 Who Goes There? — Simple Authentication 73

the right PIN. A further problem is that the mechanisms can be used in a

range of protocols; if you have to give a one-off authentication code over the

phone to buy a book with your bank card, and the bookseller can then use

that code to log on to your bank, it’s clearly a bad thing. A deeper problem

is that once lots of banks use one-time passwords, the phishermen will just

rewrite their scripts to do real-time man-in-the-middle attacks. These have

already been used against the early adopter banks in the Netherlands and

Scandinavia. To see how they work, we will now look at a military example.

3.3.2 The MIG-in-the-Middle Attack

The ever-increasing speeds of warplanes in the 1930s and 1940s, together

with the invention of the jet engine, radar and rocketry, made it ever more

difﬁcult for air defence forces to tell their own craft apart from the enemy’s. This

led to a serious risk of ‘fratricide’ — people shooting down their colleagues

by mistake — and drove the development of systems to ‘identify-friend-or-

foe’ (IFF). These were ﬁrst ﬁelded in World War II, and in their early form

enabled an airplane illuminated by radar to broadcast an identifying number

to signal friendly intent. In 1952, this system was adopted to identify civil

aircraft to air trafﬁc controllers and, worried about the loss of security once

it became widely used, the U.S. Air Force started a research programme to

incorporate cryptographic protection in the system. Nowadays, the typical air

defense system sends random challenges with its radar signals, and friendly

aircraft have equipment and keys that enable them to identify themselves

with correct responses. The chapter on electronic warfare has more details on

modern systems.

It’s tricky to design a good IFF system. One of the problems is illustrated

by the following story, which I heard from an ofﬁcer in the South African

Air Force (SAAF). After it was published in the ﬁrst edition of this book,

the story was disputed — as I’ll discuss below. Be that as it may, similar

games have been played with other electronic warfare systems since World

War 2. The ‘Mig-in-the-middle’ story has in any event become part of the

folklore, and it nicely illustrates how attacks can be carried out in real time on

challenge-response authentication protocols.

In the late 1980’s, South African troops were ﬁghting a war in northern

Namibia and southern Angola. The goals were to keep Namibia under white

rule, and impose a client government (UNITA) on Angola. Because the South

African Defence Force consisted largely of conscripts from a small white

population, it was important to limit casualties, so most South African soldiers

remained in Namibia on policing duties while the ﬁghting to the north was

done by UNITA troops. The role of the SAAF was twofold: to provide tactical

support to UNITA by bombing targets in Angola, and to ensure that the

Angolans and their Cuban allies did not return the compliment in Namibia.

74 Chapter 3 ■Protocols

Suddenly, the Cubans broke through the South African air defenses and

carried out a bombing raid on a South African camp in northern Namibia,

killing a number of white conscripts. This proof that their air supremacy had

been lost helped the Pretoria government decide to hand over Namibia to the

insurgents — itself a huge step on the road to majority rule in South Africa

several years later. The raid may also have been the last successful military

operation ever carried out by Soviet bloc forces.

Some years afterwards, a SAAF ofﬁcer told me how the Cubans had pulled

it off. Several MIGs had loitered in southern Angola, just north of the South

African air defense belt, until a ﬂight of SAAF Impala bombers raided a target

in Angola. Then the MIGs turned sharply and ﬂew openly through the SAAF’s

air defenses, which sent IFF challenges. The MIGs relayed them to the Angolan

air defense batteries, which transmitted them at a SAAF bomber; the responses

were relayed back in real time to the MIGs, who retransmitted them and were

allowed through — as in Figure 3.2. According to my informant, this had a

signiﬁcant effect on the general staff in Pretoria. Being not only outfought by

black opponents, but actually outsmarted, was not consistent with the world

view they had held up till then.

After this tale was published in the ﬁrst edition of my book, I was contacted

by a former ofﬁcer in SA Communications Security Agency who disputed the

story’s details. He said that their IFF equipment did not use cryptography

yet at the time of the Angolan war, and was always switched off over enemy

territory. Thus, he said, any electronic trickery must have been of a more

primitive kind. However, others tell me that ‘Mig-in-the-middle’ tricks were

signiﬁcant in Korea, Vietnam and various Middle Eastern conﬂicts.

In any case, the tale illustrates the basic idea behind an attack known

to the cryptographic community as the man-in-the-middle or (more recently)

the middleperson attack. It applies in a straightforward way to the challenge-

response authentication performed by password calculators: the phishing site

invites the mark to log on and simultaneously opens a logon session with his

bank. The bank sends a challenge; the phisherman relays this to the mark,

who uses his device to respond to it; the phisherman relays it to the bank,

and is now authenticated to the bank as the mark. This is why, as I discussed

above, European banks are introducing not just a simple response to a single

challenge, but an authentication code based on input ﬁelds such as the amount,

the payee account number and a transaction sequence number.

However, once the protocol-level vulnerabilities are ﬁxed by including all

the transaction data, the big problem will be usability. If it takes two minutes

and the entry of dozens of digits to make a payment, then a lot of customers

will get digits wrong, give up, and then either call the call center or send paper

checks — undermining the cost savings of online banking. Also, the bad guys

will be able to exploit the fallback mechanisms, perhaps by spooﬁng customers

3.3 Who Goes There? — Simple Authentication 75

ANGOLA

NAMIBIA

SAAF

MIG

SAAF

Figure 3.2: The MIG-in-the middle attack

into calling voice phishing phone numbers that run a middleperson attack

between the customer and the call center.

We will come across the man-in-the-middle attack again and again in

applications ranging from pay-TV to Internet security protocols. It even

applies in online gaming. As the mathematician John Conway once remarked,

it’s easy to get at least a draw against a grandmaster at postal chess: just play

two grandmasters at once, one as white and the other as black, and relay the

moves between them!

In many cases, middleperson attacks are possible but not economic. In the

case of car keys, it should certainly be possible to steal a car by having an

accomplice follow the driver and electronically relay the radio challenge to

you as you work the lock. (One of our students has actually demonstrated

76 Chapter 3 ■Protocols

this for our RFID door locks.) But, for the average car thief, it would be a lot

simpler to just pick the target’s pocket or mug him.

In early 2007, it became clear that there is a practical middleperson attack on

the protocols used by the EMV smartcards issued to bank customers in Europe.

A bad man could build a wicked terminal that masqueraded, for example, as

a parking meter; when you entered your card and PIN to pay a £2.50 parking

fee, the transaction could be relayed to a crook loitering near a self-service

terminal in a hardware store, who would use a card emulator to order goods.

When you get your statement, you might ﬁnd you’ve been debited £2,500 for

a wide-screen TV [915]. The basic problem here is the lack of a trustworthy

user interface on the card; the cardholder doesn’t really know which terminal

his card is doing business with. I’ll discuss such attacks further in the chapter

on Banking and Bookkeeping.

3.3.3 Reﬂection Attacks

Further interesting problems arise with mutual authentication, that is, when

two principals have to identify each other. Suppose, for example, that a sim-

ple challenge-response IFF system designed to prevent anti-aircraft gunners

attacking friendly aircraft had to be deployed in a ﬁghter-bomber too. Now

suppose that the air force simply installed one of their air gunners’ challenge

units in each aircraft and connected it to the ﬁre-control radar. But now an

enemy bomber might reﬂect a challenge back at our ﬁghter, get a correct

response, and then reﬂect that back as its own response:

F−→ B:N

B−→ F:N

F−→ B:{N}K

B−→ F:{N}K

So we will want to integrate the challenge system with the response gener-

ator. It is still not enough just for the two units to be connected and share a list

of outstanding challenges, as an enemy attacked by two of our aircraft might

reﬂect a challenge from one of them to be answered by the other. It might also

not be acceptable to switch manually from ‘attack’ to ‘defense’ during combat.

There are a number of ways of stopping this ‘reﬂection attack’: in many cases,

it is sufﬁcient to include the names of the two parties in the authentication

exchange. In the above example, we might require a friendly bomber to reply

to the challenge:

F−→ B:N

3.3 Who Goes There? — Simple Authentication 77

with a response such as:

B−→ F:{B,N}K

Thus a reﬂected response {F,N}(or even {F,N}from the ﬁghter pilot’s

wingman) could be detected.

This is a much simpliﬁed account of IFF, but it serves to illustrate the

subtelty of the trust assumptions that underlie an authentication protocol. If

you send out a challenge Nand receive, within 20 milliseconds, a response

{N}K, then — since light can travel a bit under 3,730 miles in 20 ms — you

know that there is someone with the key Kwithin 2000 miles. But that’s all you

know. If you can be sure that the response was not computed using your own

equipment, you now know that there is someone else with the key Kwithin two

thousand miles. If you make the further assumption that all copies of the key

Kare securely held in equipment which may be trusted to operate properly,

and you see {B,N}K, you might be justiﬁed in deducing that the aircraft with

callsign Bis within 2000 miles. A clear understanding of trust assumptions

and their consequences is at the heart of security protocol design.

By now you might think that the protocol design aspects of IFF have been

exhaustively discussed. But we’ve omitted one of the most important prob-

lems — and one which the designers of early IFF systems did not anticipate. As

radar returns are weak, the signal from the IFF transmitter on board an aircraft

will often be audible at a much greater range than the return. The Allies learned

this the hard way; in January 1944, decrypts of Enigma messages revealed that

the Germans were plotting British and American bombers at twice the normal

radar range by interrogating their IFF. So many modern systems authenticate

the challenge as well as the response. The NATO mode XII, for example, has

a 32 bit encrypted challenge, and a different valid challenge is generated for

every interrogation signal, of which there are typically 250 per second. Theo-

retically there is no need to switch off over enemy territory, but in practice an

enemy who can record valid challenges can replay them as part of an attack.

Relays are also possible, as with the Mig in the middle.

Many other IFF design problems are less protocol-related, such as the

difﬁculties posed by neutrals, error rates in dense operational environments,

how to deal with equipment failure, how to manage keys, and how to cope

with multinational coalitions such as that put together for Operation Desert

Storm. I’ll return to IFF in Chapter 19. For now, the spurious-challenge problem

serves to reinforce an important point: that the correctness of a security protocol

depends on the assumptions made about the requirements. A protocol that

can protect against one kind of attack (being shot down by your own side) but

which increases the exposure to an even more likely attack (being shot down

by the other side) does more harm than good. In fact, the spurious-challenge

problem became so serious in World War II that some experts advocated

abandoning IFF altogether, rather than taking the risk that one bomber pilot

78 Chapter 3 ■Protocols

in a formation of hundreds would ignore orders and leave his IFF switched on

while over enemy territory.

3.4 Manipulating the Message

We’ve now seen a number of middleperson attacks that reﬂect or spoof the

information used to authenticate a participant’s identity — from ATM cards

that could be reprogrammed to ‘identify’ the wrong customer, to attacks on

IFF. However, there are more complex attacks where the attacker does not just

obtain false identiﬁcation, but manipulates the message content in some way.

An example is when dishonest cabbies insert pulse generators in the cable

that connects their taximeter to a sensor in their taxi’s gearbox. The sensor

sends pulses as the prop shaft turns, which lets the meter work out how far

the taxi has gone. A pirate device, which inserts extra pulses, makes the taxi

appear to have gone further. We’ll discuss such attacks at much greater length

in the chapter on ‘Monitoring Systems’, in section 12.3.

Another example is a key log attack which defeated many pay-TV systems

in Europe in the 1990s and still appears to work in China. The attack is also

known as delayed data transfer, or DDT. First-generation pay-TV equipment

has a decoder, which deciphers the video signal, and a customer smartcard

which generates the deciphering keys. These keys are recomputed every few

hundred milliseconds by using a one-way encryption function applied to

various ‘entitlement control messages’ that appear in the signal. Such systems

can be very elaborate (and we’ll discuss some more complex attacks on them

later) but there is a very simple attack which works against a lot of them. If the

messages that pass between the smartcard and the decoder are the same for

all decoders (which is usually the case) then a subscriber can log all the keys

sent by his card to his decoder and post it online somewhere. People without a

subscription, but who have video-recorded the enciphered program, can then

download the key log and use it to decipher the tape.

Changing pay-TV protocols to prevent DDT attacks can be difﬁcult. The

base of installed equipment is huge, and many of the obvious countermeasures

have an adverse effect on legitimate customers (such as by preventing them

videotaping movies). Pay-TV companies generally ignore this attack, since

connecting a PC up to a satellite TV decoder through a special hardware

adaptor is something only hobbyists do; it is too inconvenient to be a real

threat to their revenue stream. In the rare cases where it becomes a nuisance,

the strategy is usually to identify the troublesome subscribers and send

entitlement control messages that deactivate their cards.

Message-manipulation attacks aren’t limited to ‘consumer’ grade systems.

The Intelsat satellites used for international telephone and data trafﬁc have

robust mechanisms to prevent a command being accepted twice — otherwise

3.5 Changing the Environment 79

an attacker could repeatedly order the same maneuver to be carried out until

the satellite ran out of fuel [1027].

3.5 Changing the Environment

A very common cause of protocol failure is that the environment changes, so

that assumptions which were originally true no longer hold and the security

protocols cannot cope with the new threats.

One nice example comes from the ticketing systems used by the urban

transport authority in London. In the early 1980’s, passengers devised a

number of scams to cut the cost of commuting. For example, a passenger

who commuted a long distance from a suburban station to downtown might

buy two cheaper, short distance season tickets — one between his suburban

station and a nearby one, and the other between his destination and another

downtown station. These would let him get through the barriers, and on the

rare occasions he was challenged by an inspector in between, he would claim

that he’d boarded at a rural station which had a broken ticket machine.

A large investment later, the system had all the features necessary to stop

such scams: all barriers were automatic, tickets could retain state, and the laws

had been changed so that people caught without tickets got ﬁned on the spot.

But suddenly the whole environment changed, as the national transport

system was privatized to create dozens of rail and bus companies. Some of the

new operating companies started cheating each other, and there was nothing

the system could do about it! For example, when a one-day travel pass was

sold, the revenue was distributed between the various bus, train and subway

operators using a formula that depended on where it was sold. Suddenly,

the train companies had a motive to book all their ticket sales through the

outlet that let them keep the largest percentage. As well as bad outsiders

(passengers), we now had bad insiders (rail companies), and the design just

hadn’t allowed for them. Chaos and litigation ensued.

The transport system’s problem was not new; it had been observed in the

Italian ski resort of Val di Fassa in the mid-1970’s. There, one could buy a

monthly pass for all the ski lifts in the valley. An attendant at one of the lifts

was observed with a deck of cards, one of which he swiped through the reader

between each of the guests. It turned out that the revenue was divided up

between the various lift operators according to the number of people who had

passed their turnstiles. So each operator sought to inﬂate its own ﬁgures as

much as it could [1217].

Another nice example comes from the world of cash machine fraud. In 1993

and 1994, Holland suffered an epidemic of ‘phantom withdrawals’; there was

much controversy in the press, with the banks claiming that their systems

were secure while many people wrote in to the papers claiming to have been

80 Chapter 3 ■Protocols

cheated. Eventually the banks were shamed into actively investigating the

claims, and noticed that many of the victims had used their bank cards at a

certain ﬁlling station near Utrecht. This was staked out and one of the staff

was arrested. It turned out that he had tapped the line from the card reader

to the PC that controlled it; his tap recorded the magnetic stripe details from

their cards while he used his eyeballs to capture their PINs [33].

Why had the system been designed so badly? Well, when the standards

for managing magnetic stripe cards and PINs were developed in the early

1980’s by organizations such as IBM and VISA, the engineers had made two

assumptions. The ﬁrst was that the contents of the magnetic strip — the card

number, version number and expiration date — were not secret, while the

PIN was [880]. (The analogy used was that the magnetic strip was your name

and the PIN your password. I will have more to say on the difﬁculties of

naming below.) The second assumption was that bank card equipment would

only be operated in trustworthy environments, such as in a physically robust

automatic teller machine, or by a bank clerk at a teller station. So it was ‘clearly’

only necessary to encrypt the PIN, on its way from the PIN pad to the server;

the magnetic strip data could be sent in clear from the card reader.

Both of these assumptions had changed by 1993. An epidemic of card

forgery, mostly in the Far East in the late 1980’s, drove banks to introduce

authentication codes on the magnetic strips. Also, the commercial success of

the bank card industry led banks in many countries to extend the use of debit

cards from ATMs to terminals in all manner of shops. The combination of these

two environmental changes undermined the original system design: instead of

putting a card whose magnetic strip contained no security data into a trusted

machine, people were putting a card with security data in clear on the strip

into an untrusted machine. These changes had come about so gradually, and

over such a long period, that the industry didn’t see the problem coming.

3.6 Chosen Protocol Attacks

Some ﬁrms are trying to sell the idea of a ‘multifunction smartcard’ — an

authentication device that could be used in a wide range of transactions

to save you having to carry around dozens of different cards and keys.

Governments keen to push ID cards in the wake of 9/11 have tried to get them

used for many other transactions; some want a single card to be used for ID,

banking and even transport ticketing. Singapore went so far as to experiment

with a bank card that doubled as military ID. This introduced some interesting

new risks: if a Navy captain tries to withdraw some cash from an ATM after a

good dinner and forgets his PIN, will he be unable to take his ship to sea until

Monday morning when they open the bank and give him his card back?

3.6 Chosen Protocol Attacks 81

sigK X

Picture 143!

Mafia porn

site

Customer

Buy 10 gold coins

Prove your age Sign ‘X’

by signing ‘X’

BANK

sigK X

Figure 3.3: The Maﬁa-in-the-middle attack

Suppose that the banks in Europe were to introduce the CAP protocol to get

their customers to authenticate themselves to electronic banking websites, but

rather than forcing their customers to ﬁddle about with a calculator device they

just issued all customers with smartcard readers that could be attached to their

PC. This would certainly improve convenience and usability. You might think

it would improve security too; the EMV protocol enables the card to calculate

a message authentication code (MAC) on transaction data such as the amount,

merchant number, date and transaction serial number. Message manipulation

attacks against electronic banking payments would be prevented.

Or would they? The idea behind the ‘Chosen Protocol Attack’ is that given a

target protocol, you design a new protocol that will attack it if the users can be

inveigled into reusing the same token or crypto key. So how might the Maﬁa

design a protocol to attack CAP?

Here’s one approach. It used to be common for people visiting a porn

website to be asked for ‘proof of age,’ which usually involves giving a

credit card number, whether to the site itself or to an age checking service.

If credit and debit cards become usable in PCs, it would be natural for

the porn site to ask the customer to authenticate a random challenge as

proof of age. A porn site can then mount a ‘Maﬁa-in-the-middle’ attack as

shown in Figure 3.3. They wait until an unsuspecting customer visits their site,

then order something resellable (such as gold coins) from a dealer, playing

the role of the coin dealer’s customer. When the coin dealer sends them the

transaction data for authentication, they relay it through their porn site to

the waiting customer. The poor man OKs it, the Maﬁa gets the gold coins, and

when thousands of people suddenly complain about the huge charges to their

cards at the end of the month, the porn site has vanished — along with the

gold [702].

This is a more extreme variant on the Utrecht scam, and in the 1990s

a vulnerability of this kind found its way into international standards: the

standards for digital signature and authentication could be run back-to-back

in this way. It has since been shown that many protocols, though secure in

themselves, can be broken if their users can be inveigled into reusing the same

keys in other applications [702]. This is why, for CAP to be secure, it may

82 Chapter 3 ■Protocols

well have to be implemented in a stand-alone device into which the customer

enters all the transaction parameters directly. Even so, some way has to be

found to make it hard for the phishermen to trick the customer into computing

an authentication code on data that they supply to the victim. The use of the

customer’s bank card in the CAP calculator may at least help to bring home

that a banking transaction is being done.

In general, using crypto keys (or other authentication mechanisms) in

more than one application is dangerous, while letting other people bootstrap

their own application security off yours can be downright foolish. If a bank

lets its smartcards also be used to load credit into prepayment electricity

meters, it would have to worry very hard about whether bad software could

be used in electricity vending stations (or even electricity meters) to steal

money. Even if those risks could be controlled somehow, liability issues can

arise from unplanned or emergent dependencies. A bank that changed its card

speciﬁcation might break the metering system — leaving its customers literally

in the dark and risking a lawsuit from the power company. If the bank heeds

these risks and tests system changes properly with all the dependant systems,

then changes will be much more expensive. Crooks who hack the bank could

black out the neighbourhood. The bank might still want to take this risk,

though, reckoning that power company customers would be locked in more

tightly to the bank, enabling it to charge them more. Security dependencies

can have all sorts of strange effects, and we will return to this subject again

and again later.

3.7 Managing Encryption Keys

The examples of security protocols that we have discussed so far are mostly

about authenticating a principal’s name, or application data such as the

impulses driving a taximeter. There is one further class of authentication

protocols that is very important — the protocols used to manage cryptographic

keys. Until recently, such protocols were largely used in the background to

support other operations; much of the technology was developed to manage

the keys used by cash machines and banks to communicate with each other.

But now, systems such as pay-TV use key management to control access to the

system directly.

Authentication protocols are now also used in distributed computer systems

for general key management purposes, and are therefore becoming ever more

important. Kerberos was the ﬁrst such system to come into widespread use,

and a variant of it is used in Windows. I’ll now lay the foundations for an

understanding of Kerberos.

3.7 Managing Encryption Keys 83

3.7.1 Basic Key Management

The basic idea behind key distribution protocols is that where two princi-

pals want to communicate, they may use a trusted third party to effect an

introduction.

When discussing authentication protocols, it is conventional to give the

principals human names in order to avoid getting lost in too much algebraic

notation. So we will call the two communicating principals ‘Alice’ and ‘Bob’,

and the trusted third party ‘Sam’. But please don’t assume that we are

talking about human principals. Alice and Bob are likely to be programs

while Sam is a server; for example, Alice might be a program in a taximeter,

Bob the program in a gearbox sensor and Sam the computer at the taxi

inspection station.

Anyway, a simple authentication protocol could run as follows.

1. Alice ﬁrst calls Sam and asks for a key for communicating with Bob.

2. Sam responds by sending Alice a pair of certiﬁcates. Each contains a copy

of a key, the ﬁrst encrypted so only Alice can read it, and the second

encrypted so only Bob can read it.

3. Alice then calls Bob and presents the second certiﬁcate as her introduction.

Each of them decrypts the appropriate certiﬁcate under the key they share

with Sam and thereby gets access to the new key. Alice can now use the

key to send encrypted messages to Bob, and to receive messages from him

in return.

Replay attacks are a known problem with authentication protocols, so in

order that both Bob and Alice can check that the certiﬁcates are fresh, Sam may

include a timestamp in each of them. If certiﬁcates never expire, there might

be serious problems dealing with users whose privileges have been revoked.

Using our protocol notation, we could describe this as

A→S:A,B

S→A:{A,B,KAB,T}KAS ,{A,B,KAB,T}KBS

A→B:{A,B,KAB,T}KBS ,{M}KAB

Expanding the notation, Alice calls Sam and says she’d like to talk to

Bob. Sam makes up a session key message consisting of Alice’s name, Bob’s

name, a key for them to use, and a timestamp. He encrypts all this under

the key he shares with Alice, and he encrypts another copy of it under the

key he shares with Bob. He gives both ciphertexts to Alice. Alice retrieves

the key from the ciphertext that was encrypted to her, and passes on to

Bob the ciphertext encrypted for him. She now sends him whatever message

she wanted to send, encrypted using this key.

84 Chapter 3 ■Protocols

3.7.2 The Needham-Schroeder Protocol

Many things can go wrong, and here is a famous historical example. Many

existing key distribution protocols are derived from the Needham-Schroeder

protocol, which appeared in 1978 [960]. It is somewhat similar to the above,

but uses nonces rather than timestamps. It runs as follows:

Message 1 A→S:A,B,NA

Message 2 S→A:{NA,B,KAB,{KAB,A}KBS }KAS

Message 3 A→B:{KAB,A}KBS

Message 4 B→A:{NB}KAB

Message 5 A→B:{NB−1}KAB

Here Alice takes the initiative, and tells Sam: ‘I’m Alice, I want to talk to Bob,

and my random nonce is NA.’ Sam provides her with a session key, encrypted

using the key she shares with him. This ciphertext also contains her nonce so

she can conﬁrm it’s not a replay. He also gives her a certiﬁcate to convey this

key to Bob. She passes it to Bob, who then does a challenge-response to check

that she is present and alert.

There is a subtle problem with this protocol — Bob has to assume that the

key KAB he receives from Sam (via Alice) is fresh. This is not necessarily so:

Alice could have waited a year between steps 2 and 3. In many applications

this may not be important; it might even help Alice to cache keys against

possible server failures. But if an opponent — say Charlie — ever got hold of

Alice’s key, he could use it to set up session keys with many other principals.

Suppose, for example, that Alice had also asked for and received a key to

communicate with Dave, and after Charlie stole her key he sent messages to

Sam pretending to be Alice and got keys for Freddie and Ginger. He might

also have observed message 2 in her protocol exchanges with Dave. So now

Charlie could impersonate Alice to Dave, Freddie and Ginger. So when Alice

ﬁnds out that her key has been stolen, perhaps by comparing message logs

with Dave, she’d have to get Sam to contact everyone for whom she’d ever

been issued a key, and tell them that her old key was no longer valid. She could

not do this herself as she doesn’t know anything about Freddie and Ginger. In

other words, revocation is a problem: Sam may have to keep complete logs of

everything he’s ever done, and these logs would grow in size forever unless

the principals’ names expired at some ﬁxed time in the future.

Almost 30 years later, this example still generates controversy in the security

protocols community. The simplistic view is that Needham and Schroeder just

got it wrong; the view argued by Susan Pancho and Dieter Gollmann (for

which I have much sympathy) is that this is one more example of a protocol

failure brought on by shifting assumptions [538, 1002]. 1978 was a kinder,

gentler world; computer security then concerned itself with keeping ‘bad

guys’ out, while nowadays we expect the ‘enemy’ to be the users of the

3.7 Managing Encryption Keys 85

system. The Needham-Schroeder paper explicitly assumes that all principals

behave themselves, and that all attacks come from outsiders [960]. With these

assumptions, the protocol remains sound.

3.7.3 Kerberos

An important practical derivative of the Needham-Schroeder protocol may be

found in Kerberos, a distributed access control system that originated at MIT

and is now one of the standard authentication tools in Windows [1224]. Instead

of a single trusted third party, Kerberos has two kinds: an authentication server

to which users log on, and a ticket granting server which gives them tickets

allowing access to various resources such as ﬁles. This enables more scalable

access management. In a university, for example, one might manage students

through their halls of residence but manage ﬁle servers by departments; in

a company, the personnel people might register users to the payroll system

while departmental administrators manage resources such as servers and

printers.

First, Alice logs on to the authentication server using a password. The client

software in her PC fetches a ticket from this server that is encrypted under her

password and that contains a session key KAS. Assuming she gets the password

right, she now controls KAS and to get access to a resource Bcontrolled by the

ticket granting server S, the following protocol takes place. Its outcome is a

key KAB with timestamp TSand lifetime L, which will be used to authenticate

Alice’s subsequent trafﬁc with that resource:

A→S:A,B

S→A:{TS,L,KAB,B,{TS,L,KAB,A}KBS }KAS

A→B:{TS,L,KAB,A}KBS ,{A,TA}KAB

B→A:{TA+1}KAB

Translating this into English: Alice asks the ticket granting server for access

to B. If this is permissible, the ticket {TS,L,KAB,A}KBS is created containing a

suitable key KAB and given to Alice to use. She also gets a copy of the key

in a form readable by her, namely encrypted under KAS. She now veriﬁes the

ticket by sending a timestamp TAto the resource, which conﬁrms it’s alive by

sending back the timestamp incremented by one (this shows it was able to

decrypt the ticket correctly and extract the key KAB).

The vulnerability of Needham-Schroeder has been ﬁxed by introducing

timestamps rather than random nonces. But, as in most of life, we get little

in security for free. There is now a new vulnerability, namely that the clocks

on our various clients and servers might get out of synch; they might even be

desynchronized deliberately as part of a more complex attack.

86 Chapter 3 ■Protocols

3.7.4 Practical Key Management

So we can use a protocol like Kerberos to set up and manage working keys

between users given that each user shares one or more long-term keys with

a server that acts as a key distribution centre. I’ll describe a number of

other similar protocols later; for example, in the chapter on ‘Banking and

Bookkeeping’ I’ll discuss how a bank can set up a long-term key with each of

its ATMs and with each of the interbank networks with which it’s associated.

The bank then uses protocols not too unlike Kerberos to establish a ‘key of

the day’ with each ATM and with each network switch; so when you turn up

at the ATM belonging to a foreign bank and ask for money from your own

bank via the Cirrus network, the ATM will encrypt the transaction using the

working key it shares with the bank that owns it, and the bank will then pass

on the transaction to Cirrus encrypted with the key of the day for that network.

So far so good. But a moment’s thought will reveal that the bank has to

maintain several keys for each of the several hundred ATMs that it owns — a

long-term master key, plus perhaps an encryption key and an authentication

key; several keys for each of the several dozen bank networks of which it’s a

member; passwords and other security information for each of several million

electronic banking customers, and perhaps keys for them as well if they’re

given client software that uses cryptography. Oh, and there may be encrypted

passwords for each of several thousand employees, which might also take the

form of Kerberos keys encrypted under user passwords. That’s a lot of key

material. How is it to be managed?

Key management is a complex and difﬁcult business and is often got

wrong because it’s left as an afterthought. A good engineer will sit down

and think about how many keys are needed, how they’re to be generated,

how long they need to remain in service and how they’ll eventually be

destroyed. There is a much longer list of concerns — many of them articulated

in the Federal Information Processing Standard for key management [948]. In

addition, things go wrong as applications evolve; it’s important to provide extra

keys to support next year’s functionality, so that you don’t compromise your

existing ones by reusing them in protocols that turn out to be incompatible.

It’s also important to support recovery from security failure. Yet there are no

standard ways of doing either.

As for practical strategies, there are a number — none of them straightfor-

ward. Public-key crypto, which I’ll discuss in Chapter 5, can slightly simplify

the key management task. Long-term keys can be split into a private part and a

public part; you don’t have to keep the public part secret (as its name implies)

but you do have to guarantee its integrity. In banking the usual answer is

to use dedicated cryptographic processors called security modules, which I’ll

describe in detail in the chapter on ‘Tamper Resistance’. These do all the cryp-

tography and contain internal keys with which application keys are protected.

3.8 Getting Formal 87

Thus you get your security module to generate master keys for each of your

ATMs; you store their encrypted values in your ATM master ﬁle. Whenever

a transaction comes in from that ATM, you retrieve the encrypted key from

the ﬁle and pass it to the security module along with the encrypted data. The

module then does what’s necessary: it decrypts the PIN and veriﬁes it, perhaps

against an encrypted value kept locally. Unfortunately, the protocols used to

set all this up are also liable to failure. Many attacks have been found that

exploit the application programming interface, or API, of the security module,

where these protocols are exposed. I will describe these attacks in detail in

the chapter on API Security. For now, it’s enough to note that getting security

protocols right is hard. You should not design them at home, any more than

you design your own explosives.

3.8 Getting Formal

Subtle difﬁculties of the kind we have seen with the above protocols, and the

many ways in which protection properties depend on quite subtle starting

assumptions that protocol designers may get wrong (or that may be misunder-

stood later), have led researchers to apply formal methods to key distribution

protocols. The goal of this exercise was originally to decide whether a protocol

was right or wrong: it should either be proved correct, or an attack should be

exhibited. More recently this has expanded to clarifying the assumptions that

underlie a given protocol.

There are a number of different approaches to verifying the correctness

of protocols. The best known is the logic of belief,orBAN logic, named after

its inventors Burrows, Abadi and Needham [249]. It reasons about what a

principal might reasonably believe having seen of certain messages, time-

stamps and so on. A second is the random oracle model, which I touch on in the

chapter on cryptology and which is favored by people working on the theory

of cryptography; this appears less expressive than logics of belief, but can tie

protocol properties to the properties of the underlying encryption algorithms.

Finally, a number of researchers have applied mainstream formal methods

such as CSP and veriﬁcation tools such as Isabelle.

Some history exists of ﬂaws being found in protocols that had been

proved correct using formal methods; the following subsection offers a

typical example.

3.8.1 A Typical Smartcard Banking Protocol

The COPAC system is an electronic purse used by VISA in countries with poor

telecommunications [48]. It was the ﬁrst live ﬁnancial system whose underly-

ing protocol suite was designed and veriﬁed using such formal techniques, and

88 Chapter 3 ■Protocols

in particular a variant of the BAN logic. A similar protocol is now used in the

‘Geldkarte,’ an electronic purse issued by banks in Germany, and adopted also

by French banks as ‘Moneo’. There’s also a system in Belgium called ‘Proton’.

The European applications focus on low-value transactions with devices such

as parking meters and vending machines for which it may not be economical

to provide a network connection.

Transactions take place from a customer smartcard to a merchant smartcard

(which in the case of a vending machine is kept in the machine and changed

when it’s replenished). The customer gives the merchant an electronic check

with two authentication codes on it; one that can be checked by the network,

and one that can be checked by the customer’s bank. A simpliﬁed version of

the protocol is as follows.

C−→ R:{C,NC}K

R−→ C:{R,NR,C,NC}K

C−→ R:{C,NC,R,NR,X}K

In English: the customer and the retailer share a key K. Using this key, the

customer encrypts a message containing its account number Cand a customer

transaction serial number NC. The retailer conﬁrms its own account number

Rand his own transaction serial number NR, as well as the information it’s

just received from the customer. The customer now sends the electronic check

X, along with all the data exchanged so far in the protocol. One can think of

the electronic check as being stapled to a payment advice with the customer’s

and retailer’s account numbers and their respective reference numbers. (The

reason for repeating all previous data in each message is to prevent message

manipulation attacks using cut-and-paste.)

3.8.2 The BAN Logic

The BAN logic provides a formal method for reasoning about the beliefs

of principals in cryptographic protocols. Its underlying idea is that we will

believe that a message is authentic if it is encrypted with a relevant key and it

is also fresh (that is, generated during the current run of the protocol). Further

assumptions include that principals will only assert statements they believe

in, and that some principals are authorities for certain kinds of statement. This

is formalized using a notation which includes:

A|≡ X A believes X, or, more accurately, that Ais entitled to believe X;

A|∼ X A once said X (without implying that this utterance was recent or not);

A|⇒ X A has jurisdiction over X, in other words Ais the authority on Xand is

to be trusted on it;

3.8 Getting Formal 89

AX A sees X, that is, someone sent a message to Acontaining Xin such a

way that he can read and repeat it;

X X is fresh, that is, contains a current timestamp or some information

showing that it was uttered by the relevant principal during the current

run of the protocol;

{X}KX encrypted under the key K, as in the rest of this chapter;

A↔KB A and B share the key K, in other words it is an appropriate key for

them to use to communicate.

There are further symbols dealing, for example, with public key operations

and with passwords, that need not concern us here.

These symbols are manipulated using a set of postulates which include:

the message meaning rule states that if Asees a message encrypted under K,

and Kis a good key for communicating with B, then he will believe that the

message was once said by B. (We assume that each principal can recognize

and ignore his or her own messages.) Formally, A|≡ A↔KB,A{X}K

A|≡ B|∼ X

the nonce-veriﬁcation rule states that if a principal once said a message,

and the message is fresh, then that principal still believes it. Formally,

A|≡ X,A|≡ B|∼ X

A|≡ B|≡ X

the jurisdiction rule states that if a principal believes something, and is an

authority on the matter, then he or she should be believed. Formally, we

write that A|≡ B|⇒ X,A|≡ B|≡ X

A|≡ X

In this notation, the statements on the top are the conditions, and the one on

the bottom is the result. There are a number of further rules to cover the more

mechanical aspects of manipulation; for example, if A sees a statement then

he sees its components provided he knows the necessary keys, and if part of a

formula is known to be fresh, then the whole formula must be.

3.8.3 Verifying the Payment Protocol

Assuming that the key Kis only available to principals who can be trusted to

execute the protocol faithfully, formal veriﬁcation is now straightforward. The

trick is to start from the desired result and work backwards. In this case, we

wish to prove that the retailer should trust the check, i.e., R|≡ X(the syntax

of checks and cryptographic keys is similar for our purposes here; a check is

good if and only if it is genuine and the date on it is sufﬁciently recent).

90 Chapter 3 ■Protocols

Now R|≡ Xwill follow under the jurisdiction rule from R|≡ C|⇒ X(R

believes Chas jurisdiction over X)andR|≡ C|≡ X(Rbelieves Cbelieves X).

The former condition follows from the hardware constraint, that no-one

except Ccould have uttered a text of the form {C,...}K.

The latter, that R|≡ C|≡ X, must be deduced using the nonce veriﬁcation

rule from X(X is fresh) and R|≡ C|∼ X(R believes C uttered X).

Xfollows from its occurrence in {C,NC,R,NR,X}Kwhich contains the

sequence number NR, while R|≡ C|∼ Xfollows from the hardware constraint.

The above summary of the proof is, of necessity, telegraphic. If you want to

understand logics of authentication in detail, you should consult the original

papers [48] and see the recommendations for further reading at the end of this

chapter.

3.8.4 Limitations of Formal Veriﬁcation

Formal methods can be an excellent way of ﬁnding bugs in security protocol

designs as they force the designer to make everything explicit and thus

confront difﬁcult design choices that might otherwise be fudged. However,

they have their limitations, too.

One problem is in the external assumptions we make. For example, we

assumed that the key wasn’t available to anyone who might use it in an

unauthorized manner. In practice, this is not always true. Although our purse

protocol is executed in tamper-resistant smartcards, their software can have

bugs, and in any case the tamper-resistance they offer is never complete. (I’ll

discuss this in the chapter on Tamper Resistance.) So the system has various

fallback mechanisms to detect and react to card forgery, such as shadow

accounts which track the amount of money that should be on each card and

which are updated as transactions are cleared. It also has lists of hot cards that

are distributed to terminals; these are needed anyway for stolen cards, and

can be used for forged cards too.

Second, there are often problems with the idealisation of the protocol. An

interesting ﬂaw was found in an early version of this system. The key Kactually

consisted of two keys — the encryption was done ﬁrst with a ‘transaction key’

which was diversiﬁed (that is, each card had its own variant) and then again

with a ‘bank key’, which was not diversiﬁed. The former was done by the

network operator, and the latter by the bank which issued the card. The

reasons for this included dual control, and to ensure that even if an attacker

managed to drill the keys out of a single card, he would only be able to forge

that card, not make forgeries which would pass as other cards (and thus defeat

the hot card mechanism). But since the bank key was not diversiﬁed, it must

be assumed to be known to any attacker who has broken a card. This means

that he can undo the outer wrapping of encryption, and in some circumstances

message replay was possible. (The bank key was diversiﬁed in a later version

before any villains discovered and exploited the ﬂaw.)

3.9 Summary 91

In this case there was no failure of the formal method, as no attempt was ever

made to verify the diversiﬁcation mechanism. But it does illustrate a common

problem in security engineering — that vulnerabilities arise at the boundary

between two protection technologies. In this case, there were three technolo-

gies: the hardware tamper resistance, the authentication protocol and the

shadow account / hot card list mechanisms. Different protection technologies

are often the domain of different experts who don’t completely understand the

assumptions made by the others. (In fact, that’s one reason security engineers

need a book such as this one: to help subject specialists understand each others’

tools and communicate with each other more effectively.)

For these reasons, people have explored alternative ways of assuring the

design of authentication protocols, including the idea of protocol robustness. Just

as structured programming techniques aim to ensure that software is designed

methodically and nothing of importance is left out, so robust protocol design is

largely about explicitness. Robustness principles include that the interpretation

of a protocol should depend only on its content, not its context; so everything

of importance (such as principals’ names) should be stated explicitly in the

messages. There are other issues concerning the freshness provided by serial

numbers, timestamps and random challenges, and on the way encryption

is used. If the protocol uses public key cryptography or digital signature

mechanisms, there are further more technical robustness issues.

3.9 Summary

Passwords are just one (simple) example of a more general concept, the

security protocol. Protocols specify the series of steps that principals use

to establish trust relationships in a system, such as authenticating a claim

to identity, demonstrating ownership of a credential, or granting a claim

on a resource. Cryptographic authentication protocols, whether one-pass

(e.g., using random nonces) or two-pass (challenge-response) are used for

a wide range of such purposes, from basic entity authentication to provide

infrastructure for distributed systems that allows trust to be taken from where it

exists to where it is needed. Security protocols are ﬁelded in all sorts of systems

from remote car door locks through military IFF systems to authentication in

distributed computer systems.

It is difﬁcult to design effective security protocols. They suffer from a

number of potential problems, including middleperson attacks, modiﬁcation

attacks, reﬂection attacks, and replay attacks. These threats can interact with

implementation vulnerabilities such as poor random number generators. Using

mathematical techniques to verify the correctness of protocols can help, but it

won’t catch all the bugs. Some of the most pernicious failures are caused by

creeping changes in the environment for which a protocol was designed, so

that the protection it gives is no longer adequate.

92 Chapter 3 ■Protocols

Research Problems

At several times during the past 20 years, some people have thought that

protocols had been ‘done’ and that we should turn to new research topics.

They have been repeatedly proved wrong by the emergence of new protocol

applications with a new crop of errors and attacks to be explored. Formal

methods blossomed in the early 1990s, then key management protocols; during

the mid-1990’s the ﬂood of proposals for electronic commerce mechanisms

kept us busy; and in the later 1990’s a whole series of mechanisms proposed

for protecting copyright on the Internet provided us with targets. Since

2000, one strand of protocol research has acquired an economic ﬂavour as

security mechanisms are used more and more to support business models; the

designer’s ‘enemy’ is often a commercial competitor, or even the customer.

Another has applied protocol analysis tools to look at the security of application

programming interfaces (APIs), a topic to which I’ll return later.

Will people continue to develop faulty protocols which other people attack,

or will we manage to develop a methodology for designing them right ﬁrst

time? What are the exact uses and limitations of formal methods, and other

mathematical approaches such as the random oracle model?

At the system level, how do we manage the tension between the principle

that robust protocols are generally those in which everything is completely

speciﬁed and checked (principals’ names, roles, security policy statement,

protocol version, time, date, sequence number, security context, maker of

grandmother’s kitchen sink) and the system engineering principle that a good

speciﬁcation should not overconstrain the implementer?

Further Reading

Research papers on security protocols are scattered fairly widely throughout

the literature. The main introductory papers to read are probably the original

Needham-Schroeder paper [960]; the Burrows-Abadi-Needham authentica-

tion logic [249]; papers by Abadi and Needham, and Anderson and Needham,

on protocol robustness [2, 73]; and there is a survey paper by Anderson and

Needham [74]. In [707] there is an analysis of a defective security protocol, car-

ried out using three different formal methods. Beyond that, the proceedings of

the security protocols workshops [290, 291] provide leads to current research,

and there are many papers scattered around a wide range of conferences.

CHAPTER

Access Control

Going all the way back to early time-sharing systems we systems

people regarded the users, and any code they wrote, as the mortal

enemies of us and each other. We were like the police force

in a violent slum.

— Roger Needham

Microsoft could have incorporated effective security measures as

standard, but good sense prevailed. Security systems have a nasty

habit of backﬁring and there is no doubt they would cause

enormous problems.

— Rick Maybury

4.1 Introduction

Access control is the traditional center of gravity of computer security. It is

where security engineering meets computer science. Its function is to control

which principals (persons, processes, machines, ...) have access to which

resources in the system — which ﬁles they can read, which programs they can

execute, how they share data with other principals, and so on.

Access control works at a number of levels (Figure 4.1).

Application

Middleware

Operating system

Hardware

Figure 4.1: Access controls at different levels in a system

94 Chapter 4 ■Access Control

1. The access control mechanisms the user sees at the application level may

express a very rich and complex security policy. A modern online busi-

ness could assign staff to one of dozens of different roles, each of which

could initiate some subset of several hundred possible transactions in

the system. Some of these (such as refunds) might require dual control or

approval from a supervisor. And that’s nothing compared with the com-

plexity of the access controls on a modern social networking site, which

will have a thicket of rules and options about who can see, copy, and

search what data from whom.

2. The applications may be written on top of middleware, such as a

database management system or bookkeeping package, which enforces

a number of protection properties. For example, bookkeeping soft-

ware may ensure that a transaction which debits one ledger for a

certain amount must credit another ledger for the same amount, while

database software typically has access controls specifying which dictio-

naries a given user can select, and which procedures they can run.

3. The middleware will use facilities provided by the underlying operating

system. As this constructs resources such as ﬁles and communications

ports from lower level components, it acquires the responsibility for pro-

viding ways to control access to them.

4. Finally, the operating system access controls will usually rely on hard-

ware features provided by the processor or by associated memory

management hardware. These control which memory addresses a given

process can access.

As we work up from the hardware through the operating system and

middleware to the application layer, the controls become progressively more

complex and less reliable. Most actual computer frauds involve staff acciden-

tally discovering features of the application code that they can exploit in an

opportunistic way, or just abusing features of the application that they were

trusted not to. However, loopholes in access-control mechanisms — such as in

database software used in a large number of web servers — can expose many

systems simultaneously and compel large numbers of companies to patch or

rewrite their products. So in this chapter, we will focus on the fundamentals:

access control at the hardware, operating system and database levels. You

have to understand the basic principles to design serviceable application-level

controls too (I give many examples in Part II of how to combine access controls

with the needs of speciﬁc applications).

As with the other building blocks discussed so far, access control makes

sense only in the context of a protection goal, typically expressed as a security

policy. PCs carry an unfortunate legacy, in that the old single-user operating

4.1 Introduction 95

systems such as DOS and Win95/98 let any process modify any data; as a

result many applications won’t run unless they are run with administrator

privileges and thus given access to the whole machine. Insisting that your

software run as administrator is also more convenient for the programmer.

But people do have implicit protection goals; you don’t expect a shrink-

wrap program to trash your hard disk. So an explicit security policy is a

good idea.

Now one of the biggest challenges in computer security is preventing one

program from interfering with another. You don’t want a virus to be able to

steal the passwords from your browser, or to patch a banking application so

as to steal your money. In addition, many everyday reliability problems stem

from applications interacting with each other or with the system conﬁguration.

However, it’s difﬁcult to separate applications when the customer wants to

share data. It would make phishing much harder if people were simply unable

to paste URLs from emails into a browser, but that would make everyday

life much harder too. The single-user history of personal computing has got

people used to all sorts of ways of working that are not really consistent

with separating applications and their data in useful ways. Indeed, one senior

manager at Microsoft took the view in 2000 that there was really nothing for

operating-system access controls to do, as client PCs were single-user and

server PCs were single-application.

The pendulum is now swinging back. Hosting centres make increasing

use of virtualization; having machines exclusively for your own use costs

several times what you pay for the same resource on shared machines. The

Trusted Computing initiative, and Microsoft Vista, place more emphasis on

separating applications from each other; even if you don’t care about security,

preventing your programs from overwriting each others’ conﬁguration ﬁles

should make your PC much more reliable. More secure operating systems

have led to ever more technical attacks on software other than the operating

system; you don’t want your brokerage account hacked via a computer game

you downloaded that later turns out to be insecure. And employers would like

ways of ensuring that employees’ laptops don’t pick up malware at home, so

it makes sense to have one partition (or virtual machine) for work and another

for play. Undoing the damage done by many years of information-sharing

promiscuity will be hard, but in the medium term we might reasonably hope

for a framework that enables interactions between applications to be restricted

to controllable interfaces.

Many access control systems build on the mechanisms provided by the

operating system. I will start off by discussing operating-system protection

mechanisms that support the isolation of multiple processes. These came ﬁrst

historically — being invented along with the ﬁrst time-sharing systems in the

96 Chapter 4 ■Access Control

1960s — and they remain the foundation on which many higher-layer mecha-

nisms are built. I will then go on to discuss database systems, which provide

broadly similar access control mechanisms that may or may not be tied to the

operating-systems mechanisms. Finally I’ll discuss three advanced protection

techniques — sandboxing, virtualization and ‘Trusted Computing’. Sandboxing

is an application-level control, run for example in a browser to restrict what

mobile code can do; virtualization runs underneath the operating system,

creating two or more independent virtual machines between which infor-

mation ﬂows can be controlled or prevented; and Trusted Computing is a

project to create two virtual machines side-by-side, one being the ‘old, inse-

cure’ version of an operating system and the second being a more restricted

environment in which security-critical operations such as cryptography can

be carried out.

The latest Microsoft system, Vista, is trying to move away from running

all code with administrator privilege, and these three modern techniques are

each, independently, trying to achieve the same thing — to get us back where

we’d be if all applications had to run with user privileges rather than as the

administrator. That is more or less where computing was in the 1970s, when

people ran their code as unprivileged processes on time-shared minicomputers

and mainframes. Only time will tell whether we can recapture the lost Eden

of order and control alluded to in the quote from Roger Needham at the

start of this chapter, and to escape the messy reality of today to which Rick

Maybury’s quote refers; but certainly the attempt is worth making.

4.2 Operating System Access Controls

The access controls provided with an operating system typically authenticate

principals using a mechanism such as passwords or Kerberos, then mediate

their access to ﬁles, communications ports and other system resources.

Their effect can often be modelled by a matrix of access permissions, with

columns for ﬁles and rows for users. We’ll write rfor permission to read, w

for permission to write, xfor permission to execute a program, and -for no

access at all, as shown in Figure 4.2.

Operating Accounts Accounting Audit

System Program Data Trail

Sam rwx rwx rw r

Alice x x rw –

Bob rx r r r

Figure 4.2: Naive access control matrix

4.2 Operating System Access Controls 97

In this simpliﬁed example, Sam is the system administrator and has universal

access (except to the audit trail, which even he should only be able to read).

Alice, the manager, needs to execute the operating system and application,

but only through the approved interfaces — she mustn’t have the ability to

tamper with them. She also needs to read and write the data. Bob, the auditor,

can read everything.

This is often enough, but in the speciﬁc case of a bookkeeping system

it’s not quite what we need. We want to ensure that transactions are well-

formed — that each debit is matched by a credit somewhere else — so we

would not want Alice to have uninhibited write access to the account ﬁle.

We would also rather that Sam didn’t have this access. So we would prefer

that write access to the accounting data ﬁle be possible only via the accounting

program. The access permissions might now look like in Figure 4.3:

User Operating Accounts Accounting Audit

System Program Data Trail

Sam rwx rwx r r

Alice rx x – –

Accounts program rx rrw w

Bob rx r r r

Figure 4.3: Access control matrix for bookkeeping

Another way of expressing a policy of this type would be with access triples

of (user, program, ﬁle). In the general case, our concern isn’t with a program

so much as a protection domain which is a set of processes or threads which

share access to the same resources (though at any given time they might have

different ﬁles open or different scheduling priorities).

Access control matrices (whether in two or three dimensions) can be used to

implement protection mechanisms as well as just model them. But they do not

scale well. For instance, a bank with 50,000 staff and 300 applications would

have an access control matrix of 15,000,000 entries. This is inconveniently

large. It might not only impose a performance problem but also be vulnerable

to administrators’ mistakes. We will usually need a more compact way of

storing and managing this information. The two main ways of doing this are

to compress the users and to compress the rights. As for the ﬁrst of these, the

simplest is to use groups or roles to manage the privileges of large sets of users

simultaneously, while in the second we may store the access control matrix

either by columns (access control lists) or rows (capabilities, sometimes known

as ‘tickets’) [1102, 1344]. (There are more complex approaches involving policy

engines, but let’s learn to walk before we try to run.)

98 Chapter 4 ■Access Control

4.2.1 Groups and Roles

When we look at large organisations, we usually ﬁnd that most staff ﬁt into

one or other of a small number of categories. A bank might have 40 or 50:

teller, chief teller, branch accountant, branch manager, and so on. Only a few

dozen people (security manager, chief foreign exchange dealer, ...) will need

to have their access rights deﬁned individually.

So we want a small number of pre-deﬁned groups, or functional roles,

to which staff can be assigned. Some people use the words group and role

interchangeably, and with many systems they are; but the more careful

deﬁnition is that a group is a list of principals, while a role is a ﬁxed set of

access permissions that one or more principals may assume for a period of time

using some deﬁned procedure. The classic example of a role is the ofﬁcer of the

watch on a ship. There is exactly one watchkeeper at any one time, and there

is a formal procedure whereby one ofﬁcer relieves another when the watch

changes. In fact, in most government and business applications, it’s the role

that matters rather than the individual.

Groups and roles can be combined. The ofﬁcers of the watch of all ships currently

at sea is a group of roles. In banking, the manager of the Cambridge branch

might have his or her privileges expressed by membership of the group

manager and assumption of the role acting manager of Cambridge branch. The

group manager might express a rank in the organisation (and perhaps even a

salary band) while the role acting manager might include an assistant accountant

standing in while the manager, deputy manager, and branch accountant are

all off sick.

Whether we need to be careful about this distinction is a matter for the

application. In a warship, we want even an ordinary seaman to be allowed to

stand watch if everyone more senior has been killed. In a bank, we might have

a policy that ‘transfers over $10m must be approved by two staff, one with

rank at least manager and one with rank at least assistant accountant’. If the

branch manager is sick, then the assistant accountant acting as manager might

have to get the regional head ofﬁce to provide the second signature on a large

transfer.

Operating-system level support is available for groups and roles, but its

appearance has been fairly recent and its uptake is still slow. Developers

used to implement this kind of functionality in application code, or as custom

middleware (in the 1980s I worked on two bank projects where group support

was hand-coded as extensions to the mainframe operating system). Windows

2000 introduced extensive support for groups, while academic researchers

have done quite a lot of work since the mid-90s on role-based access control

(RBAC), which I’ll discuss further in Part II, and which is starting to be rolled

out in some large applications.

4.2 Operating System Access Controls 99

4.2.2 Access Control Lists

Another way of simplifying the management of access rights is to store the

access control matrix a column at a time, along with the resource to which

the column refers. This is called an access control list or ACL (pronounced

‘ackle’). In the ﬁrst of our above examples, the ACL for ﬁle 3 (the account ﬁle)

might look as shown here in Figure 4.4.

ACLs have a number of advantages and disadvantages as a means of

managing security state. These can be divided into general properties of ACLs,

and speciﬁc properties of particular implementations.

ACLs are a natural choice in environments where users manage their

own ﬁle security, and became widespread in the Unix systems common in

universities and science labs from the 1970s. They are the basic access control

mechanism in Unix-based systems such as GNU/Linux and Apple’s OS/X;

the access controls in Windows are also based on ACLs, but have become

more complex over time. Where access control policy is set centrally, ACLs

are suited to environments where protection is data-oriented; they are less

suited where the user population is large and constantly changing, or where

users want to be able to delegate their authority to run a particular program

to another user for some set period of time. ACLs are simple to implement,

but are not efﬁcient as a means of doing security checking at runtime, as the

typical operating system knows which user is running a particular program,

rather than what ﬁles it has been authorized to access since it was invoked.

The operating system must either check the ACL at each ﬁle access, or keep

track of the active access rights in some other way.

Finally, distributing the access rules into ACLs means that it can be tedious

to ﬁnd all the ﬁles to which a user has access. Revoking the access of an

employee who has just been ﬁred will usually have to be done by cancelling

their password or other authentication mechanism. It may also be tedious to

run system-wide checks; for example, verifying that no ﬁles have been left

world-writable could involve checking ACLs on millions of user ﬁles.

Let’s look at two important examples of ACLs — their implementation in

Unix and Windows.

User Accounting

Data

Sam rw

Alice rw

Bob r

Figure 4.4: Access control list (ACL)

100 Chapter 4 ■Access Control

4.2.3 Unix Operating System Security

In Unix (including its popular variant Linux), ﬁles are not allowed to have

arbitrary access control lists, but simply rwx attributes for the resource owner,

the group, and the world. These attributes allow the ﬁle to be read, written

and executed. The access control list as normally displayed has a ﬂag to show

whether the ﬁle is a directory, then ﬂags r, w and x for world, group and owner

respectively; it then has the owner’s name and the group name. A directory

with all ﬂags set would have the ACL:

drwxrwxrwx Alice Accounts

In our ﬁrst example in Figure 4.3, the ACL of ﬁle 3 would be:

-rw-r- ---- Alice Accounts

This records that the ﬁle is not a directory; the ﬁle owner can read and write

it; group members can read it but not write it; non-group members have no

access at all; the ﬁle owner is Alice; and the group is Accounts.

In Unix, the program that gets control when the machine is booted (the

operating system kernel) runs as the supervisor, and has unrestricted access

to the whole machine. All other programs run as users and have their access

mediated by the supervisor. Access decisions are made on the basis of the

userid associated with the program. However if this is zero (root), then

the access control decision is ‘yes’. So root can do what it likes — access any

ﬁle, become any user, or whatever. What’s more, there are certain things that

only root can do, such as starting certain communication processes. The root

userid is typically made available to the system administrator.

This means that (with most ﬂavours of Unix) the system administrator can

do anything, so we have difﬁculty implementing an audit trail as a ﬁle that

he cannot modify. This not only means that, in our example, Sam could tinker

with the accounts, and have difﬁculty defending himself if he were falsely

accused of tinkering, but that a hacker who managed to become the system

administrator could remove all evidence of his intrusion.

The Berkeley distributions, including FreeBSD and OS/X, go some way

towards ﬁxing the problem. Files can be set to be append-only, immutable or

undeletable for user, system or both. When set by a user at a sufﬁcient security

level during the boot process, they cannot be overridden or removed later, even

by root. Various military variants go to even greater trouble to allow separation

of duty. However the simplest and most common way to protect logs against

root compromise is to keep them separate. In the old days that meant sending

the system log to a printer in a locked room; nowadays, given the volumes of

data, it means sending it to another machine, administered by somebody else.

Second, ACLs only contain the names of users, not of programs; so there

is no straightforward way to implement access triples of (user, program, ﬁle).

4.2 Operating System Access Controls 101

Instead, Unix provides an indirect method: the set-user-id (suid) ﬁle attribute.

The owner of a program can mark it as suid, which enables it to run with the

privilege of its owner rather than the privilege of the user who has invoked

it. So in order to achieve the functionality needed by our second example

above, we could create a user ‘account-package’ to own ﬁle 2 (the accounts

package), make the ﬁle suid and place it in a directory to which Alice has

access. This special user can then be given the access control attributes the

accounts program needs.

One way of looking at this is that an access control problem that is naturally

modelled in three dimensions — by triples (user, program, data) — is being

implemented using two-dimensional mechanisms. These mechanisms are

much less intuitive than triples and people make many mistakes implementing

them. Programmers are often lazy or facing tight deadlines; so they just make

the application suid root, so it can do anything. This practice leads to

some rather shocking security holes. The responsibility for making access

control decisions is moved from the operating system environment to the

program, and most programmers are insufﬁciently experienced and careful

to check everything they should. (It’s hard to know what to check, as the

person invoking a suid root program controls its environment and can often

manipulate this to cause protection failures.)

Third, ACLs are not very good at expressing mutable state. Suppose we

want a transaction to be authorised by a manager and an accountant before

it’s acted on; we can either do this at the application level (say, by having

queues of transactions awaiting a second signature) or by doing something

fancy with suid. In general, managing stateful access rules is difﬁcult; this can

even complicate the revocation of users who have just been ﬁred, as it can be

hard to track down the ﬁles they might have open.

Fourth, the Unix ACL only names one user. Older versions allow a process

to hold only one group id at a time and force it to use a privileged program

to access other groups; newer Unix systems put a process in all groups

that the user is in. This is still much less expressive than one might like. In

theory, the ACL and suid mechanisms can often be used to achieve the desired

effect. In practice, programmers often can’t be bothered to ﬁgure out how to

do this, and design their code to require much more privilege than it really

ought to have.

4.2.4 Apple’s OS/X

Apple’s OS/X operating system is based on the FreeBSD version of Unix run-

ning on top of the Mach kernel. The BSD layer provides memory protection;

applications cannot access system memory (or each others’) unless running

with advanced permissions. This means, for example, that you can kill a

wedged application using the ‘Force Quit’ command; you usually do not have

102 Chapter 4 ■Access Control

to reboot the system. On top of this Unix core are a number of graphics compo-

nents, including OpenGL, Quartz, Quicktime and Carbon, while at the surface

the Aqua user interface provides an elegant and coherent view to the user.

At the ﬁle system level, OS/X is almost a standard Unix. The default

installation has the root account disabled, but users who may administer the

system are in a group ‘wheel’ that allows them to su to root. The most visible

implication is that if you are such a user, you can install programs (you are

asked for the root password when you do so). This may be a slightly better

approach than Windows (up till XP) or Linux, which in practice let only

administrators install software but do not insist on an authentication step

when they do so; the many Windows users who run as administrator for

convenience do dreadful things by mistake (and malware they download does

dreadful things deliberately). Although Microsoft is struggling to catch up

with Vista, as I’ll discuss below, Apple’s advantage may be increased further

by OS/X version 10.5 (Leopard), which is based on TrustedBSD, a variant of

BSD developed for government systems that incorporates mandatory access

control. (I’ll discuss this in Chapter 8.)

4.2.5 Windows – Basic Architecture

The most widespread PC operating system is Windows, whose protection

has been largely based on access control lists since Windows NT. The current

version of Windows (Vista) is fairly complex, so it’s helpful to trace its

antecedents. The protection in Windows NT (Windows v4) was very much

like Unix, and was inspired by it, and has since followed the Microsoft

philosophy of ‘embrace and extend’.

First, rather than just read,write and execute there are separate attributes for

take ownership,change permissions and delete, which means that more ﬂexible

delegation can be supported. These attributes apply to groups as well as users,

and group permissions allow you to achieve much the same effect as suid

programs in Unix. Attributes are not simply on or off, as in Unix, but have

multiple values: you can set AccessDenied,AccessAllowed or SystemAudit. These

are parsed in that order. If an AccessDenied is encountered in an ACL for the

relevant user or group, then no access is permitted regardless of any conﬂicting

AccessAllowed ﬂags. A beneﬁt of the richer syntax is that you can arrange

matters so that everyday conﬁguration tasks, such as installing printers, don’t

require full administrator privileges. (This is rarely done, though.)

Second, users and resources can be partitioned into domains with distinct

administrators, and trust can be inherited between domains in one direction

or both. In a typical large company, you might put all the users into a

personnel domain administered by Human Resources, while resources such

as servers and printers could be in resource domains under departmental

control; individual workstations may even be administered by their users.

4.2 Operating System Access Controls 103

Things would be arranged so that the departmental resource domains trust

the user domain, but not vice versa — so a corrupt or careless departmental

administrator can’t do much damage outside his own domain. The individual

workstations would in turn trust the department (but not vice versa) so that

users can perform tasks that require local privilege (installing many software

packages requires this). Administrators are still all-powerful (so you can’t

create truly tamper-resistant audit trails without using write-once storage

devices or writing to machines controlled by others) but the damage they can

do can be limited by suitable organisation. The data structure used to manage

all this, and hide the ACL details from the user interface, is called the Registry.

Problems with designing a Windows security architecture in very large

organisations include naming issues (which I’ll explore in Chapter 6), the way

domains scale as the number of principals increases (badly), and the restriction

that a user in another domain can’t be an administrator (which can cause

complex interactions between local and global groups).

One peculiarity of Windows is that everyone is a principal, not a default or an

absence of control, so ‘remove everyone’ means just stop a ﬁle being generally

accessible. A resource can be locked quickly by setting everyone to have no

access. This brings us naturally to the subject of capabilities.

4.2.6 Capabilities

The next way to manage the access control matrix is to store it by rows. These

are called capabilities, and in our example in Figure 4.2 above, Bob’s capabilities

would be as in Figure 4.5 here:

User Operating Accounts Accounting Audit

System Program Data Trail

Bob rx r r r

Figure 4.5: A capability

The strengths and weaknesses of capabilities are more or less the opposite

of ACLs. Runtime security checking is more efﬁcient, and we can delegate a

right without much difﬁculty: Bob could create a certiﬁcate saying ‘Here is my

capability and I hereby delegate to David the right to read ﬁle 4 from 9am to

1pm, signed Bob’. On the other hand, changing a ﬁle’s status can suddenly

become more tricky as it can be difﬁcult to ﬁnd out which users have access.

This can be tiresome when we have to investigate an incident or prepare

evidence of a crime.

There were a number of experimental implementations in the 1970s, which

were rather like ﬁle passwords; users would get hard-to-guess bitstrings for

the various read, write and other capabilities to which they were entitled. It

104 Chapter 4 ■Access Control

was found that such an arrangement could give very comprehensive protec-

tion [1344]. It was not untypical to ﬁnd that almost all of an operating system

could run in user mode rather than as supervisor, so operating system bugs

were not security critical. (In fact, many operating system bugs caused security

violations, which made debugging the operating system much easier.)

The IBM AS/400 series systems employed capability-based protection, and

enjoyed some commercial success. Now capabilities have made a limited

comeback in the form of public key certiﬁcates. I’ll discuss the mechanisms

of public key cryptography in Chapter 5, and give more concrete details of

certiﬁcate-based systems, such as SSL/TLS, in Part II. For now, think of a

public key certiﬁcate as a credential, signed by some authority, which declares

that the holder of a certain cryptographic key is a certain person, or a member

of some group, or the holder of some privilege.

As an example of where certiﬁcate-based capabilities can be useful, consider

a hospital. If we implemented a rule like ‘a nurse shall have access to all the

patients who are on her ward, or who have been there in the last 90 days’

naively, each access control decision in the patient record system will require

several references to administrative systems, to ﬁnd out which nurses and

which patients were on which ward, when. So a failure of the administrative

systems can now affect patient safety much more directly than before, and

this is clearly a bad thing. Matters can be much simpliﬁed by giving nurses

certiﬁcates which entitle them to access the ﬁles associated with their current

ward. Such a system has been used for several years at our university hospital.

Public key certiﬁcates are often considered to be ‘crypto’ rather than ‘access

control’, with the result that their implications for access control policies

and architectures are not thought through. The lessons that could have been

learned from the capability systems of the 1970s are generally having to be

rediscovered (the hard way). In general, the boundary between crypto and

access control is a fault line where things can easily go wrong. The experts often

come from different backgrounds, and the products from different suppliers.

4.2.7 Windows – Added Features

A number of systems, from mainframe access control products to research

systems, have combined ACLs and capabilities in an attempt to get the best of

both worlds. But the most important application of capabilities is in Windows.

Windows 2000 added capabilities in two ways which can override or

complement the ACLs of Windows NT. First, users or groups can be either

whitelisted or blacklisted by means of proﬁles. (Some limited blacklisting was

also possible in NT4.) Security policy is set by groups rather than for the

system as a whole. Groups are intended to be the primary method for

centralized conﬁguration management and control (group policy overrides

individual proﬁles). Group policy can be associated with sites, domains or

4.2 Operating System Access Controls 105

organizational units, so it can start to tackle some of the real complexity

problems with naming. Policies can be created using standard tools or custom

coded. Groups are deﬁned in the Active Directory, an object-oriented database

that organises users, groups, machines, and organisational units within a

domain in a hierarchical namespace, indexing them so they can be searched

for on any attribute. There are also ﬁner grained access control lists on

individual resources.

As already mentioned, Windows adopted Kerberos from Windows 2000

as its main means of authenticating users across networks1. This is encap-

sulated behind the Security Support Provider Interface (SSPI) which enables

administrators to plug in other authentication services.

This brings us to the second way in which capabilities insinuate their way

into Windows: in many applications, people use the public key protocol TLS,

which is widely used on the web, and which is based on public key certiﬁcates.

The management of these certiﬁcates can provide another, capability-oriented,

layer of access control outside the purview of the Active Directory.

The latest version of Windows, Vista, introduces a further set of protection

mechanisms. Probably the most important is a package of measures aimed at

getting away from the previous default situation of all software running as root.

First, the kernel is closed off to developers; second, the graphics subsystem

is removed from the kernel, as are most drivers; and third, User Account

Control (UAC) replaces the default administrator privilege with user defaults

instead. This involved extensive changes; in XP, many routine tasks required

administrative privilege and this meant that enterprises usually made all their

users administrators, which made it difﬁcult to contain the effects of malware.

Also, developers wrote their software on the assumption that it would have

access to all system resources.

In Vista, when an administrator logs on, she is given two access tokens: a

standard one and an admin one. The standard token is used to start the desktop,

explorer.exe, which acts as the parent process for later user processes. This

means, for example, that even administrators browse the web as normal users,

and malware they download can’t overwrite system ﬁles unless given later

authorisation. When a task is started that requires admin privilege, then a user

who has it gets an elevation prompt asking her to authorise it by entering an

admin password. (This brings Windows into line with Apple’s OS/X although

the details under the hood differ somewhat.)

1It was in fact a proprietary variant, with changes to the ticket format which prevent Windows

clients from working with existing Unix Kerberos infrastructures. The documentation for the

changes was released on condition that it was not used to make compatible implementations.

Microsoft’s goal was to get everyone to install Win2K Kerberos servers. This caused an outcry in

the open systems community [121]. Since then, the European Union prosecuted an antitrust case

against Microsoft that resulted in interface speciﬁcations being made available in late 2006.

106 Chapter 4 ■Access Control

Of course, admin users are often tricked into installing malicious software,

and so Vista provides further controls in the form of ﬁle integrity levels. I’ll

discuss these along with other mandatory access controls in Chapter 8 but

the basic idea is that low-integrity processes (such as code you download

from the Internet) should not be able to modify high-integrity data (such

as system ﬁles). It remains to be seen how effective these measures will be;

home users will probably want to bypass them to get stuff to work, while

Microsoft is providing ever-more sophisticated tools to enable IT managers to

lock down corporate networks — to the point, for example, of preventing most

users from installing anything from removable media. UAC and mandatory

integrity controls can certainly play a role in this ecology, but we’ll have to

wait and see how things develop.

The ﬁnal problem with which the Vista developers grappled is the fact that

large numbers of existing applications expect to run as root, so that they can

fool about with registry settings (for a hall of shame, see [579]). According to

the Microsoft folks, this is a major reason for Windows’ lack of robustness:

applications monkey with system resources in incompatible ways. So there is

an Application Information Service that launches applications which require

elevated privileges to run. Vista uses virtualization technology for legacy

applications: if they modify the registry, for example, they don’t modify the

‘real’ registry but simply the version of it that they can see. This is seen as a

‘short-term ﬁx’ [885]. I expect it will be around for a long time, and I’m curious

to see whether the added complexity will be worth the reduced malware risk.

Despite virtualisation, the bugbear with Vista is compatibility. As this book

went to press in early January 2008, sales of Vista were still sluggish, with

personal users complaining that games and other applications just didn’t

work, while business users were waiting for service pack 1 and postponing

large-scale roll-out to late 2008 or even early 2009. It has clearly been expensive

for Microsoft to move away from running everything as root, but it’s clearly a

necessary move and they deserve full credit for biting the bullet.

To sum up, Windows provides a richer and more ﬂexible set of access

control tools than any system previously sold in mass markets. It does still

have design limitations. Implementing roles whose requirements differ from

those of groups could be tricky in some applications; SSL certiﬁcates are the

obvious way to do this but require an external management infrastructure.

Second, Windows is still (in its consumer incarnations) a single-user operating

system in the sense that only one person can operate a PC at a time. Thus

if I want to run an unprivileged, sacriﬁcial user on my PC for accessing

untrustworthy web sites that might contain malicious code, I have to log off

and log on again, or use other techniques which are so inconvenient that

few users will bother. (On my Mac, I can run two users simultaneously and

switch between them quickly.) So Vista should be seen as the latest step

on a journey, rather than a destination. The initial version also has some

4.2 Operating System Access Controls 107

undesirable implementation quirks. For example, it uses some odd heuristics

to try to maintain backwards compatibility with programs that assume they’ll

run as administrator: if I compile a C++ program called Fred Installer.exe

then Vista will ask for elevated privilege to run it, and tell it that it’s running

on Windows XP, while if I call the program simply Fred.exe it will run as

user and be told that it’s running on Vista [797]. Determining a program’s

privileges on the basis of its ﬁlename is just bizarre.

And ﬁnally, there are serious usability issues. For example, most users still

run administrator accounts all the time, and will be tempted to disable UAC;

if they don’t, they’ll become habituated to clicking away the UAC dialog box

that forever asks them if they really meant to do what they just tried to. For

these reasons, UAC may be much less effective in practice than it might be in

theory [555]. We will no doubt see in due course.

It’s interesting to think about what future access controls could support,

for example, an electronic banking application that would be protected from

malware running on the same machine. Microsoft did come up with some

ideas in the context of its ‘Trusted Computing’ project, which I’ll describe

below in section 4.2.11, but they didn’t make it into Vista.

4.2.8 Middleware

Doing access control at the level of ﬁles and programs was all very well in the

early days of computing, when these were the resources that mattered. Since

about the 1980s, growing scale and complexity has meant led to access control

being done at other levels instead of (sometimes as well as) at the operating

system level. For example, a bank’s branch bookkeeping system will typically

run on top of a database product, and the database looks to the operating

system as one large ﬁle. This means that the access control has to be done in

the database; all the operating system supplies it may be an authenticated ID

for each user who logs on.

4.2.8.1 Database Access Controls

Until the dotcom boom, database security was largely a back-room concern.

But it is now common for enterprises to have critical databases, that handle

inventory, dispatch and e-commerce, fronted by web servers that pass trans-

actions to the databases directly. These databases now contain much of the

data of greatest relevance to our lives — such as bank accounts, vehicle regis-

trations and employment records — and front-end failures sometimes expose

the database itself to random online users.

Database products, such as Oracle, DB2 and MySQL, have their own access

control mechanisms. As the database looks to the operating system as a single

large ﬁle, the most the operating system can do is to identify users and to

108 Chapter 4 ■Access Control

separate the database from other applications running on the same machine.

The database access controls are in general modelled on operating-system

mechanisms, with privileges typically available for both users and objects

(so the mechanisms are a mixture of access control lists and capabilities).

However, the typical database access control architecture is more complex

even than Windows: Oracle 10g has 173 system privileges of which at least six

can be used to take over the system completely [804]. There are more privileges

because a modern database product is very complex and some of the things

one wants to control involve much higher levels of abstraction than ﬁles or

processes. The ﬂip side is that unless developers know what they’re doing,

they are likely to leave a back door open.

Some products let developers bypass operating-system controls. For

example, Oracle has both operating system accounts (whose users must be

authenticated externally by the platform) and database accounts (whose users

are authenticated directly by the Oracle software). It is often more convenient

to use database accounts as it saves the developer from synchronising his work

with the details of what other departments are doing. In many installations,

the database is accessible directly from the outside; this raises all sorts of

issues from default passwords to ﬂaws in network protocols. Even where the

database is shielded from the outside by a web service front-end, this often

contains loopholes that let SQL code be inserted into the database.

Database security failures can also cause problems directly. The Slammer

worm in January 2003 propagated itself using a stack-overﬂow in Microsoft

SQL Server 2000 and created large amounts of trafﬁc and compromised

machines sent large numbers of attack packets to random IP addresses.

Just as Windows is trickier to conﬁgure securely, because it’s more complex,

so the typical database system is trickier still, and it takes specialist knowledge

that’s beyond the scope of this book. Database security is now a discipline

in its own right; if you have to lock down a database system — or even just

review one as part of a broader assignment — I’d strongly recommend that

you read a specialist text, such as David Litchﬁeld’s [804].

4.2.8.2 General Middleware Issues

There are a number of aspects common to middleware security and application-

level controls. The ﬁrst is granularity: as the operating system works with ﬁles,

these are usually the smallest objects with which its access control mechanisms

can deal. The second is state. An access rule such as ‘a nurse can see the records

of any patient on her ward’ or ‘a transaction over $100,000 must be authorised

by a manager and an accountant’ both involve managing state: in the ﬁrst

case the duty roster, and in the second the list of transactions that have so

far been authorised by only one principal. The third is level: we may end up

with separate access control systems at the machine, network and application

4.2 Operating System Access Controls 109

levels, and as these typically come from different vendors it may be difﬁcult

to keep them consistent.

Ease of administration is often a critical bottleneck. In companies I’ve

advised, the administration of the operating system and the database system

have been done by different departments, which do not talk to each other;

and often user pressure drives IT departments to put in crude hacks which

make the various access control systems seem to work as one, but which open

up serious holes. An example is ‘single sign-on’. Despite the best efforts

of computer managers, most large companies accumulate systems of many

different architectures, so users get more and more logons to different systems

and the cost of administering them escalates. Many organisations want to give

each employee a single logon to all the machines on the network. Commercial

solutions may involve a single security server through which all logons must

pass, and the use of a smartcard to do multiple authentication protocols

for different systems. Such solutions are hard to engineer properly, and the

security of the best system can very easily be reduced to that of the worst.

4.2.8.3 ORBs and Policy Languages

These problems led researchers to look for ways in which access control for a

number of applications might be handled by standard middleware. Research

in the 1990s focussed on object request brokers (ORBs). An ORB is a software

component that mediates communications between objects (an object consists

of code and data bundled together, an abstraction used in object-oriented

languages such as C++). An ORB typically provides a means of controlling

calls that are made across protection domains. The Common Object Request

Broker Architecture (CORBA) is an attempt at an industry standard for object-

oriented systems; a book on CORBA security is [182]. This technology is

starting to be adopted in some industries, such as telecomms.

Research since 2000 has included work on languages to express security pol-

icy, with projects such as XACML (Sun), XrML (ISO) and SecPAL (Microsoft).

They followed early work on ‘Policymaker’ by Matt Blaze and others [188],

and vary in their expressiveness. XrML deals with subjects and objects but

not relationships, so cannot easily express a concept such as ‘Alice is Bob’s

manager’. XACML does relationships but does not support universally quan-

tiﬁed variables, so it cannot easily express ‘a child’s guardian may sign its

report card’ (which we might want to program as ‘if xis a child and yis x’s

guardian and zis x’s report card, then ymay sign z). The initial interest in

these languages appears to come from the military and the rights-management

industry, both of which have relatively simple state in their access control poli-

cies. Indeed, DRM engineers have already developed a number of specialised

rights-management languages that are built into products such as Windows

Media Player and can express concepts such as ‘User Xcan play this ﬁle as

110 Chapter 4 ■Access Control

audio until the end of September and can burn it to a CD only once.’ The push

for interoperable DRM may create a demand for more general mechanisms

that can embrace and extend the products already in the ﬁeld.

If a suitably expressive policy language emerges, and is adopted as a

standard scripting language on top of the access-control interfaces that major

applications provide to their administrators, it might provide some value by

enabling people to link up access controls when new services are constructed

on top of multiple existing services. There are perhaps two caveats. First,

people who implement access control when customizing a package are not

likely to do so as a full-time job, and so it may be better to let them use a

language with which they are familiar, in which they will be less likely to make

mistakes. Second, security composition is a hard problem; it’s easy to come up

with examples of systems that are secure in isolation but that break horribly

when joined up together. We’ll see many examples in Part II.

Finally, the higher in a system we build the protection mechanisms, the

more complex they’ll be, the more other software they’ll rely on, and the closer

they’ll be to the error-prone mark 1 human being — so the less dependable

they are likely to prove. Platform vendors such as Microsoft have more security

PhDs, and more experience in security design, than almost any application

vendor; and a customer who customises an application package usually has

less experience still. Code written by users is most likely to have glaring ﬂaws.

For example, the fatal accidents that happened in healthcare as a result of the

Y2K bug were not platform failures, but errors in spreadsheets developed by

individual doctors, to do things like processing lab test results and calculating

radiology dosages. Letting random users write security-critical code carries

the same risks as letting them write safety-critical code.

4.2.9 Sandboxing and Proof-Carrying Code

The late 1990s saw the emergence of yet another way of implementing access

control: the software sandbox. This was introduced by Sun with its Java

programming language. The model is that a user wants to run some code

that she has downloaded from the web as an applet, but is concerned that

the applet might do something nasty, such as taking a list of all her ﬁles and

mailing it off to a software marketing company.

The designers of Java tackled this problem by providing a ‘sandbox’ for such

code — a restricted environment in which it has no access to the local hard

disk (or at most only temporary access to a restricted directory), and is only

allowed to communicate with the host it came from. These security objectives

are met by having the code executed by an interpreter — the Java Virtual

Machine (JVM) — which has only limited access rights [539]. Java is also used

on smartcards but (in current implementations at least) the JVM is in effect a

compiler external to the card, which raises the issue of how the code it outputs

4.2 Operating System Access Controls 111

can be got to the card in a trustworthy manner. Another application is in the

new Blu-ray format for high-deﬁnition DVDs; players have virtual machines

that execute rights-management code bundled with the disk. (I describe the

mechanisms in detail in section 22.2.6.2.)

An alternative is proof-carrying code. Here, code to be executed must carry

with it a proof that it doesn’t do anything that contravenes the local security

policy. This way, rather than using an interpreter with the resulting speed

penalty, one merely has to trust a short program that checks the proofs

supplied by downloaded programs before allowing them to be executed. The

overhead of a JVM is not necessary [956].

Both of these are less general alternatives to an architecture supporting

proper supervisor-level conﬁnement.

4.2.10 Virtualization

This refers to systems that enable a single machine to emulate a number

of machines independently. It was invented in the 1960s by IBM [336]; back

when CPUs were very expensive, a single machine could be partitioned using

VM/370 into multiple virtual machines, so that a company that bought two

mainframes could use one for its production environment and the other as

a series of logically separate machines for development, testing, and minor

applications.

The move to PCs saw the emergence of virtual machine software for this

platform, with offerings from various vendors, notably VMware and (in open-

source form) the Xen project. Virtualization is very attractive to the hosting

industry, as clients can be sold a share of a machine in a hosting centre for

much less than a whole machine. In the few years that robust products have

been available, their use has become extremely widespread.

At the client end, virtualization allows people to run a host operating system

on top of a guest (for example, Windows on top of Linux or OS/X) and this

offers not just ﬂexibility but the prospect of better containment. For example,

an employee might have two copies of Windows running on his laptop — a

locked-down version with her ofﬁce environment, and another for use at

home. The separation can be high-grade from the technical viewpoint; the

usual problem is operational. People may feel the need to share data between

the two virtual machines and resort to ad-hoc mechanisms, from USB sticks to

webmail accounts, that undermine the separation. Military system designers

are nonetheless very interested in virtualization; I discuss their uses of it in

section 8.5.3.

4.2.11 Trusted Computing

The ‘Trusted Computing’ initiative was launched by Microsoft, Intel, IBM, HP

and Compaq to provide a more secure PC. Their stated aim was to provide

112 Chapter 4 ■Access Control

software and hardware add-ons to the PC architecture that would enable

people to be sure that a given program was running on a machine with a

given speciﬁcation; that is, that software had not been patched (whether by

the user or by other software) and was running on a identiﬁable type and

conﬁguration of PC rather than on an emulator. The initial motivation was

to support digital rights management. The problem there was this: if Disney

was going to download a copy of a high-deﬁnition movie to Windows Media

Player on your PC, how could they be sure it wasn’t a hacked version, or

running on a copy of Windows that was itself running on top of Xen? In either

case, the movie might be ripped and end up on ﬁle-sharing systems.

The hardware proposal was to add a chip, the Trusted Platform Module

or TPM, which could monitor the PC at boot time and report its state to the

operating system; cryptographic keys would be made available depending on

this state. Thus if a platform were modiﬁed — for example, by changing the

boot ROM or the hard disk controller — different keys would be derived and

previously encrypted material would not be available. A PC would also be able

to use its TPM to certify to other PCs that it was in an ‘approved’ conﬁguration,

a process called remote attestation. Of course, individual PCs might be hacked in

less detectable ways, such as by installing dual-ported memory or interfering

with the bus from the TPM to the CPU — but the idea was to exclude low-cost

break-once-run-anywhere attacks. Then again, the operating system will break

from time to time, and the media player; so the idea was to make the content

protection depend on as little as possible, and have revocation mechanisms

that would compel people to upgrade away from broken software.

Thus a vendor of digital content might only serve premium products to a

machine in an approved conﬁguration. Furthermore, data-based access control

policies could be enforced. An example of these is found in the ‘Information

Rights Management’ mechanisms introduced with Ofﬁce 2003; here, a ﬁle

can be marked with access controls in a rights expression language which can

state, for example, that it may only be read by certain named users and only

for a certain period of time. Word-processing documents (as well as movies)

could be marked ‘view three times only’; a drug dealer could arrange for

the spreadsheet with November’s cocaine shipments to be unreadable after

December, and so on.

There are objections to data-based access controls based on competition

policy, to which I’ll return in Part III. For now, my concern is the mechanisms.

The problem facing Microsoft was to maintain backwards compatibility with

the bad old world where thousands of buggy and insecure applications run as

administrator, while creating the possibility of new access domains to which

the old buggy code has no access. One proposed architecture, Palladium, was

unveiled in 2002; this envisaged running the old, insecure, software in parallel

with new, more secure components.

4.3 Hardware Protection 113

In addition to the normal operating system, Windows would have a ‘Nexus’,

a security kernel small enough to be veriﬁed, that would talk directly to the

TPM hardware and monitor what went on in the rest of the machine; and each

application would have a Nexus Control Program (NCP) that would run on

top of the Nexus in the secure virtual machine and manage critical things like

cryptographic keys. NCPs would have direct access to hardware. In this way,

a DRM program such as a media player could keep its crypto keys in its NCP

and use them to output content to the screen and speakers directly — so that

the plaintext content could not be stolen by spyware.

At the time of writing, the curtained memory features are not used in Vista;

presentations at Microsoft research workshops indicated that getting ﬁne-

grained access control and virtualization to work at the middle layers of such

a complex operating system has turned out to be a massive task. Meanwhile

the TPM is available for secure storage of root keys for utilities such as hard

disk encryption; this is available as ’BitLocker’ in the more expensive versions

of Vista. It remains to be seen whether the more comprehensive vision of

Trusted Computing can be made to work; there’s a growing feeling in the

industry that it was too hard and, as it’s also politically toxic, it’s likely to be

quietly abandoned. Anyway, TPMs bring us to the more general problem of

the hardware protection mechanisms on which access controls are based.

4.3 Hardware Protection

Most access control systems set out not just to control what users can do, but

to limit what programs can do as well. In most systems, users can either write

programs, or download and install them. So programs may be buggy or even

malicious.

Preventing one process from interfering with another is the protection prob-

lem. The conﬁnement problem is usually deﬁned as that of preventing programs

communicating outward other than through authorized channels. There are

several ﬂavours of each. The goal may be to prevent active interference, such

as memory overwriting, or to stop one process reading another’s memory

directly. This is what commercial operating systems set out to do. Military

systems may also try to protect metadata — data about other data, or subjects,

or processes — so that, for example, a user can’t ﬁnd out what other users are

logged on to the system or what processes they’re running. In some applica-

tions, such as processing census data, conﬁnement means allowing a program

to read data but not release anything about it other than the results of certain

constrained queries.

Unless one uses sandboxing techniques (which are too restrictive for general

programming environments), solving the conﬁnement problem on a single

processor means, at the very least, having a mechanism that will stop one

114 Chapter 4 ■Access Control

program from overwriting another’s code or data. There may be areas of

memory that are shared in order to allow interprocess communication; but

programs must be protected from accidental or deliberate modiﬁcation, and

they must have access to memory that is similarly protected.

This usually means that hardware access control must be integrated with the

processor’s memory management functions. A typical mechanism is segment

addressing. Memory is addressed by two registers, a segment register which

points to a segment of memory, and another address register which points to

a location within that segment. The segment registers are controlled by the

operating system, and often by a special component of it called the reference

monitor which links the access control mechanisms with the hardware.

The actual implementation has become more complex as the processors

themselves have. Early IBM mainframes had a two state CPU: the machine was

either in authorized state or it was not. In the latter case, the program was

restricted to a memory segment allocated by the operating system. In the

former, it could alter the segment registers at will. An authorized program

was one that was loaded from an authorized library.

Any desired access control policy can be implemented on top of this,

given suitable authorized libraries, but this is not always efﬁcient; and system

security depends on keeping bad code (whether malicious or buggy) out of

the authorized libraries. So later processors offered more complex hardware

mechanisms. Multics, an operating system developed at MIT in the 1960’s and

which inspired the development of Unix, introduced rings of protection which

express differing levels of privilege: ring 0 programs had complete access to

disk, supervisor states ran in ring 2, and user code at various less privileged

levels [1139]. Its features have to some extent been adopted in more recent

processors, such as the Intel main processor line from the 80286 onwards.

There are a number of general problems with interfacing hardware and

software security mechanisms. For example, it often happens that a less

privileged process such as application code needs to invoke a more privileged

process such as a device driver. The mechanisms for doing this need to

be designed with some care, or security bugs can be expected. The IBM

mainframe operating system MVS, for example, had a bug in which a program

which executed a normal and an authorized task concurrently could make the

former authorized too [774]. Also, performance may depend quite drastically

on whether routines at different privilege levels are called by reference or by

value [1139].

4.3.1 Intel Processors, and ‘Trusted Computing’

Early Intel processors, such as the 8088/8086 used in early PCs, had no

distinction between system and user mode, and thus no protection at all — any

running program controlled the whole machine. The 80286 added protected

4.3 Hardware Protection 115

segment addressing and rings, so for the ﬁrst time it could run proper operating

systems. The 80386 had built in virtual memory, and large enough memory

segments (4 Gb) that they could be ignored and the machine treated as a

32-bit ﬂat address machine. The 486 and Pentium series chips added more

performance (caches, out of order execution and MMX).

The rings of protection are supported by a number of mechanisms. The

current privilege level can only be changed by a process in ring 0 (the kernel).

Procedures cannot access objects in lower level rings directly but there are

gates which allow execution of code at a different privilege level and which

manage the supporting infrastructure, such as multiple stack segments for

different privilege levels and exception handling. For more details, see [646].

The Pentium 3 ﬁnally added a new security feature — a processor serial

number. This caused a storm of protest because privacy advocates feared it

could be used for all sorts of ‘big brother’ purposes, which may have been

irrational as computers have all sorts of unique numbers in them that software

can use to tell which machine it’s running on (examples range from MAC

addresses to the serial numbers of hard disk controllers). At the time the serial

number was launched, Intel had planned to introduce cryptographic support

into the Pentium by 2000 in order to support DRM. Their thinking was that

as they earned 90% of their proﬁts from PC microprocessors, where they had

90% of the market, they could only grow their business by expanding the

market for PCs; and since the business market was saturated, that meant sales

to homes where, it was thought, DRM would be a requirement.

Anyway, the outcry against the Pentium serial number led Intel to set up

an industry alliance, now called the Trusted Computing Group, to introduce

cryptography into the PC platform by means of a separate processor, the

Trusted Platform Module (TPM), which is a smartcard chip mounted on

the PC motherboard. The TPM works together with curtained memory features

introduced in the Pentium to enable operating system vendors to create

memory spaces isolated from each other, and even against a process in

one memory space running with administrator privileges. The mechanisms

proposed by Microsoft are described above, and have not been made available

in commercial releases of Windows at the time of writing.

One Intel hardware feature that has been implemented and used is the

x86 virtualization support, known as Intel VT (or its development name,

Vanderpool). AMD has an equivalent offering. Processor architectures such

as S/370 and M68000 are easy to virtualize, and the theoretical requirements

for this have been known for many years [1033]. The native Intel instruction

set, however, had instructions that were hard to virtualize, requiring messy

workarounds, such as patches to hosted operating systems. Processors with

these extensions can use products such as Xen to run unmodiﬁed copies of

guest operating systems. (It does appear, though, that if the Trusted Computing

116 Chapter 4 ■Access Control

mechanisms are ever implemented, it will be complex to make them work

alongside virtualization.)

4.3.2 ARM Processors

The ARM is the 32-bit processor core most commonly licensed to third party

vendors of embedded systems; hundreds of millions are used in mobile phones

and other consumer electronic devices. The original ARM (which stood for

Acorn Risc Machine) was the ﬁrst commercial RISC design. ARM chips are

also used in many security products, from the Capstone chips used by the

US government to protect secret data, to the crypto accelerator boards from

ﬁrms like nCipher that do cryptography for large web sites. A fast multiply-

and-accumulate instruction and low power consumption made the ARM very

attractive for embedded applications doing crypto and/or signal processing.

The standard reference is [508].

The ARM is licensed as a processor core, which chip designers can include

in their products, plus a number of optional add-ons. The basic core contains

separate banks of registers for user and system processes, plus a software

interrupt mechanism that puts the processor in supervisor mode and transfers

control to a process at a ﬁxed address. The core contains no memory manage-

ment, so ARM-based designs can have their hardware protection extensively

customized. A system control coprocessor is available to help with this. It can

support domains of processes that have similar access rights (and thus share

the same translation tables) but that retain some protection from each other.

This gives fast context switching. Standard product ARM CPU chips, from the

model 600 onwards, have this memory support built in.

There is a version, the Amulet, which uses self-timed logic. Eliminating

the clock saved power and reduces RF interference, but made it necessary to

introduce hardware protection features, such as register locking, into the main

processor itself so that contention between different hardware processes could

be managed. This is an interesting example of protection techniques typical of

an operating system being recycled in main-line processor design.

4.3.3 Security Processors

Specialist security processors range from the chips in smartcards, through the

TPM chips now ﬁxed to most PC motherboards (which are basically smartcard

chips with parallel interfaces) and crypto accelerator boards, to specialist

crypto devices.

Many of the lower-cost smartcards still have 8-bit processors. Some of them

have memory management routines that let certain addresses be read only

when passwords are entered into a register in the preceding few instructions.

The goal was that the various principals with a stake in the card — perhaps

4.4 What Goes Wrong 117

a card manufacturer, an OEM, a network and a bank — can all have their

secrets on the card and yet be protected from each other. This may be a matter

of software; but some cards have small, hardwired access control matrices to

enforce this protection.

Many of the encryption devices used in banking to handle ATM PINs have

a further layer of application-level access control in the form of an ‘authorized

state’ which must be set (by two console passwords, or a physical key) when

PINs are to be printed. This is reminiscent of the old IBM mainframes, but is

used for manual rather than programmatic control: it enables a shift supervisor

to ensure that he is present when this job is run. Similar devices are used by

the military to distribute keys. I’ll discuss cryptoprocessors in more detail in

Chapter 16.

4.4 What Goes Wrong

Popular operating systems such as Unix / Linux and Windows are very large

and complex, so they have many bugs. They are used in a huge range of

systems, so their features are tested daily by millions of users under very

diverse circumstances. Consequently, many bugs are found and reported.

Thanks to the net, knowledge spreads widely and rapidly. Thus at any one

time, there may be dozens of security ﬂaws that are known, and for which

attack scripts are circulating on the net. A vulnerability has a typical lifecycle

whereby it is discovered; reported to CERT or to the vendor; a patch is

shipped; the patch is reverse-engineered, and an exploit is produced for the

vulnerability; and people who did not apply the patch in time ﬁnd that their

machines have been recruited to a botnet when their ISP cuts them off for

sending spam. There is a variant in which the vulnerability is exploited at

once rather than reported — often called a zero-day exploit as attacks happen

from day zero of the vulnerability’s known existence. The economics, and

the ecology, of the vulnerability lifecycle are the subject of intensive study by

security economists; I’ll discuss their ﬁndings in Part III.

The traditional goal of an attacker was to get a normal account on the system

and then become the system administrator, so he could take over the system

completely. The ﬁrst step might have involved guessing, or social-engineering,

a password, and then using one of the many known operating-system bugs

that allow the transition from user to root. A taxonomy of such technical

ﬂaws was compiled in 1993 by Carl Landwehr [774]. These involved failures

in the technical implementation, of which I will give examples in the next two

sections, and also in the higher level design; for example, the user interface

might induce people to mismanage access rights or do other stupid things

which cause the access control to be bypassed. I will give some examples in

section 4.4.3 below.

118 Chapter 4 ■Access Control

The user/root distinction has become less important in the last few years for

two reasons. First, Windows PCs have predominated, running applications

that insist on being run as administrator, so any application that can be

compromised gives administrator access. Second, attackers have come to

focus on compromising large numbers of PCs, which they can organise into

a botnet in order to send spam or phish and thus make money. Even if your

mail client were not running as administrator, it would still be useful to a

spammer who could control it. However, botnet herders tend to install rootkits

which, as their name suggests, run as root; and the user/root distinction does

still matter in business environments, where you do not want a compromised

web server or database application to expose your other applications as well.

Perhaps if large numbers of ordinary users start running Vista with User

Account Control enabled, it will make the botnet herders’ lives a bit harder.

We may at least hope.

In any case, the basic types of technical attack have not changed hugely

since the early 1990s and I’ll now consider them brieﬂy.

4.4.1 Smashing the Stack

About half of the technical attacks on operating systems that are reported in

Computer Emergency Response Team (CERT) bulletins and security mailing lists

involve memory overwriting attacks, colloquially known as ‘smashing the

stack’. The proportion was even higher in the late 1990s and early 2000s but is

now dropping slowly.

The basic idea behind the stack-smashing attack is that programmers are

often careless about checking the size of arguments, so an attacker who passes

a long argument to a program may ﬁnd that some of it gets treated as code

rather than data. A classic example was a vulnerability in the Unix finger

command. A widespread implementation of this would accept an argument

of any length, although only 256 bytes had been allocated for this argument

by the program. When an attacker used the command with a longer argument,

the trailing bytes of the argument ended up being executed by the system.

The usual technique is to arrange for the trailing bytes of the argument to

have a landing pad — a long space of no-operation (NOP) commands, or other

catch the processor if it executes any of them. The landing pad delivers the

processor to the attack code which will do something like creating a root

account with no password, or starting a shell with administrative privilege

directly (see Figure 4.6).

Many of the vulnerabilities reported routinely by CERT and bugtraq are

variants on this theme. I wrote in the ﬁrst edition of this book, in 2001, ‘There

is really no excuse for the problem to continue, as it’s been well known for a

generation’. Yet it remains a problem.

4.4 What Goes Wrong 119

‘Landing pad’

Area allocated for

input buffer

Target machine

memory map

Malicious

argument

Malicious code

Over-long

input

Executable

code

Figure 4.6: Stack smashing attack

Most of the early 1960’s time sharing systems suffered from it, and ﬁxed

it [549]. Penetration analysis efforts at the System Development Corporation

in the early ’70s showed that the problem of ‘unexpected parameters’ was still

one of the most frequently used attack strategies [799]. Intel’s 80286 proces-

sor introduced explicit parameter checking instructions — verify read, verify

write, and verify length — in 1982, but they were avoided by most software

designers to prevent architecture dependencies. In 1988, large numbers of

Unix computers were brought down simultaneously by the ‘Internet worm’,

which used the finger vulnerability described above, and thus brought mem-

ory overwriting attacks to the notice of the mass media [1206]. A 1998 survey

paper described memory overwriting attacks as the ‘attack of the decade’ [329].

Yet programmers still don’t check the size of arguments, and holes keep on

being found. The attack isn’t even limited to networked computer systems: at

least one smartcard could be defeated by passing it a message longer than its

programmer had anticipated.

4.4.2 Other Technical Attacks

In 2002, Microsoft announced a security initiative that involved every pro-

grammer being trained in how to write secure code. (The book they produced

for this, ‘Writing Secure Code’ by Michael Howard and David LeBlanc, is good;

I recommend it to my students [627].) Other tech companies have launched

similar training programmes. Despite the training and the tools, memory

overwriting attacks are still appearing, to the great frustration of software

company managers. However, they are perhaps half of all new vulnerabilities

now rather than the 90% they were in 2001.

The other new vulnerabilities are mostly variations on the same general

theme, in that they occur when data in grammar A is interpreted as being in

grammar B. A stack overﬂow is when data are accepted as input (e.g. a URL)

120 Chapter 4 ■Access Control

and end up being executed as machine code. They are essentially failures of

type safety.

Aformat string vulnerability arises when a machine accepts input data

as a formatting instruction (e.g. %n in the C command printf()). These

commonly arise when a programmer tries to print user-supplied data and

carelessly allows the print command to interpret any formatting instructions

in the string; this may allow the string’s author to write to the stack. There

are many other variants on the theme; buffer overﬂows can be induced by

improper string termination, passing an inadequately sized buffer to a path

manipulation function, and many other subtle errors. See Gary McGraw’s

book ‘Software Security’ [858] for a taxonomy.

SQL insertion attacks commonly arise when a careless web developer

passes user input to a back-end database without checking to see whether it

contains SQL code. The game is often given away by error messages, from

which a capable and motivated user may infer enough to mount an attack.

(Indeed, a survey of business websites in 2006 showed that over 10% were

potentially vulnerable [1234].) There are similar command-injection problems

afﬂicting other languages used by web developers, such as PHP and perl. The

remedy in general is to treat all user input as suspicious and validate it.

Checking data sizes is all very well when you get the buffer size calculation

correct, but when you make a mistake — for example, if you fail to consider

all the edge cases — you can end up with another type of attack called an

integer manipulation attack. Here, an overﬂow, underﬂow, wrap-around or

truncation can result in the ‘security’ code writing an inappropriate number

of bytes to the stack.

Once such type-safety attacks are dealt with, race conditions are probably

next. These occur when a transaction is carried out in two or more stages, and

it is possible for someone to alter it after the stage which involves verifying

access rights. I mentioned in Chapter 2 how a race condition can allow users

to log in as other users if the userid can be overwritten while the password

validation is in progress. Another classic example arose in the Unix command

to create a directory, ‘mkdir’, which used to work in two steps: the storage

was allocated, and then ownership was transferred to the user. Since these

steps were separate, a user could initiate a ‘mkdir’ in background, and if this

completed only the ﬁrst step before being suspended, a second process could

be used to replace the newly created directory with a link to the password

ﬁle. Then the original process would resume, and change ownership of the

password ﬁle to the user. The /tmp directory, used for temporary ﬁles, can

often be abused in this way; the trick is to wait until an application run by a

privileged user writes a ﬁle here, then change it to a symbolic link to another

ﬁle somewhere else — which will be removed when the privileged user’s

application tries to delete the temporary ﬁle.

4.4 What Goes Wrong 121

A wide variety of other bugs have enabled users to assume root status

and take over the system. For example, the PDP-10 TENEX operating system

had the bug that the program address could overﬂow into the next bit of

the process state word which was the privilege-mode bit; this meant that a

program overﬂow could put a program in supervisor state. In another example,

some Unix implementations had the feature that if a user tried to execute the

command su when the maximum number of ﬁles were open, then su was

unable to open the password ﬁle and responded by giving the user root status.

In more modern systems, the most intractable user-to-root problems tend to be

feature interactions. For example, we’ve struggled with backup and recovery

systems. It’s convenient if you can let users recover their own ﬁles, rather than

having to call a sysadmin — but how do you protect information assets from

a time traveller, especially if the recovery system allows him to compose parts

of pathnames to get access to directories that were always closed to him? And

what if the recovery functionality is buried in an application to which he needs

access in order to do his job, and can be manipulated to give root access?

There have also been many bugs that allowed service denial attacks. For

example, Multics had a global limit on the number of ﬁles that could be open at

once, but no local limits. So a user could exhaust this limit and lock the system

so that not even the administrator could log on [774]. And until the late 1990’s,

most implementations of the Internet protocols allocated a ﬁxed amount of

buffer space to process the SYN packets with which TCP/IP connections are

initiated. The result was SYN ﬂooding attacks: by sending a large number of

SYN packets, an attacker could exhaust the available buffer space and prevent

the machine accepting any new connections. This is now ﬁxed using syncookies,

which I’ll discuss in Part II.

The most recently discovered family of attacks of this kind are on system

call wrappers. These are software products that modify software behaviour

by intercepting the system calls it makes and performing some ﬁltering or

manipulation. Some wrapper products do virtualization; others provide secu-

rity extensions to operating systems. However Robert Watson has discovered

that such products may have synchronization bugs and race conditions that

allow an attacker to become root [1325]. (I’ll describe these in more detail in

section 18.3.) The proliferation of concurrency mechanisms everywhere, with

multiprocessor machines suddenly becoming the norm after many years in

which they were a research curiosity, may lead to race conditions being the

next big family of attacks.

4.4.3 User Interface Failures

One of the earliest attacks to be devised was the Trojan Horse, a program that

the administrator is invited to run and which will do some harm if he does

so. People would write games which checked occasionally whether the player

122 Chapter 4 ■Access Control

was the system administrator, and if so would create another administrator

account with a known password.

Another trick is to write a program that has the same name as a commonly

used system utility, such as the ls command which lists all the ﬁles in a Unix

directory, and design it to abuse the administrator privilege (if any) before

invoking the genuine utility. The next step is to complain to the administrator

that something is wrong with this directory. When the administrator enters

the directory and types ls to see what’s there, the damage is done. The ﬁx

is simple: an administrator’s ‘PATH’ variable (the list of directories which

will be searched for a suitably named program when a command is invoked)

should not contain ‘.’ (the symbol for the current directory). Recent Unix

versions are shipped with this as a default; but it’s still an unnecessary trap

for the unwary.

Perhaps the most serious example of user interface failure, in terms of

the number of systems at risk, is in Windows. I refer to the fact that, until

Vista came along, a user needed to be the system administrator to install

anything2. In theory this might have helped a bank preventing its branch

staff from running games on their PCs at lunchtime and picking up viruses.

But most environments are much less controlled, and many people need

to be able to install software to get their work done. So millions of people

have administrator privileges who shouldn’t need them, and are vulnerable

to attacks in which malicious code simply pops up a box telling them to

do something. Thank goodness Vista is moving away from this, but UAC

provides no protection where applications such as web servers must run as

root, are visible to the outside world, and contain software bugs that enable

them to be taken over.

Another example, which might be argued is an interface failure, comes

from the use of active content of various kinds. These can be a menace

because users have no intuitively clear way of controlling them. Javascript and

ActiveX in web pages, macros in Ofﬁce documents and executables in email

attachments have all been used to launch serious attacks. Even Java, for all

its supposed security, has suffered a number of attacks that exploited careless

implementations [360]. However, many people (and many companies) are

unwilling to forego the bells and whistles which active content can provide,

and we saw in Chapter 2 how the marketing folks usually beat the security

folks (even in applications like banking).

4.4.4 Why So Many Things Go Wrong

We’ve already mentioned the basic problem faced by operating system security

designers: their products are huge and therefore buggy, and are tested by large

2In theory a member of the Power Users Group in XP could but that made little difference.

4.4 What Goes Wrong 123

numbers of users in parallel, some of whom will publicize their discoveries

rather than reporting them to the vendor. Even if all bugs were reported

responsibly, this wouldn’t make much difference; almost all of the widely

exploited vulnerabilities over the last few years had already been patched.

(Indeed, Microsoft’s ‘Patch Tuesday’ each month is followed by ‘Exploit

Wednesday’ as the malware writers reverse the new vulnerabilities and attack

them before everyone’s patched them.)

There are other structural problems too. One of the more serious causes of

failure is kernel bloat. Under Unix, all device drivers, ﬁlesystems etc. must be

in the kernel. Until Vista, the Windows kernel used to contain drivers for a

large number of smartcards, card readers and the like, many of which were

written by the equipment vendors. So large quantities of code were trusted,

in that they are put inside the security perimeter. Some other systems, such as

MVS, introduced mechanisms that decrease the level of trust needed by many

utilities. However the means to do this in the most common operating systems

are few and relatively nonstandard.

Even more seriously, most application developers make their programs

run as root. The reasons for this are economic rather than technical, and are

easy enough to understand. A company trying to build market share for a

platform, such as an operating system, must appeal to its complementers — its

application developers — as well as to its users. It is easier for developers

if programs can run as root, so early Windows products allowed just that.

Once the vendor has a dominant position, the business logic is to increase the

security, and also to customise it so as to lock in both application developers

and users more tightly. This is now happening with Windows Vista as the

access control mechanisms become ever more complex, and different from

Linux and OS/X. A similar pattern, or too little security in the early stages of a

platform lifecycle and too much (of the wrong kind) later, has been observed

in other platforms from mainframes to mobile phones.

Making many applications and utilities run as root has repeatedly intro-

duced horrible vulnerabilities where more limited privilege could have been

used with only a modicum of thought and a minor redesign. There are many

systems such as lpr/lpd — the Unix lineprinter subsystem — which does not

need to run as root but does anyway on most systems. This has also been a

source of security failures in the past (e.g., getting the printer to spool to the

password ﬁle).

Some applications need a certain amount of privilege. For example, mail

delivery agents must be able to deal with user mailboxes. But while a prudent

designer would restrict this privilege to a small part of the application, most

agents are written so that the whole program needs to run as root. The classic

example is sendmail, which has a long history of serious security holes; but

many other MTAs also have problems. The general effect is that a bug which

124 Chapter 4 ■Access Control

ought to compromise only one person’s mail may end up giving root privilege

to an outside attacker.

So we’re going to have some interesting times as developers come to grips

with UAC. The precedents are not all encouraging. Some programmers histori-

cally avoided the difﬁculty of getting non-root software installed and working

securely by simply leaving important shared data structures and resources

accessible to all users. Many old systems stored mail in a ﬁle per user in

a world-writeable directory, which makes mail forgery easy. The Unix ﬁle

utmp — the list of users logged in — was frequently used for security checking

of various kinds, but is also frequently world-writeable! This should have

been built as a service rather than a ﬁle — but ﬁxing problems like these once

the initial design decisions have been made can be hard. I expect to see all

the old problems of 1970s multiuser systems come back again, as the complex-

ity of using the Vista mechanisms properly just defeats many programmers

who aren’t security specialists and are just desparate to get something sort of

working so they can end the assignment, collect their bonus and move on.

4.4.5 Remedies

Some classes of vulnerability can be ﬁxed using automatic tools. Stack over-

writing attacks, for example, are largely due to the lack of proper bounds

checking in C (the language most operating systems are written in). There

are various tools (including free tools) available for checking C programs for

potential problems, and there is even a compiler patch called StackGuard

which puts a canary next to the return address on the stack. This can be a

random 32-bit value chosen when the program is started, and checked when a

function is torn down. If the stack has been overwritten meanwhile, then with

high probability the canary will change [329]. The availability of these tools,

and training initiatives such as Microsoft’s, have slowly reduced the number

of stack overﬂow errors. However, attack tools also improve, and attackers are

now ﬁnding bugs such as format string vulnerabilities and integer overﬂows

to which no-one paid much attention in the 1990s.

In general, much more effort needs to be put into design, coding and testing.

Architecture matters; having clean interfaces that evolve in a controlled way,

under the eagle eye of someone experienced who has a long-term stake in the

security of the product, can make a huge difference. (I’ll discuss this at greater

length in Part III.) Programs should only have as much privilege as they need:

the principle of least privilege [1102]. Software should also be designed so that

the default conﬁguration, and in general, the easiest way of doing something,

should be safe. Sound architecture is critical in achieving safe defaults and

using least privilege. However, many systems are shipped with dangerous

defaults and messy code that potentially exposes all sorts of interfaces to

attacks like SQL injection that just shouldn’t happen.

4.4 What Goes Wrong 125

4.4.6 Environmental Creep

I have pointed out repeatedly that many security failures result from environ-

mental change undermining a security model. Mechanisms that were adequate

in a restricted environment often fail in a more general one.

Access control mechanisms are no exception. Unix, for example, was origi-

nally designed as a ‘single user Multics’ (hence the name). It then became an

operating system to be used by a number of skilled and trustworthy people

in a laboratory who were sharing a single machine. In this environment the

function of the security mechanisms is mostly to contain mistakes; to prevent

one user’s typing errors or program crashes from deleting or overwriting

another user’s ﬁles. The original security mechanisms were quite adequate for

this purpose.

But Unix security became a classic ‘success disaster’. Unix was repeatedly

extended without proper consideration being given to how the protection

mechanisms also needed to be extended. The Berkeley versions assumed an

extension from a single machine to a network of machines that were all on

one LAN and all under one management. Mechanisms such as rhosts were

based on a tuple (username,hostname) rather than just a username, and saw the

beginning of the transfer of trust.

The Internet mechanisms (telnet, ftp, DNS, SMTP), which grew out of

Arpanet in the 1970’s, were written for mainframes on what was originally

a secure WAN. Mainframes were autonomous, the network was outside the

security protocols, and there was no transfer of authorization. Thus remote

authentication, which the Berkeley model was starting to make prudent, was

simply not supported. The Sun contributions (NFS, NIS, RPC etc.) were based

on a workstation model of the universe, with a multiple LAN environment

with distributed management but still usually in a single organisation. (A

proper tutorial on topics such as DNS and NFS is beyond the scope of this

book, but there is some more detailed background material in the section on

Vulnerabilities in Network Protocols in Chapter 21.)

Mixing all these different models of computation together has resulted in

chaos. Some of their initial assumptions still apply partially, but none of them

apply globally any more. The Internet now has hundreds of millions of PCs,

millions of LANs, thousands of interconnected WANs, and managements

which are not just independent but may be in conﬂict (including nation

states and substate groups that are at war with each other). Many PCs have

no management at all, and there’s a growing number of network-connected

Windows and Linux boxes in the form of fax machines, routers and other

embedded products that don’t ever get patched.

Users, instead of being trustworthy but occasionally incompetent, are now

largely incompetent — but some are both competent and hostile. Code used

to be simply buggy — but now there is a signiﬁcant amount of malicious

126 Chapter 4 ■Access Control

code out there. Attacks on communications networks used to be the purview

of national intelligence agencies — now they can be done by script kiddies,

relatively unskilled people who have downloaded attack tools from the net

and launched them without any real idea of how they work.

So Unix and Internet security gives us yet another example of a system

that started out reasonably well designed but which was undermined by a

changing environment.

Windows Vista and its predecessors in the NT product series have more

extensive protection mechanisms than Unix, but have been around for much

less time. The same goes for database products such as Oracle. Realistically,

all we can say is that the jury is still out.

4.5 Summary

Access control mechanisms operate at a number of levels in a system, from

applications down through middleware to the operating system and the

hardware. Higher level mechanisms can be more expressive, but also tend to

be more vulnerable to attack for a variety of reasons ranging from intrinsic

complexity to implementer skill levels. Most attacks involve the opportunistic

exploitation of bugs, and software products that are very large, very widely

used, or both (as with operating systems and databases) are particularly likely

to have security bugs found and publicized. Systems at all levels are also

vulnerable to environmental changes which undermine the assumptions used

in their design.

The main function of access control is to limit the damage that can be

done by particular groups, users, and programs whether through error or

malice. The most important ﬁelded examples are Unix and Windows, which

are similar in many respects, though Windows is more expressive. Database

products are often more expressive still (and thus even harder to implement

securely.) Access control is also an important part of the design of special

purpose hardware such as smartcards and other encryption devices. New

techniques are being developed to push back on the number of implementation

errors, such as stack overﬂow attacks; but new attacks are being found

continually, and the overall dependability of large software systems improves

only slowly.

The general concepts of access control from read, write and execute permis-

sions to groups and roles will crop up again and again. In some distributed

systems, they may not be immediately obvious as the underlying mechanisms

can be quite different. An example comes from public key infrastructures,

which are a reimplementation of an old access control concept, the capability.

However, the basic mechanisms (and their problems) are pervasive.

Further Reading

The best textbook to go to for a more detailed introduction to access control

issues is Dieter Gollmann’s ‘Computer Security’ [537]. A technical report from

Carl Landwehr gives a useful reference to many of the ﬂaws found in

operating systems over the last 30 years or so [774]. One of the earliest

reports on the subject (and indeed on computer security in general) is by

Willis Ware [1319]. One of the most inﬂuential early papers is by Jerry Saltzer

and Mike Schroeder [1102], while Butler Lampson’s inﬂuential paper on the

conﬁnement problem is at [768].

The classic description of Unix security is in the paper by Fred Grampp and

Robert Morris [550]. The most comprehensive textbook on this subject is Sim-

son Garﬁnkel and Eugene Spafford’s Practical Unix and Internet Security [517],

while the classic on the Internet side of things is Bill Cheswick and Steve

128 Chapter 4 ■Access Control

Bellovin’s Firewalls and Internet Security [157], with many examples of network

attacks on Unix systems.

The protection mechanisms of Windows are described brieﬂy in Gollmann.

For more detail, see the Microsoft online documentation; no doubt a number

of textbooks on Vista will appear soon. There is a history of microprocessor

architectures at [128], and a reference book for Java security written by its

architect Li Gong [539].

The ﬁeld of software security is fast moving; the attacks that are catching

the headlines change signiﬁcantly (at least in their details) from one year to the

next. The best recent book I’ve read is Gary McGraw’s [858]. But to keep up,

you should not just read textbooks, but follow the latest notices from CERT and

mailing lists such as bugtraq and books about the dark side such as Markus

Jakobsson and Zulﬁkar Ramzan’s [660].

CHAPTER

Cryptography

ZHQM ZMGM ZMFM

— G Julius Caesar

KXJEY UREBE ZWEHE WRYTU HEYFS KREHE GOYFI WTTTU OLKSY CAJPO BOTEI

ZONTX BYBWT GONEY CUZWR GDSON SXBOU YWRHE BAAHY USEDQ

— John F Kennedy

5.1 Introduction

Cryptography is where security engineering meets mathematics. It provides

us with the tools that underlie most modern security protocols. It is probably

the key enabling technology for protecting distributed systems, yet it is

surprisingly hard to do right. As we’ve already seen in Chapter 3, ‘Protocols’,

cryptography has often been used to protect the wrong things, or used to

protect them in the wrong way. We’ll see plenty more examples when we start

looking in detail at real applications.

Unfortunately, the computer security and cryptology communities have

drifted apart over the last 25 years. Security people don’t always understand

the available crypto tools, and crypto people don’t always understand the

real-world problems. There are a number of reasons for this, such as different

professional backgrounds (computer science versus mathematics) and differ-

ent research funding (governments have tried to promote computer security

research while suppressing cryptography). It reminds me of a story told by

a medical friend. While she was young, she worked for a few years in a

country where, for economic reasons, they’d shortened their medical degrees

and concentrated on producing specialists as quickly as possible. One day,

129

130 Chapter 5 ■Cryptography

a patient who’d had both kidneys removed and was awaiting a transplant

needed her dialysis shunt redone. The surgeon sent the patient back from the

theater on the grounds that there was no urinalysis on ﬁle. It just didn’t occur

to him that a patient with no kidneys couldn’t produce any urine.

Just as a doctor needs to understand physiology as well as surgery, so

a security engineer needs to be familiar with cryptology as well as computer

security (and much else). This chapter is aimed at people without any training

in cryptology; cryptologists will ﬁnd little in it that they don’t already know.

As I only have a few dozen pages, and a proper exposition of modern cryp-

tography would run into thousands, I won’t go into much of the mathematics

(there are lots of books that do that; see the end of the chapter for further

reading). I’ll just explain the basic intuitions and constructions that seem

to cause the most confusion. If you have to use cryptography in anything

resembling a novel way, then I strongly recommend that you read a lot more

about it — and talk to some real experts. The security engineer Paul Kocher

remarked, at a keynote speech at Crypto 2007, that you could expect to break

any crypto product designed by ‘any company that doesn’t employ someone

in this room’. There is a fair bit of truth in that.

Computer security people often ask for non-mathematical deﬁnitions of

cryptographic terms. The basic terminology is that cryptography refers to

the science and art of designing ciphers; cryptanalysis to the science and

art of breaking them; while cryptology, often shortened to just crypto, is

the study of both. The input to an encryption process is commonly called the

plaintext, and the output the ciphertext. Thereafter, things get somewhat more

complicated. There are a number of cryptographic primitives — basic building

blocks, such as block ciphers,stream ciphers,andhash functions. Block ciphers

may either have one key for both encryption and decryption, in which case

they’re called shared-key (also secret-key or symmetric), or have separate keys

for encryption and decryption, in which case they’re called public-key or

asymmetric.Adigital signature scheme is a special type of asymmetric crypto

primitive.

In the rest of this chapter, I will ﬁrst give some simple historical examples to

illustrate the basic concepts. I’ll then try to ﬁne-tune deﬁnitions by introducing

the random oracle model, which many cryptologists use. Finally, I’ll show how

some of the more important cryptographic algorithms actually work, and

how they can be used to protect data.

5.2 Historical Background

Suetonius tells us that Julius Caesar enciphered his dispatches by writing

‘D’ for ‘A’, ‘E’ for ‘B’ and so on [1232]. When Augustus Caesar ascended the

throne, he changed the imperial cipher system so that ‘C’ was now written for

5.2 Historical Background 131

‘A’, ‘D’ for ‘B’ etcetera. In modern terminology, we would say that he changed

the key from ‘D’ to ‘C’. Remarkably, a similar code was used by Bernardo

Provenzano, allegedly the capo di tutti capi of the Sicilian maﬁa, who wrote ‘4’

for ‘a’, ‘5’ for ‘b’ and so on. This led directly to his capture by the Italian police

in 2006 after they intercepted and deciphered some of his messages [1034].

The Arabs generalised this idea to the monoalphabetic substitution,inwhich

a keyword is used to permute the cipher alphabet. We will write the plaintext

in lower case letters, and the ciphertext in upper case, as shown in Figure 5.1:

abcdefghijklmnopqrstuvwxyz

SECURITYABDFGHJKLMNOPQVWXZ

Figure 5.1: Monoalphabetic substitution cipher

OYAN RWSGKFR AN AH RHTFANY MSOYRM OYSH SMSEAC NCMAKO; but breaking

ciphers of this kind is a straightforward pencil and paper puzzle, which you

may have done in primary school. The trick is that some letters, and combi-

nations of letters, are much more common than others; in English the most

common letters are e,t,a,i,o,n,s,h,r,d,l,u in that order. Artiﬁcial intelligence

researchers have shown some interest in writing programs to solve monoal-

phabetic substitutions. Using letter and digram (letter pair) frequencies alone,

they typically succeed with about 600 letters of ciphertext, while smarter

strategies such as guessing probable words can cut this to about 150 letters. A

human cryptanalyst will usually require much less.

There are basically two ways to make a stronger cipher — the stream cipher

and the block cipher. In the former, you make the encryption rule depend on

a plaintext symbol’s position in the stream of plaintext symbols, while in the

latter you encrypt several plaintext symbols at once in a block. Let’s look at

early examples.

5.2.1 An Early Stream Cipher — The Vigen `

ere

This early stream cipher is commonly ascribed to the Frenchman Blaise de

Vigen`

ere, a diplomat who served King Charles IX. It works by adding a key

repeatedly into the plaintext using the convention that ‘A’ =0, ‘B’ =1, ... ,

‘Z’ =25, and addition is carried out modulo 26 — that is, if the result is greater

than 25, we subtract as many multiples of 26 as are needed to bring is into the

range [0, ..., 25], that is, [A, ..., Z]. Mathematicians write this as

C=P+Kmod 26

So, for example, when we add P (15) to U (20) we get 35, which we reduce to

9 by subtracting 26.9 corresponds to J, so the encryption of P under the key U

(and of U under the key P) is J. In this notation, Julius Caesar’s system used a

132 Chapter 5 ■Cryptography

ﬁxed key K=D1, while Augustus Caesar’s used K=C and Vigen`

ere used a

repeating key, also known as a running key. Various means were developed

to do this addition quickly, including printed tables and, for ﬁeld use, cipher

wheels. Whatever the implementation technology, the encryption using a

repeated keyword for the key would look as shown in Figure 5.2:

Plain tobeornottobethatisthequestion

Key runrunrunrunrunrunrunrunrunrun

Cipher KIOVIEEIGKIOVNURNVJNUVKHVMGZIA

Figure 5.2: Vigen`

ere (polyalphabetic substitution cipher)

A number of people appear to have worked out how to solve polyalphabetic

ciphers, from the womaniser Giacomo Casanova to the computing pioneer

Charles Babbage. However the ﬁrst published solution was in 1863 by Friedrich

Kasiski, a Prussian infantry ofﬁcer [695]. He noticed that given a long enough

piece of ciphertext, repeated patterns will appear at multiples of the keyword

length.

In Figure 5.2, for example, we see ‘KIOV’ repeated after nine letters, and ‘NU’

after six. Since three divides both six and nine, we might guess a keyword

of three letters. It follows that ciphertext letters one, four, seven and so on

all enciphered under the same keyletter; so we can use frequency analysis

techniques to guess the most likely values of this letter, and then repeat the

process for the second and third letters of the key.

5.2.2 The One-Time Pad

One way to make a stream cipher of this type proof against attacks is for the

key sequence to be as long as the plaintext, and to never repeat. This was pro-

posed by Gilbert Vernam during World War 1 [676]; its effect is that given any

ciphertext, and any plaintext of the same length, there is a key which decrypts

the ciphertext to the plaintext. Regardless of the amount of computation that

opponents can do, they are none the wiser, as all possible plaintexts are just

as likely. This system is known as the one-time pad. Leo Marks’ engaging book

on cryptography in the Special Operations Executive in World War 2 [836]

relates how one-time key material was printed on silk, which agents could

conceal inside their clothing; whenever a key had been used it was torn off

and burnt.

An example should explain all this. Suppose you had intercepted a message

from a wartime German agent which you knew started with ‘Heil Hitler’,

and the ﬁrst ten letters of ciphertext were DGTYI BWPJA. This means that

1modulo 23, as the alphabet Caesar used wrote U as V, J as I, and had no W.

5.2 Historical Background 133

the ﬁrst ten letters of the one-time pad were wclnb tdefj,asshownin

Figure 5.3:

Plain heilhitler

Key wclnbtdefj

Cipher DGTYIBWPJA

Figure 5.3: A spy’s message

But once he’s burnt the piece of silk with his key material, the spy can claim

that he’s actually a member of the anti-Nazi underground resistance, and the

message actually said ‘Hang Hitler’. This is quite possible, as the key material

could just as easily have been wggsb tdefj, as shown in Figure 5.4:

Cipher DGTYIBWPJA

Key wggsbtdefj

Plain hanghitler

Figure 5.4: What the spy claimed he said

Now we rarely get anything for nothing in cryptology, and the price of the

perfect secrecy of the one-time pad is that it fails completely to protect message

integrity. Suppose for example that you wanted to get this spy into trouble,

you could change the ciphertext to DCYTI BWPJA (Figure 5.5):

Cipher DCYTIBWPJA

Key wclnbtdefj

Plain hanghitler

Figure 5.5: Manipulating the message to entrap the spy

During the Second World War, Claude Shannon proved that a cipher has

perfect secrecy if and only if there are as many possible keys as possible

plaintexts, and every key is equally likely; so the one-time pad is the only kind

of system which offers perfect secrecy [1157, 1158].

The one-time pad is still used for some diplomatic and intelligence trafﬁc,

but it consumes as much key material as there is trafﬁc and this is too expensive

for most applications. It’s more common for stream ciphers to use a suitable

pseudorandom number generator to expand a short key into a long keystream.

The data is then encrypted by exclusive-or’ing the keystream, one bit at a time,

with the data. It’s not enough for the keystream to appear ‘‘random’’ in

the sense of passing the standard statistical randomness tests: it must also

have the property that an opponent who gets his hands on even quite a lot of

134 Chapter 5 ■Cryptography

keystream bits should not be able to predict any more of them. I’ll formalise

this more tightly in the next section.

Stream ciphers are commonly used nowadays in hardware applications

where the number of gates has to be minimised to save power. We’ll look

at some actual designs in later chapters, including the A5 algorithm used

to encipher GSM mobile phone trafﬁc (in the chapter on ‘Telecom System

Security’), and the shift register systems used in pay-per-view TV and DVD

CSS (in the chapter on ‘Copyright and Privacy Protection’). However, block

ciphers are more suited for many applications where encryption is done in

software, so let’s look at them next.

5.2.3 An Early Block Cipher — Playfair

One of the best-known early block ciphers is the Playfair system. It was

invented in 1854 by Sir Charles Wheatstone, a telegraph pioneer who

also invented the concertina and the Wheatstone bridge. The reason it’s

not called the Wheatstone cipher is that he demonstrated it to Baron Playfair,

a politician; Playfair in turn demonstrated it to Prince Albert and to Viscount

Palmerston (later Prime Minister), on a napkin after dinner.

This cipher uses a 5 by 5 grid, in which we place the alphabet, permuted by

the key word, and omitting the letter ‘J’ (see Figure 5.6):

P A L M E

R S T O N

B C D F G

H I K Q U

V W X Y Z

Figure 5.6: The Playfair enciphering tableau

The plaintext is ﬁrst conditioned by replacing ‘J’ with ‘I’ wherever it occurs,

then dividing it into letter pairs, preventing double letters occurring in a pair by

separating them with an ‘x’, and ﬁnally adding a ‘z’ if necessary to complete the

last letter pair. The example Playfair wrote on his napkin was ‘Lord Granville’s

letter’ which becomes ‘lo rd gr an vi lx le sl et te rz’.

It is then enciphered two letters at a time using the following rules:

if the two letters are in the same row or column, they are replaced by the

succeeding letters. For example, ‘am’ enciphers to ‘LE’

otherwise the two letters stand at two of the corners of a rectangle in

the table, and we replace them with the letters at the other two corners of

this rectangle. For example, ‘lo’ enciphers to ‘MT’.

5.2 Historical Background 135

We can now encipher our specimen text as follows:

Plain lo rd gr an vi lx le sl et te rz

Cipher MT TB BN ES WH TL MP TA LN NL NV

Figure 5.7: Example of Playfair enciphering

Variants of this cipher were used by the British army as a ﬁeld cipher in

World War 1, and by the Americans and Germans in World War 2. It’s a

substantial improvement on Vigen`

ere as the statistics which an analyst can

collect are of digraphs (letter pairs) rather than single letters, so the distribution

is much ﬂatter and more ciphertext is needed for an attack.

Again, it’s not enough for the output of a block cipher to just look intuitively

‘random’. Playfair ciphertexts look random; but they have the property that if

you change a single letter of a plaintext pair, then often only a single letter of

the ciphertext will change. Thus using the key in Figure 5.7, rd enciphers to

TB while rf enciphers to OB and rg enciphers to NB. One consequence is that

given enough ciphertext, or a few probable words, the table (or an equivalent

one) can be reconstructed [512]. So we will want the effects of small changes

in a block cipher’s input to diffuse completely through its output: changing

one input bit should, on average, cause half of the output bits to change. We’ll

tighten these ideas up in the next section.

The security of a block cipher can be greatly improved by choosing a longer

block length than two characters. For example, the Data Encryption Standard

(DES), which is widely used in banking, has a block length of 64 bits, which

equates to eight ascii characters and the Advanced Encryption Standard (AES),

which is replacing it in many applications, has a block length of twice this. I

discuss the internal details of DES and AES below; for the time being, I’ll just

remark that an eight byte or sixteen byte block size is not enough of itself.

For example, if a bank account number always appears at the same place

in a transaction, then it’s likely to produce the same ciphertext every time a

transaction involving it is encrypted with the same key.

This might allow an opponent to cut and paste parts of two different cipher-

texts in order to produce a seemingly genuine but unauthorized transaction.

Suppose a bad man worked for a bank’s phone company, and could intercept

their trafﬁc. If he monitored an enciphered transaction that he knew said ‘‘Pay

IBM $10,000,000’’ he might wire $1,000 to his brother causing the bank com-

puter to insert another transaction saying ‘‘Pay John Smith $1,000’’, intercept

this instruction, and make up a false instruction from the two ciphertexts that

decrypted as ‘‘Pay John Smith $10,000,000’’. So unless the cipher block is as

large as the message, the ciphertext will contain more than one block and we

will usually need some way of binding the blocks together.

136 Chapter 5 ■Cryptography

5.2.4 One-Way Functions

The third classical type of cipher is the one-way function. This evolved to

protect the integrity and authenticity of messages, which as we’ve seen

is not protected at all by many simple ciphers where it is often easy to

manipulate the ciphertext in such a way as to cause a predictable change in the

plaintext.

After the invention of the telegraph in the mid-19th century, banks rapidly

became its main users and developed systems for transferring money elec-

tronically. Of course, it isn’t the money itself which is ‘wired’ but a payment

instruction, such as:

‘To Lombard Bank, London. Please pay from our account with you no. 1234567890

the sum of £1000 to John Smith of 456 Chesterton Road, who has an account with

HSBC Bank Cambridge no. 301234 4567890123, and notify him that this was

for ‘‘wedding present from Doreen Smith’’. From First Cowboy Bank of Santa

Barbara, CA, USA. Charges to be paid by us.’

Since telegraph messages were relayed from one ofﬁce to another by human

operators, it was possible for an operator to manipulate a payment message.

In the nineteenth century, banks, telegraph companies and shipping com-

panies developed code books that could not only protect transactions but also

shorten them — which was very important given the costs of international

telegrams at the time. A code book was essentially a block cipher which

mapped words or phrases to ﬁxed-length groups of letters or numbers. So

‘Please pay from our account with you no.’ might become ‘AFVCT’. A compet-

ing technology from the 1920s was rotor machines, mechanical cipher devices

which produce a very long sequence of pseudorandom numbers and combine

them with plaintext to get ciphertext; these were independently invented by

a number of people, many of whom dreamed of making a fortune selling

them to the banking industry. Banks weren’t in general interested, but rotor

machines became the main high-level ciphers used by the combatants in

World War 2.

The banks realised that neither mechanical stream ciphers nor code books

protect message authenticity. If, for example, the codeword for ‘1000’ is

‘mauve’ and for ‘1,000,000’ is ‘magenta’, then the crooked telegraph clerk who

can compare the coded trafﬁc with known transactions should be able to ﬁgure

this out and substitute one for the other.

The critical innovation, for the banks’ purposes, was to use a code book

but to make the coding one-way by adding the code groups together into

a number called a test key. (Modern cryptographers would describe it as a

hash value or message authentication code, terms I’ll deﬁne more carefully

later.)

5.2 Historical Background 137

Here is a simple example. Suppose the bank has a code book with a table of

numbers corresponding to payment amounts as in Figure 5.8:

0123456789

x 1000 14 22 40 87 69 93 71 35 06 58

x 10,000 73 38 15 46 91 82 00 29 64 57

x 100,000 95 70 09 54 82 63 21 47 36 18

x 1,000,000 53 77 66 29 40 12 31 05 87 94

Figure 5.8: A simple test key system

Now in order to authenticate a transaction for £376,514 we add together 53

(no millions), 54 (300,000), 29 (70,000) and 71 (6,000). (It’s common to ignore

the less signiﬁcant digits of the amount.) This gives us a test key of 207.

Most real systems were more complex than this; they usually had tables

for currency codes, dates and even recipient account numbers. In the better

systems, the code groups were four digits long rather than two, and in order

to make it harder for an attacker to reconstruct the tables, the test keys were

compressed: a key of ‘7549’ might become ‘23’ by adding the ﬁrst and second

digits, and the third and fourth digits, and ignoring the carry.

This made such test key systems into one-way functions in that although given

knowledge of the key it was possible to compute a test from a message, it was

not possible to reverse the process and recover a message from a test — the

test just did not contain enough information. Indeed, one-way functions had

been around since at least the seventeenth century. The scientist Robert Hooke

published in 1678 the sorted anagram ‘ceiiinosssttuu’ and revealed two years

later that it was derived from ‘Ut tensio sic uis’ — ‘the force varies as the

tension’, or what we now call Hooke’s law for a spring. (The goal was to

establish priority for the idea while giving him time to continue developing it.)

Test keys are not strong by the standards of modern cryptography. Given

somewhere between a few dozen and a few hundred tested messages, depend-

ing on the design details, a patient analyst could reconstruct enough of the

tables to forge a transaction. With a few carefully chosen messages inserted

into the banking system by an accomplice, it’s even easier still. But the banks

got away with it: test keys worked ﬁne from the late nineteenth century

through the 1980’s. In several years working as a bank security consultant, and

listening to elderly bank auditors’ tales over lunch, I only ever heard of two

cases of fraud that exploited it: one external attempt involving cryptanalysis,

which failed because the attacker didn’t understand bank procedures, and one

successful but small fraud involving a crooked staff member. I’ll discuss the

systems which replaced test keys, and the whole issue of how to tie cryp-

tographic authentication mechanisms to procedural protection such as dual

138 Chapter 5 ■Cryptography

control, in the chapter on ‘Banking and Bookkeeping’. For the meantime, test

keys are the classic example of a one-way function used for authentication.

Later examples included functions for applications discussed in the previous

chapters, such as storing passwords in a one-way encrypted password ﬁle,

and computing a response from a challenge in an authentication protocol.

5.2.5 Asymmetric Primitives

Finally, some modern cryptosystems are asymmetric, in that different keys

are used for encryption and decryption. So, for example, many people publish

on their web page a public key with which people can encrypt messages to

send to them; the owner of the web page can then decrypt them using the

corresponding private key.

There are some pre-computer examples of this too; perhaps the best is the

postal service. You can send me a private message just as simply by addressing

it to me and dropping it into a post box. Once that’s done, I’m the only person

who’ll be able to read it. There are of course many things that can go wrong.

You might get the wrong address for me (whether by error or as a result of

deception); the police might get a warrant to open my mail; the letter might be

stolen by a dishonest postman; a fraudster might redirect my mail without my

knowledge; or a thief might steal the letter from my mailbox. There are similar

things that can go wrong with public key cryptography. False public keys can

be inserted into the system, computers can be hacked, people can be coerced

and so on. We’ll look at these problems in more detail in later chapters.

Another asymmetric application of cryptography is the digital signature. The

idea here is that I can sign a message using a private signature key and then

anybody can check this using my public signature veriﬁcation key. Again, there

are pre-computer analogues in the form of manuscript signatures and seals;

and again, there is a remarkably similar litany of things that can go wrong,

both with the old way of doing things and with the new.

5.3 The Random Oracle Model

Before delving into the detailed design of modern ciphers, I want to take a few

pages to reﬁne the deﬁnitions of the various types of cipher. (Readers who

are phobic about theoretical computer science should skip this section at a

ﬁrst pass; I’ve included it because a basic grasp of the terminology of random

oracles is needed to decipher many recent research papers on cryptography.)

The random oracle model seeks to formalize the idea that a cipher is ‘good’ if,

when viewed in a suitable way, it is indistinguishable from a random function

of a certain type. I will call a cryptographic primitive pseudorandom if it passes

all the statistical and other tests which a random function of the appropriate

5.3 The Random Oracle Model 139

type would pass, in whatever model of computation we are using. Of course,

the cryptographic primitive will actually be an algorithm, implemented as an

array of gates in hardware or a program in software; but the outputs should

‘look random’ in that they’re indistinguishable from a suitable random oracle

given the type and the number of tests that our computation model permits.

In this way, we can hope to separate the problem of designing ciphers from

the problem of using them correctly. Mathematicians who design ciphers can

provide evidence that their cipher is pseudorandom. Quite separately, a

computer scientist who has designed a cryptographic protocol can try to

prove that it is secure on the assumption that the crypto primitives used

to implement it are pseudorandom. The process isn’t infallible, as we saw

with proofs of protocol correctness. Theorems can have bugs, just like pro-

grams; the problem could be idealized wrongly; and the mathematicians

might be using a different model of computation from the computer scien-

tists. In fact, there is a live debate among crypto researchers about whether

formal models and proofs are valuable [724]. But crypto theory can help us

sharpen our understanding of how ciphers behave and how they can safely

be used.

We can visualize a random oracle as an elf sitting in a black box with a

source of physical randomness and some means of storage (see Figure 5.9) —

represented in our picture by the dice and the scroll. The elf will accept inputs

of a certain type, then look in the scroll to see whether this query has ever

been answered before. If so, it will give the answer it ﬁnds there; if not,

it will generate an answer at random by throwing the dice. We’ll further

assume that there is some kind of bandwidth limitation — that the elf will

only answer so many queries every second. This ideal will turn out to be

useful as a way of reﬁning our notions of a stream cipher, a hash function,

a block cipher, a public key encryption algorithm and a digital signature

scheme.

Figure 5.9: The random oracle

140 Chapter 5 ■Cryptography

Finally, we can get a useful simpliﬁcation of our conceptual model by

noting that encryption can be used to protect data across time as well as across

distance. A good example is when we encrypt data before storing it with a

third-party backup service, and may decrypt it later if we have to recover from

a disk crash. In this case, we only need a single encryption/decryption device,

rather than having one at each end of a communications link. For simplicity,

let us assume it is this sort of application we are modelling here. The user takes

a diskette to the cipher machine, types in a key, issues an instruction, and the

data get transformed in the appropriate way. A year later, she comes back to

get the data decrypted and veriﬁed.

We shall now look at this model in more detail for various different

cryptographic primitives.

5.3.1 Random Functions — Hash Functions

The ﬁrst type of random oracle is the random function. A random function

accepts an input string of any length and outputs a random string of ﬁxed

length, say nbits long. So the elf just has a simple list of inputs and outputs,

which grows steadily as it works. (We’ll ignore any effects of the size of the

scroll and assume that all queries are answered in constant time.)

Random functions are our model for one-way functions,alsoknownas

cryptographic hash functions, which have many practical uses. They were ﬁrst

used in computer systems for one-way encryption of passwords in the 1960s

and, as I mentioned in the chapter on security protocols, are used today in

a number of authentication systems. They are used to compute checksums

on ﬁles in forensic applications: presented with a computer seized from a

suspect, you can compute hash values of the ﬁles to identify which ﬁles are

already known (such as system ﬁles) and which are novel (such as user data).

Hash values are also used as a means of checking the integrity of ﬁles, as

they will change if a ﬁle is corrupted. In messaging applications, hashes are

often known as message digests; given a message Mwe can pass it through a

pseudorandom function to get a digest, say h(M), which can stand in for the

message in various applications. One example is digital signature: signature

algorithms tend to be slow if the message is long, so it’s usually convenient to

sign a message digest rather than the message itself.

Another application is timestamping. If we want evidence that we possessed

a given electronic document by a certain date, we might submit it to an online

time-stamping service. However, if the document is still secret — for example

an invention which we plan to patent, and for which we merely want to

establish a priority date — then we might not send the timestamping service

the whole document, but just the message digest.

5.3 The Random Oracle Model 141

5.3.1.1 Properties

The ﬁrst main property of a random function is one-wayness. Given knowledge

of an input xwe can easily compute the hash value h(x), but it is very difﬁcult

given the hash value h(x) to ﬁnd a corresponding preimage x if one is not

already known. (The elf will only pick outputs for given inputs, not the other

way round.) As the output is random, the best an attacker who wants to invert

a random function can do is to keep on feeding in more inputs until he gets

lucky. A pseudorandom function will have the same properties, or they could

be used to distinguish it from a random function, contrary to our deﬁnition. It

follows that a pseudorandom function will also be a one-way function, provided

there are enough possible outputs that the opponent can’t ﬁnd a desired target

output by chance. This means choosing the output to be an n-bit number

where the opponent can’t do anything near 2ncomputations.

A second property of pseudorandom functions is that the output will not

give any information at all about even part of the input. Thus a one-way

encryption of the value xcan be accomplished by concatenating it with a

secret key kand computing h(x,k). If the hash function isn’t random enough,

though, using it for one-way encryption in this manner is asking for trouble.

A topical example comes from the authentication in GSM mobile phones,

where a 16 byte challenge from the base station is concatenated with a 16 byte

secret key known to the phone into a 32 byte number, and passed through

a hash function to give an 11 byte output [226]. The idea is that the phone

company also knows kand can check this computation, while someone who

eavesdrops on the radio link can only get a number of values of the random

challenge xand corresponding output from h(x,k). So the eavesdropper must

not be able to get any information about k, or compute h(y,k)foranew

input y. But the one-way function used by many phone companies isn’t one-

way enough, with the result that an eavesdropper who can pretend to be

a base station and send a phone about 150,000 suitable challenges and get

the responses can compute the key. I’ll discuss this failure in more detail in

section 20.3.2.

A third property of pseudorandom functions with sufﬁciently long outputs

is that it is hard to ﬁnd collisions, that is, different messages M1= M2with

h(M1)=h(M2). Unless the opponent can ﬁnd a shortcut attack (which would

mean the function wasn’t really pseudorandom) then the best way of ﬁnding a

collision is to collect a large set of messages Miand their corresponding hashes

h(Mi), sort the hashes, and look for a match. If the hash function output is an

n-bit number, so that there are 2npossible hash values, then the number of

hashes the enemy will need to compute before he can expect to ﬁnd a match

will be about the square root of this, namely 2n/2hashes. This fact is of huge

importance in security engineering, so let’s look at it more closely.

142 Chapter 5 ■Cryptography

5.3.1.2 The Birthday Theorem

The birthday theorem gets its name from the following problem. A maths

teacher asks a typical class of 30 pupils what they think is the probability that

two of them have the same birthday. Most pupils will intuitively think it’s

unlikely, and the maths teacher then asks the pupils to state their birthdays

one after another. As the result seems unlikely to most people, it’s also known

as the ‘birthday paradox’. The odds of a match exceed 50% once 23 pupils have

been called.

The birthday theorem was ﬁrst invented in the 1930’s to count ﬁsh, which

led to its also being known as capture-recapture statistics [1123]. Suppose there

are Nﬁsh in a lake and you catch mof them, ring them and throw them back,

then when you ﬁrst catch a ﬁsh you’ve ringed already, mshould be ‘about’

the square root of N. The intuitive reason why this holds is that once you have

√Nsamples, each could potentially match any of the others, so the number of

possible matches is about √Nx√Nor N, which is what you need2.

This theorem has many applications for the security engineer. For example,

if we have a biometric system which can authenticate a person’s claim to

identity with a probability of only one in a million that two randomly selected

subjects will be falsely identiﬁed as the same person, this doesn’t mean that

we can use it as a reliable means of identiﬁcation in a university with a user

population of twenty thousand staff and students. This is because there will

be almost two hundred million possible pairs. In fact, you expect to ﬁnd the

ﬁrst collision — the ﬁrst pair of people who can be mistaken for each other by

the system — once you have somewhat over a thousand people enrolled.

There are some applications where collision-search attacks aren’t a problem,

such as in challenge-response protocols where an attacker would have to be

able to ﬁnd the answer to the challenge just issued, and where you can prevent

challenges repeating. (For example, the challenge might be generated by

encrypting a counter.) So in identify-friend-or-foe (IFF) systems, for example,

common equipment has a response length of 48 to 80 bits.

However, there are other applications in which collisions are unacceptable.

In a digital signature application, if it were possible to ﬁnd collisions with

h(M1)=h(M2)butM1= M2, then a Maﬁa owned bookstore’s web site might

get you to sign a message M1saying something like ‘I hereby order a copy

of Rubber Fetish volume 7 for $32.95’ and then present the signature together

with an M2saying something like ‘I hereby mortgage my house for $75,000

and please make the funds payable to Maﬁa Holdings Inc., Bermuda’.

For this reason, hash functions used with digital signature schemes generally

have nlarge enough to make them collision-free, that is, that 2n/2computations

2More precisely, the probability that mﬁsh chosen randomly from Nﬁsh are different is

β=N(N−1) ...(N−m+1)/Nmwhich is asymptotically solved by Nm2/2log(1/β) [708].

5.3 The Random Oracle Model 143

are impractical for an opponent. The two most common are MD5, which has

a 128-bit output and will thus require at most 264 computations to break, and

SHA1 with a 160-bit output and a work factor for the cryptanalyst of at most

280. However, collision search gives at best an upper bound on the strength

of a hash function, and both these particular functions have turned out to be

disappointing, with cryptanalytic attacks that I’ll describe later in section 5.6.2.

Collisions are easy to ﬁnd for MD4 and MD5, while for SHA-1 it takes about

260 computations to ﬁnd a collision — something that a botnet of half a million

machines should be able to do in a few days.

In any case, a pseudorandom function is also often referred to as being

collision free or collision intractable. This doesn’t mean that collisions don’t exist

— they must, as the set of possible inputs is larger than the set of pos-

sible outputs — just that you will never ﬁnd any of them. The (usually

unstated) assumptions are that the output must be long enough, and that the

cryptographic design of the hash function must be sound.

5.3.2 Random Generators — Stream Ciphers

The second basic cryptographic primitive is the random generator,alsoknown

as a keystream generator or stream cipher. This is also a random function, but

unlike in the hash function case it has a short input and a long output. (If we

had a good pseudorandom function whose input and output were a billion

bits long, and we never wanted to handle any objects larger than this, we could

turn it into a hash function by throwing away all but a few hundred bits of the

output, and turn it into a stream cipher by padding all but a few hundred bits

of the input with a constant.) At the conceptual level, however, it’s common to

think of a stream cipher as a random oracle whose input length is ﬁxed while

the output is a very long stream of bits, known as the keystream.

It can be used quite simply to protect the conﬁdentiality of backup data:

we go to the keystream generator, enter a key, get a long ﬁle of random bits,

and exclusive-or it with our plaintext data to get ciphertext, which we then

send to our backup contractor. We can think of the elf generating a random

tape of the required length each time he is presented with a new key as input,

giving it to us and keeping a copy of it on his scroll for reference in case he’s

given the same input again. If we need to recover the data, we go back to

the generator, enter the same key, get the same long ﬁle of random data, and

exclusive-or it with our ciphertext to get our plaintext data back again. Other

people with access to the keystream generator won’t be able to generate the

same keystream unless they know the key.

I mentioned the one-time pad, and Shannon’s result that a cipher has perfect

secrecy if and only if there are as many possible keys as possible plaintexts, and

every key is equally likely. Such security is called unconditional (or statistical)

security as it doesn’t depend either on the computing power available to the

144 Chapter 5 ■Cryptography

opponent, or on there being no future advances in mathematics which provide

a shortcut attack on the cipher.

One-time pad systems are a very close ﬁt for our theoretical model, except

in that they are typically used to secure communications across space rather

than time: there are two communicating parties who have shared a copy of

the randomly-generated keystream in advance. Vernam’s original telegraph

cipher machine used punched paper tape; a modern diplomatic system might

use DVDs, shipped in a tamper-evident container in a diplomatic bag. Various

techniques have been used to do the random generation. Marks describes

how SOE agents’ silken keys were manufactured in Oxford by little old ladies

shufﬂing counters.

One important problem with keystream generators is that we want to pre-

vent the same keystream being used more than once, whether to encrypt more

than one backup tape or to encrypt more than one message sent on a com-

munications channel. During World War 2, the amount of Russian diplomatic

trafﬁc exceeded the quantity of one-time tape they had distributed in advance

to their embassies, so it was reused. This was a serious blunder. If M1+K=C1

and M2+K=C2, then the opponent can combine the two ciphertexts to get

a combination of two messages: C1−C2=M1−M2, and if the messages Mi

have enough redundancy then they can be recovered. Text messages do in

fact contain enough redundancy for much to be recovered, and in the case

of the Russian trafﬁc this led to the Venona project in which the US and UK

decrypted large amounts of wartime Russian trafﬁc afterwards and broke up

a number of Russian spy rings. The saying is: ‘Avoid the two-time tape!’

Exactly the same consideration holds for any stream cipher, and the normal

engineering practice when using an algorithmic keystream generator is to

have a seed as well as a key. Each time the cipher is used, we want it to generate

a different keystream, so the key supplied to the cipher should be different.

So if the long-term key which two users share is K, they may concatenate it

with a seed which is a message number N(or some other nonce) and then

pass it through a hash function to form a working key h(K,N). This working

key is the one actually fed to the cipher machine. The nonce may be a separate

pre-agreed key, or it may be generated at random and sent along with the

ciphertext. However, the details of key management can be quite tricky, and

the designer has to watch out for attacks in which a principal is tricked into

synchronising on the wrong key. In effect, a protocol has to be designed to

ensure that both parties can synchronise on the right working key even in the

presence of an adversary.

5.3.3 Random Permutations — Block Ciphers

The third type of primitive, and the most important in modern commercial

cryptography, is the block cipher, which we model as a random permutation.

5.3 The Random Oracle Model 145

Here, the function is invertible, and the input plaintext and the output

ciphertext are of a ﬁxed size. With Playfair, both input and output are two

characters; with DES, they’re both bit strings of 64 bits. Whatever the number

of symbols and the underlying alphabet, encryption acts on a block of ﬁxed

length. (So if you want to encrypt a shorter input, you have to pad it as with

the ﬁnal ‘z’ in our Playfair example.)

We can visualize block encryption as follows. As before, we have an elf in a

box with dice and a scroll. This has on the left a column of plaintexts and on

the right a column of ciphertexts. When we ask the elf to encrypt a message,

it checks in the left hand column to see if it has a record of it. If not, it uses

the dice to generate a random ciphertext of the appropriate size (and which

doesn’t appear yet in the right hand column of the scroll), and then writes

down the plaintext/ciphertext pair in the scroll. If it does ﬁnd a record, it gives

us the corresponding ciphertext from the right hand column.

When asked to decrypt, the elf does the same, but with the function of

the columns reversed: he takes the input ciphertext, checks it (this time on the

right hand scroll) and if he ﬁnds it he gives the message with which it was

previously associated. If not, he generates a message at random (which does

not already appear in the left column) and notes it down.

Ablock cipher is a keyed family of pseudorandom permutations. For each

key, we have a single permutation which is independent of all the others. We

can think of each key as corresponding to a different scroll. The intuitive idea

is that a cipher machine should output the ciphertext given the plaintext and

the key, and output the plaintext given the ciphertext and the key, but given

only the plaintext and the ciphertext it should output nothing.

We will write a block cipher using the notation established for encryption

in the chapter on protocols:

C={M}K

The random permutation model also allows us to deﬁne different types of

attack on block ciphers. In a known plaintext attack, the opponent is just given a

number of randomly chosen inputs and outputs from the oracle corresponding

to a target key. In a chosen plaintext attack, the opponent is allowed to put a

certain number of plaintext queries and get the corresponding ciphertexts. In

achosen ciphertext attack he gets to make a number of ciphertext queries. In a

chosen plaintext/ciphertext attack he is allowed to make queries of either type.

Finally, in a related key attack he can make queries that will be answered using

keys related to the target key K,suchasK+1andK+2.

In each case, the objective of the attacker may be either to deduce the answer

to a query he hasn’t already made (a forgery attack), or to recover the key

(unsurprisingly known as a key recovery attack).

This precision about attacks is important. When someone discovers a vul-

nerability in a cryptographic primitive, it may or may not be relevant to your

146 Chapter 5 ■Cryptography

application. Often it won’t be, but will have been hyped by the media — so

you will need to be able to explain clearly to your boss and your customers

why it’s not a problem. So you have to look carefully to ﬁnd out exactly what

kind of attack has been found, and what the parameters are. For example,

the ﬁrst major attack announced on the Data Encryption Standard algorithm

requires 247 chosen plaintexts to recover the key, while the next major attack

improved this to 243 known plaintexts. While these attacks were of great sci-

entiﬁc importance, their practical engineering effect was zero, as no practical

systems make that much known (let alone chosen) text available to an attacker.

Such attacks are often referred to as certiﬁcational. They can have a commercial

effect, though: the attacks on DES undermined conﬁdence in it and started

moving people to other ciphers. In some other cases, an attack that started off

as certiﬁcational has been developed by later ideas into an exploit.

Which sort of attacks you should be worried about depends on your

application. With a broadcast entertainment system, for example, a bad man

can buy a decoder, observe a lot of material and compare it with the enciphered

broadcast signal; so a known-plaintext attack is the main threat. But there are

surprisingly many applications where chosen-plaintext attacks are possible.

Obvious ones include IFF, where the enemy can send challenges of his choice

to any aircraft in range of one of his radars; and ATMs, where if you allow

customers to change their PINs, an attacker can change his PIN through a range

of possible values and observe the enciphered equivalents by wiretapping the

line from the ATM to the bank. A more traditional example is diplomatic

messaging systems, where it’s been known for a host government to give an

ambassador a message to transmit to his capital that’s been specially designed

to help the local cryptanalysts ﬁll out the missing gaps in the ambassador’s

code book [676]. In general, if the opponent can insert any kind of message

into your system, it’s chosen-plaintext attacks you should worry about.

The other attacks are more specialized. Chosen plaintext/ciphertext attacks

may be a worry where the threat is a lunchtime attacker:someonewhogets

temporary access to some cryptographic equipment while its authorized

user is out. Related-key attacks are a concern where the block cipher is used

as a building block in the construction of a hash function (which we’ll

discuss below).

5.3.4 Public Key Encryption and Trapdoor One-Way

Permutations

Apublic-key encryption algorithm is a special kind of block cipher in which the

elf will perform the encryption corresponding to a particular key for anyone

who requests it, but will do the decryption operation only for the key’s owner.

To continue with our analogy, the user might give a secret name to the scroll

that only she and the elf know, use the elf’s public one-way function to

5.3 The Random Oracle Model 147

compute a hash of this secret name, publish the hash, and instruct the elf to

perform the encryption operation for anybody who quotes this hash.

This means that a principal, say Alice, can publish a key and if Bob wants

to, he can now encrypt a message and send it to her, even if they have never

met. All that is necessary is that they have access to the oracle. There are some

more details that have to be taken care of, such as how Alice’s name can be

bound to the key, and indeed whether it means anything to Bob; I’ll deal with

these later.

A common way of implementing public key encryption is the trapdoor

one-way permutation. This is a computation which anyone can perform, but

which can be reversed only by someone who knows a trapdoor such as a secret

key. This model is like the ‘one-way function’ model of a cryptographic hash

function. Let us state it formally nonetheless: a public key encryption primitive

consists of a function which given a random input Rwill return two keys, KR

(the public encryption key) and KR−1(the private decryption key) with the

properties that

1. Given KR, it is infeasible to compute KR−1(so it’s not possible to com-

pute Reither);

2. There is an encryption function {...}which, applied to a message M

using the encryption key KR, will produce a ciphertext C={M}KR;and

3. There is a decryption function which, applied to a ciphertext Cusing the

decryption key KR−1, will produce the original message M={C}KR−1.

For practical purposes, we will want the oracle to be replicated at both ends

of the communications channel, and this means either using tamper-resistant

hardware or (more commonly) implementing its functions using mathematics

rather than metal. There are several more demanding models than this, for

example to analyze security in the case where the opponent can get ciphertexts

of his choice decrypted, with the exception of the target ciphertext. But this

will do for now.

5.3.5 Digital Signatures

The ﬁnal cryptographic primitive which we’ll deﬁne here is the digital sig-

nature. The basic idea is that a signature on a message can be created by only

one person, but checked by anyone. It can thus perform the sort of function

in the electronic world that ordinary signatures do in the world of paper.

Applications include signing software updates, so that a PC can tell that an

update to Windows was really produced by Microsoft rather than by a villain.

Signature schemes can be deterministic or randomized: in the ﬁrst, computing

a signature on a message will always give the same result and in the second,

it will give a different result. (The latter is more like handwritten signatures;

148 Chapter 5 ■Cryptography

no two are ever alike but the bank has a means of deciding whether a given

specimen is genuine or forged). Also, signature schemes may or may not

support message recovery. If they do, then given the signature, anyone can

recover the message on which it was generated; if they don’t, then the veriﬁer

needs to know or guess the message before he can perform the veriﬁcation.

(There are further, more specialised, signature schemes such as blind signatures

and threshold signatures but I’ll postpone discussion of them for now.)

Formally, a signature scheme, like public key encryption scheme, has a

keypair generation function which given a random input Rwill return two

keys, σR(the private signing key) and VR (the public signature veriﬁcation

key) with the properties that

1. Given the public signature veriﬁcation key VR, it is infeasible to com-

pute the private signing key σR;

2. There is a digital signature function which given a message M and a

private signature key σR, will produce a signature SigσR(M); and

3. There is a signature veriﬁcation function which, given the signature

SigσR(M) and the public signature veriﬁcation key VR will output TRUE

if the signature was computed correctly with σRand otherwise output

FALSE.

We can model a simple digital signature algorithm as a random function that

reduces any input message to a one-way hash value of ﬁxed length, followed

by a special kind of block cipher in which the elf will perform the operation

in one direction, known as signature, for only one principal, while in the other

direction, it will perform veriﬁcation for anybody.

Signature veriﬁcation can take two forms. In the basic scheme, the elf (or the

signature veriﬁcation algorithm) only outputs TRUE or FALSE depending on

whether the signature is good. But in a scheme with message recovery,anyone

can input a signature and get back the message corresponding to it. In our

elf model, this means that if the elf has seen the signature before, it will give

the message corresponding to it on the scroll, otherwise it will give a random

value (and record the input and the random output as a signature and message

pair). This is sometimes desirable: when sending short messages over a low

bandwidth channel, it can save space if only the signature has to be sent rather

than the signature plus the message. An example is in the machine-printed

postage stamps, or indicia, being brought into use in many countries: the

stamp may consist of a 2-d barcode with a digital signature made by the postal

meter and which contains information such as the value, the date and the

sender’s and recipient’s post codes. We give some more detail about this at

the end of section 14.3.2.

5.4 Symmetric Crypto Primitives 149

However, in the general case we do not need message recovery, as the

message to be signed may be of arbitrary length and so we will ﬁrst pass it

through a hash function and then sign the hash value. As hash functions are

one-way, the resulting compound signature scheme does not have message

recovery — although if the underlying signature scheme does, then the hash

of the message can be recovered from the signature.

5.4 Symmetric Crypto Primitives

Now that we have deﬁned the basic crypto primitives, we will look under the

hood to see how they can be implemented in practice. While most explanations

are geared towards graduate mathematics students, the presentation I’ll give

here is based on one I’ve developed over several years with computer science

students. So I hope it will let the non-mathematician grasp the essentials. In

fact, even at the research level, most of cryptography is as much computer

science as mathematics. Modern attacks on ciphers are put together from

guessing bits, searching for patterns, sorting possible results, and so on rather

than from anything particularly highbrow.

We’ll focus in this section on block ciphers, and then see in the next section

how you can make hash functions and stream ciphers from them, and vice

versa. (In later chapters, we’ll also look at some special-purpose ciphers.)

5.4.1 SP-Networks

Claude Shannon suggested in the 1940’s that strong ciphers could be built

by combining substitution with transposition repeatedly. For example, one

might add some key material to a block of input text, and then shufﬂe subsets

of the input, and continue in this way a number of times. He described the

properties of a cipher as being confusion and diffusion — adding unknown key

values will confuse an attacker about the value of a plaintext symbol, while

diffusion means spreading the plaintext information through the ciphertext.

Block ciphers need diffusion as well as confusion.

The earliest block ciphers were simple networks which combined sub-

stitution and permutation circuits, and so were called SP-networks [681].

Figure 5.10 shows an SP-network with sixteen inputs, which we can imagine

as the bits of a sixteen-bit number, and two layers of four-bit invertible sub-

stitution boxes (or S-boxes), each of which can be visualized as a lookup table

containing some permutation of the numbers 0 to 15.

The point of this arrangement is that if we were to implement an arbitrary 16

bit to 16 bit function in digital logic, we would need 220 bits of memory — one

150 Chapter 5 ■Cryptography

lookup table of 216 bits for each single output bit. That’s hundreds of thousands

of gates, while a four bit to four bit function takes only 4 x 24or 64 bits of

memory. One might hope that with suitable choices of parameters, the function

produced by iterating this simple structure would be indistinguishable from a

random 16 bit to 16 bit function to an opponent who didn’t know the value of

the key. The key might consist of some choice of a number of four-bit S-boxes,

or it might be added at each round to provide confusion and the resulting text

fed through the S-boxes to provide diffusion.

Three things need to be done to make such a design secure:

1. the cipher needs to be ‘‘wide’’ enough

2. it needs to have enough rounds, and

3. the S-boxes need to be suitably chosen.

5.4.1.1 Block Size

First, a block cipher which operated on sixteen bit blocks would have rather

limited applicability, as an opponent could just build a dictionary of plaintext

and ciphertext blocks as he observed them. The birthday theorem tells us that

even if the input plaintexts were random, he’d expect to ﬁnd a match as soon

as he had seen a little over 28blocks. So a practical block cipher will usually

deal with plaintexts and ciphertexts of 64 bits, 128 bits or even more. So if we

are using four-bit to four-bit S-boxes, we may have 16 of them (for a 64 bit

block size) or 32 of them (for a 128 bit block size).

5.4.1.2 Number of Rounds

Second, we have to have enough rounds. The two rounds in Figure 5.10 are

completely inadequate, as an opponent can deduce the values of the S-boxes

by tweaking input bits in suitable patterns. For example, he could hold the

rightmost 12 bits constant and try tweaking the leftmost four bits, to deduce

the values in the top left S-box. (The attack is slightly more complicated than

this, as sometimes a tweak in an input bit to an S-box won’t produce a change

in any output bit, so we have to change one of its other inputs and tweak

again. But implementing it is still a simple student exercise.)

The number of rounds we require depends on the speed with which data

diffuse through the cipher. In the above simple example, diffusion is very slow

because each output bit from one round of S-boxes is connected to only one

input bit in the next round. Instead of having a simple permutation of the

wires, it is more efﬁcient to have a linear transformation in which each input

bit in one round is the exclusive-or of several output bits in the previous round.

Of course, if the block cipher is to be used for decryption as well as encryption,

this linear transformation will have to be invertible. We’ll see some concrete

examples below in the sections on Serpent and AES.

5.4 Symmetric Crypto Primitives 151



S-box S-box S-box S-box

















S-box S-box S-box S-box



Figure 5.10: A simple 16-bit SP-network block cipher

5.4.1.3 Choice of S-Boxes

The design of the S-boxes also affects the number of rounds required for

security, and studying bad choices gives us our entry into the deeper theory

of block ciphers. Suppose that the S-box were the permutation that maps the

inputs (0,1,2,...,15) to the outputs (5,7,0,2,4,3,1,6,8,10,15,12,9,11,14,13). Then

the most signiﬁcant bit of the input would come through unchanged as the

most signiﬁcant bit of the output. If the same S-box were used in both rounds

in the above cipher, then the most signiﬁcant bit of the input would pass

through to become the most signiﬁcant bit of the output. This would usually

be a bad thing; we certainly couldn’t claim that our cipher was pseudorandom.

5.4.1.4 Linear Cryptanalysis

Attacks on real block ciphers are usually harder to spot than in this artiﬁcial

example, but they use the same ideas. It might turn out that the S-box had

the property that bit one of the input was equal to bit two plus bit four

of the output; more commonly, there will be linear approximations to an S-box

which hold with a certain probability. Linear cryptanalysis [602, 843] proceeds

by collecting a number of relations such as ‘bit 2 plus bit 5 of the input to the

ﬁrst S-box is equal to bit 1 plus bit 8 of the output, with probability 13/16’

and then searching for ways to glue them together into an algebraic relation

between input bits, output bits and key bits that holds with a probability

different from one half. If we can ﬁnd a linear relationship that holds over the

whole cipher with probability p=0.5+1/M, then according to probability

theory we can expect to start recovering keybits once we have about M2

known texts. If the value of M2for the best linear relationship is greater than

the total possible number of known texts (namely 2nwhere the inputs and

outputs are nbits wide), then we consider the cipher to be secure against linear

cryptanalysis.

152 Chapter 5 ■Cryptography

5.4.1.5 Differential Cryptanalysis

Differential Cryptanalysis [170, 602] is similar but is based on the probability that

a given change in the input to an S-box will give rise to a certain change in the

output. A typical observation on an 8-bit S-box might be that ‘if we ﬂip input

bits 2, 3, and 7 at once, then with probability 11/16 the only output bits that

will ﬂip are 0 and 1’. In fact, with any nonlinear Boolean function, tweaking

some combination of input bits will cause some combination of output bits to

change with a probability different from one half. The analysis procedure is

to look at all possible input difference patterns and look for those values δi,

δosuch that an input change of δiwill produce an output change of δowith

particularly high (or low) probability.

As in linear cryptanalysis, we then search for ways to join things up so that

an input difference which we can feed into the cipher will produce a known

output difference with a useful probability over a number of rounds. Given

enough chosen inputs, we will see the expected output and be able to make

deductions about the key. As in linear cryptanalysis, it’s common to consider

the cipher to be secure if the number of texts required for an attack is greater

than the total possible number of different texts for that key. (We have to be

careful though of pathological cases, such as if you had a cipher with a 32-bit

block and a 128-bit key with a differential attack whose success probability

given a single pair was 2−40. Given a lot of text under a number of keys, we’d

eventually solve for the current key.)

There are a quite a few variants on these two themes. For example, instead of

looking for high probability differences, we can look for differences that can’t

happen (or that happen only rarely). This has the charming name of impossible

cryptanalysis, but it is quite deﬁnitely possible against many systems [169].

There are also various specialised attacks on particular ciphers.

Block cipher design involves a number of trade-offs. For example, we can

reduce the per-round information leakage, and thus the required number of

rounds, by designing the rounds carefully. However, a complex design might

be slow in software, or need a lot of gates in hardware, so using simple rounds

but more of them might have been better. Simple rounds may also be easier

to analyze. A prudent designer will also use more rounds than are strictly

necessary to block the attacks known today, in order to give some margin of

safety against improved mathematics in the future. We may be able to show

that a cipher resists all the attacks we know of, but this says little about whether

it will resist the attacks we don’t know of yet. (A general security proof for a

block cipher would appear to imply a proof about an attacker’s computational

powers, which might entail a result such as P= NP that would revolutionize

computer science.)

The point that the security engineer should remember is that block cipher

cryptanalysis is a complex subject about which we have a fairly extensive

theory. Use an off-the-shelf design that has been thoroughly scrutinized

5.4 Symmetric Crypto Primitives 153

by experts, rather than rolling your own; and if there’s a compelling reason

to use a proprietary cipher (for example, if you want to use a patented design to

stop other people copying a product) then get it reviewed by experts. Cipher

design is not an amateur sport any more.

5.4.1.6 Serpent

As a concrete example, the encryption algorithm ‘Serpent’ is an SP-network

with input and output block sizes of 128 bits. These are processed through 32

rounds, in each of which we ﬁrst add 128 bits of key material, then pass the text

through 32 S-boxes of 4 bits width, and then perform a linear transformation

that takes each output of one round to the inputs of a number of S-boxes

in the next round. Rather than each input bit in one round coming from a

single output bit in the last, it is the exclusive-or of between two and seven of

them. This means that a change in an input bit propagates rapidly through the

cipher — a so-called avalanche effect which makes both linear and differential

attacks harder. After the ﬁnal round, a further 128 bits of key material

are added to give the ciphertext. The 33 times 128 bits of key material required

are computed from a user supplied key of up to 256 bits.

This is a real cipher using the structure of Figure 5.10, but modiﬁed

to be ‘wide’ enough and to have enough rounds. The S-boxes are chosen to

make linear and differential analysis hard; they have fairly tight bounds on

the maximum linear correlation between input and output bits, and on the

maximum effect of toggling patterns of input bits. Each of the 32 S-boxes in a

given round is the same; this means that bit-slicing techniques can be used to

give a very efﬁcient software implementation on 32-bit processors.

Its simple structure makes Serpent easy to analyze, and it can be shown that

it withstands all the currently known attacks. A full speciﬁcation of Serpent

is given in [60] and can be downloaded, together with implementations in a

number of languages, from [61].

5.4.2 The Advanced Encryption Standard (AES)

This discussion has prepared us to describe the Advanced Encryption Stan-

dard, an algorithm also known as Rijndael after its inventors Vincent Rijmen

and Joan Daemen [342]. This algorithm acts on 128-bit blocks and can use a

key of 128, 192 or 256 bits in length. It is an SP-network; in order to specify it,

we need to ﬁx the S-boxes, the linear transformation between the rounds, and

the way in which the key is added into the computation.

AES uses a single S-box which acts on a byte input to give a byte output.

For implementation purposes it can be regarded simply as a lookup table of

256 bytes; it is actually deﬁned by the equation S(x)=M(1/x)+bover the

ﬁeld GF(28)whereMis a suitably chosen matrix and bis a constant. This

construction gives tight differential and linear bounds.

154 Chapter 5 ■Cryptography

The linear transformation is based on arranging the 16 bytes of the value

being enciphered in a square and then doing bytewise shufﬂing and mixing

operations. (AES is descended from an earlier cipher called Square, which

introduced this technique.)

The ﬁrst step in the linear transformation is the shufﬂe in which the top row

of four bytes is left unchanged, while the second row is shifted one place to

the left, the third row by two places and the fourth row by three places. The

second step is a column mixing step in which the four bytes in a column are

mixed using a matrix multiplication. This is illustrated in Figure 5.11 which

shows, as an example, how a change in the value of the third byte in the ﬁrst

column is propagated. The effect of this combination is that a change in the

input to the cipher can potentially affect all of the output after just two rounds.

The key material is added byte by byte after the linear transformation. This

means that 16 bytes of key material are needed per round; they are derived

from the user supplied key material by means of a recurrence relation.

The algorithm uses 10 rounds with 128-bit keys, 12 rounds with 192-bit keys

and 14 rounds with 256-bit keys. These give a reasonable margin of safety; the

best shortcut attacks known at the time of writing (2007) can tackle 7 rounds

for 128-bit keys, and 9 rounds for 192- and 256-bit keys [16]. The general belief

in the block cipher community is that even if advances in the state of the art

do permit attacks on AES with the full number of rounds, they will be purely

certiﬁcational attacks in that they will require infeasibly large numbers of texts.

(AES’s margin of safety against attacks that require only feasible numbers of

texts is about 100%.) Although there is no proof of security — whether in the

sense of pseudorandomness, or in the weaker sense of an absence of shortcut

attacks of known types — there is now a high level of conﬁdence that AES is

secure for all practical purposes. The NSA has since 2005 approved AES with

128-bit keys for protecting information up to SECRET and with 256-bit keys

for TOP SECRET.

. . .1

Shift

row

Mix

column

Figure 5.11: The AES linear transformation, illustrated by its effect on byte 3 of the input

5.4 Symmetric Crypto Primitives 155

Even although I was an author of Serpent which was an unsuccessful ﬁnalist

in the AES competition (the winner Rijndael got 86 votes, Serpent 59 votes,

Twoﬁsh 31 votes, RC6 23 votes and MARS 13 votes at the last AES conference),

and although Serpent was designed to have an even larger security margin

than Rijndael, I recommend to my clients that they use AES where a general-

purpose block cipher is required. I recommend the 256-bit-key version, and

not because I think that the 10 rounds of the 128-bit-key variant will be broken

anytime soon. Longer keys are better because some key bits often leak in real

products, as I’ll discuss at some length in the chapters on tamper-resistance

and emission security. It does not make sense to implement Serpent as well,

‘just in case AES is broken’: the risk of a fatal error in the algorithm negotiation

protocol is orders of magnitude greater than the risk that anyone will come

up with a production attack on AES. (We’ll see a number of examples later

where using multiple algorithms, or using an algorithm like DES multiple

times, caused something to break horribly.)

The deﬁnitive speciﬁcation of AES is Federal Information Processing Stan-

dard 197, and its inventors have written a book describing its design in

detail [342]. Other information, from book errata to links to implementations,

can be found on the AES Lounge web page [16].

One word of warning: the most likely practical attacks on a real imple-

mentation of AES include timing analysis and power analysis, both of which

I discuss in Part II in the chapter on emission security. In timing analysis,

the risk is that an opponent observes cache misses and uses them to work

out the key. The latest versions of this attack can extract a key given the

precise measurements of the time taken to do a few hundred cryptographic

operations. In power analysis, an opponent uses measurements of the current

drawn by the device doing the crypto — think of a bank smartcard that a

customer places in a terminal in a Maﬁa-owned shop. The two overlap; cache

misses cause a device like a smartcard to draw more power — and can also

be observed on remote machines by an opponent who can measure the time

taken to encrypt. The implementation details matter.

5.4.3 Feistel Ciphers

Many block ciphers use a more complex structure, which was invented by

Feistel and his team while they were developing the Mark XII IFF in the late

1950’s and early 1960’s. Feistel then moved to IBM and founded a research

group which produced the Data Encryption Standard, (DES) algorithm, which

is still the mainstay of ﬁnancial transaction processing security.

A Feistel cipher has the ladder structure shown in Figure 5.12. The input is

split up into two blocks, the left half and the right half. A round function f1of

156 Chapter 5 ■Cryptography

the left half is computed and combined with the right half using exclusive-or

(binary addition without carry), though in some Feistel ciphers addition with

carry is also used. (We use the notation ⊕for exclusive-or.) Then, a function

f2of the right half is computed and combined with the left half, and so

on. Finally (if the number of rounds is even) the left half and right half are

swapped.

 





⊕•

f2k

 



...

 

⊕•

 



•⊕

  

Left Half Right Half

Figure 5.12: The Feistel cipher structure

5.4 Symmetric Crypto Primitives 157

A notation which you may see for the Feistel cipher is ψ(f,g,h,...)where

f,g,h,... are the successive round functions. Under this notation, the above

cipher is ψ(f1,f2,...f2k−1,f2k). The basic result that enables us to decrypt a Feistel

cipher — and indeed the whole point of his design — is that:

ψ−1(f1,f2,...,f2k−1,f2k)=ψ(f2k,f2k−1,...,f2,f1)

In other words, to decrypt, we just use the round functions in the reverse

order. Thus the round functions fido not have to be invertible, and the Feistel

structure lets us turn any one-way function into a block cipher. This means

that we are less constrained in trying to choose a round function with good

diffusion and confusion properties, and which also satisﬁes any other design

constraints such as code size, table size, software speed, hardware gate count,

and so on.

5.4.3.1 The Luby-Rackoff Result

The key theoretical result on Feistel ciphers was proved by Mike Luby and

Charlie Rackoff in 1988. They showed that if fiwere random functions, then

ψ(f1,f2,f3) was indistinguishable from a random permutation under chosen

plaintext attack, and this result was soon extended to show that ψ(f1,f2,f3,f4)

was indistinguishable under chosen plaintext/ciphertext attack — in other

words, it was a pseudorandom permutation.

There are a number of technicalities we omit. In engineering terms, the

effect is that given a really good round function, four rounds of Feistel are

enough. So if we have a hash function in which we have conﬁdence, it is

straightforward to construct a block cipher from it: use four rounds of keyed

hash in a Feistel network.

5.4.3.2 DES

The DES algorithm is widely used in banking, government and embedded

applications. For example, it is the standard in automatic teller machine

networks. It is a Feistel cipher, with a 64-bit block and 56-bit key. Its round

function operates on 32-bit half blocks and consists of three operations:

ﬁrst, the block is expanded from 32 bits to 48;

next, 48 bits of round key are mixed in using exclusive-or;

the result is passed through a row of eight S-boxes, each of which takes a

six-bit input and provides a four-bit output;

ﬁnally, the bits of the output are permuted according to a ﬁxed pattern.

158 Chapter 5 ■Cryptography

The effect of the expansion, key mixing and S-boxes is shown in Figure 5.13:

Si – 1 Si + 1

Key added

in here

• • • • • •Si

Figure 5.13: The DES round function

The round keys are derived from the user-supplied key by using each user

key bit in twelve different rounds according to a slightly irregular pattern.

A full speciﬁcation of DES is given in [936]; code can be found in [1125] or

downloaded from many places on the web.

DES was introduced in 1974 and caused some controversy. The most telling

criticism was that the key is too short. Someone who wants to ﬁnd a 56 bit

key using brute force, that is by trying all possible keys, will have a total

exhaust time of 256 encryptions and an average solution time of half that, namely

255 encryptions. Whit Difﬁe and Martin Hellman argued in 1977 that a DES

keysearch machine could be built with a million chips, each testing a million

keys a second; as a million is about 220, this would take on average 215 seconds,

or a bit over 9 hours, to ﬁnd the key. They argued that such a machine could

be built for $20 million dollars in 1977 [386]. IBM, whose scientists invented

DES, retorted that they would charge the US government $200 million to build

such a machine. (Perhaps both were right.)

During the 1980’s, there were persistent rumors of DES keysearch machines

being built by various intelligence agencies, but the ﬁrst successful pub-

lic keysearch attack took place in 1997. In a distributed effort organised

over the net, 14,000 PCs took more than four months to ﬁnd the key to

a challenge. In 1998, the Electronic Frontier Foundation (EFF) built a DES

keysearch machine called Deep Crack for under $250,000 which broke a

DES challenge in 3 days. It contained 1,536 chips run at 40MHz, each chip

containing 24 search units which each took 16 cycles to do a test decrypt. The

search rate was thus 2.5 million test decryptions per second per search unit, or

60 million keys per second per chip. The design of the cracker is public and can

be found at [423]. By 2006, Sandeep Kumar and colleagues at the universities

of Bochum and Kiel built a machine using 120 FPGAs and costing $10,000,

which could break DES in 7 days on average [755]. A modern botnet with half

5.4 Symmetric Crypto Primitives 159

a million machines would take a few hours. So the key length of DES is now

deﬁnitely inadequate, and banks have for some years been upgrading their

payment systems.

Another criticism of DES was that, since IBM kept its design principles secret

at the request of the US government, perhaps there was a ‘trapdoor’ which

would give them easy access. However, the design principles were published

in 1992 after differential cryptanalysis was invented and published [326].

Their story was that IBM had discovered these techniques in 1972, and the

US National Security Agency (NSA) even earlier. IBM kept the design details

secret at the NSA’s request. We’ll discuss the political aspects of all this

in 24.3.9.1.

We now have a fairly thorough analysis of DES. The best known shortcut

attack, that is, a cryptanalytic attack involving less computation than keysearch,

is a linear attack using 242 known texts. DES would be secure with more than

20 rounds, but for practical purposes its security is limited by its keylength. I

don’t know of any real applications where an attacker might get hold of even

240 known texts. So the known shortcut attacks are not an issue. However, its

growing vulnerability to keysearch makes DES unusable in its original form.

If Moore’s law continues, than by 2020 it might be possible to ﬁnd a DES key

on a single PC in a few months, so even low-grade systems such as taxi meters

will be vulnerable to brute force-cryptanalysis. As with AES, there are also

attacks based on timing analysis and power analysis, but because of DES’s

structure, the latter are more serious.

The usual way of dealing with the DES keysearch problem is to use

the algorithm multiple times with different keys. Banking networks have

largely moved to triple-DES, a standard since 1999 [936]. Triple-DES does an

encryption, then a decryption, and then a further encryption, all done with

independent keys. Formally:

3DES(k0,k1,k2;M)=DES(k2;DES−1(k1;DES(k0;M)))

The reason for this design is that by setting the three keys equal, one gets the

same result as a single DES encryption, thus giving a backwards compatibility

mode with legacy equipment. (Some banking systems use two-key triple-DES

which sets k2=k0; this gives an intermediate step between single and triple

DES). New systems now use AES as of choice, but banking systems are deeply

committed to using block ciphers with an eight-byte block size, because of the

message formats used in the many protocols by which ATMs, point-of-sale

terminals and bank networks talk to each other, and because of the use of block

ciphers to generate and protect customer PINs (which I discuss in Chapter 10).

Triple DES is a perfectly serviceable block cipher for such purposes for the

foreseeable future.

Another way of preventing keysearch (and making power analysis harder) is

whitening. In addition to the 56-bit key, say k0, we choose two 64-bit whitening

160 Chapter 5 ■Cryptography

keys k1and k2, xor’ing the ﬁrst with the plaintext before encryption and the

second with the output of the encryption to get the ciphertext afterwards. This

composite cipher is known as DESX, and is used in the Win2K encrypting ﬁle

system. Formally,

DESX(k0,k1,k2;M)=DES(k0;M⊕k1)⊕k2

It can be shown that, on reasonable assumptions, DESX has the properties

you’d expect; it inherits the differential strength of DES but its resistance to

keysearch is increased by the amount of the whitening [717]. Whitened block

ciphers are used in some applications.

5.5 Modes of Operation

In practice, how you use an encryption algorithm is often more important

than which one you pick. An important factor is the ‘mode of operation’,which

speciﬁes how a block cipher with a ﬁxed block size (8 bytes for DES, 16 for

AES) can be extended to process messages of arbitrary length.

There are several standard modes of operation for using a block cipher on

multiple blocks [944]. Understanding them, and choosing the right one for the

job, is an important factor in using a block cipher securely.

5.5.1 Electronic Code Book

In electronic code book (ECB) we just encrypt each succeeding block of

plaintext with our block cipher to get ciphertext, as with the Playfair cipher

I gave above as an example. This is adequate for many simple operations

such as challenge-response and some key management tasks; it’s also used

to encrypt PINs in cash machine systems. However, if we use it to encrypt

redundant data the patterns will show through, letting an opponent deduce

information about the plaintext. For example, if a word processing format has

lots of strings of nulls, then the ciphertext will have a lot of blocks whose value

is the encryption of null characters under the current key.

In one popular corporate email system from the late 1980’s, the encryption

used was DES ECB with the key derived from an eight character password. If

you looked at a ciphertext generated by this system, you saw that a certain block

was far more common than the others — the one corresponding to a plaintext

of nulls. This gave one of the simplest attacks on a ﬁelded DES encryption

system: just encrypt a null block with each password in a dictionary and sort

the answers. You can now break at sight any ciphertext whose password was

one of those in your dictionary.

In addition, using ECB mode to encrypt messages of more than one block

length which have an authenticity requirement — such as bank payment

5.5 Modes of Operation 161

messages — would be foolish, as messages could be subject to a cut and splice

attack along the block boundaries. For example, if a bank message said ‘Please

pay account number Xthe sum Y, and their reference number is Z’thenan

attacker might initiate a payment designed so that some of the digits of X

could be replaced with some of the digits of Z.

5.5.2 Cipher Block Chaining

Most commercial applications which encrypt more than one block use cipher

block chaining, or CBC, mode. In it, we exclusive-or the previous block of

ciphertext to the current block of plaintext before encryption (see Figure 5.14).

This mode is effective at disguising any patterns in the plaintext: the

encryption of each block depends on all the previous blocks. The input IV

is an initialization vector, a random number that performs the same function

as a seed in a stream cipher and ensures that stereotyped plaintext message

headers won’t leak information by encrypting to identical ciphertext blocks.

However, an opponent who knows some of the plaintext may be able to

cut and splice a message (or parts of several messages encrypted under the

same key), so the integrity protection is not total. In fact, if an error is inserted

into the ciphertext, it will affect only two blocks of plaintext on decryption,

so if there isn’t any integrity protection on the plaintext, an enemy can insert

two-block garbles of random data at locations of his choice.

5.5.3 Output Feedback

Output feedback (OFB) mode consists of repeatedly encrypting an initial value

and using this as a keystream in a stream cipher of the kind discussed above.

  

•••

EKEKEK

  

⊕⊕⊕

  

 

P1P2P3

C1C2C3

...

Figure 5.14: Cipher Block Chaining (CBC) mode

162 Chapter 5 ■Cryptography

Writing IV for the initialization vector or seed, the i-th block of keystream will

be given by

Ki={...{{IV}K}K...total of i times}

This is one standard way of turning a block cipher into a stream cipher.

The key Kis expanded into a long stream of blocks Kiof keystream. Keystream

is typically combined with the blocks of a message Miusing exclusive-or to

give ciphertext Ci=Mi⊕Ki; this arrangement is sometimes called an additive

stream cipher as exclusive-or is just addition module 2 (and some old hand

systems used addition modulo 26).

All additive stream ciphers have an important vulnerability: they fail to

protect message integrity. I mentioned this in the context of the one-time

pad in section 5.2.2 above, but it’s important to realise that this doesn’t just

affect ‘perfectly secure’ systems but ‘real life’ stream ciphers too. Suppose, for

example, that a stream cipher were used to encipher fund transfer messages.

These messages are very highly structured; you might know, for example, that

bytes 37–42 contained the amount of money being transferred. You could then

carry out the following attack. You cause the data trafﬁc from a local bank to

go via your computer, for example by a wiretap. You go into the bank and

send a modest sum (say $500) to an accomplice. The ciphertext Ci=Mi⊕Ki,

duly arrives in your machine. You know Mifor bytes 37–42, so you know

Kiand can easily construct a modiﬁed message which instructs the receiving

bank to pay not $500 but $500,000! This is an example of an attack in depth;itis

the price not just of the perfect secrecy we get from the one-time pad, but of

much more humble stream ciphers too.

5.5.4 Counter Encryption

One possible drawback of feedback modes of block cipher encryption is

latency: feedback modes are hard to parallelize. With CBC, a whole block of

the cipher must be computed between each block input and each block output;

with OFB, we can precompute keystream but storing it requires memory. This

can be inconvenient in very high speed applications, such as protecting trafﬁc

on gigabit backbone links. There, as silicon is cheap, we would rather pipeline

our encryption chip, so that it encrypts a new block (or generates a new block

of keystream) in as few clock ticks as possible.

The simplest solution is often is to generate a keystream by just encrypting

acounter:Ki={IV +i}K. As before, this is then added to the plaintext to get

ciphertext (so it’s also vulnerable to attacks in depth).

Another problem this mode solves when using a 64-bit block cipher such

as triple-DES on a very high speed link is cycle length. An n-bit block cipher

in OFB mode will typically have a cycle length of 2n/2blocks, after which the

birthday theorem will see to it that the keystream starts to repeat. (Once we’ve

a little over 232 64-bit values, the odds are that two of them will match.) In

5.5 Modes of Operation 163

CBC mode, too, the birthday theorem ensures that after about 2n/2blocks, we

will start to see repeats. Counter mode encryption, however, has a guaranteed

cycle length of 2nrather than 2n/2.

5.5.5 Cipher Feedback

Cipher feedback, or CFB, mode is another kind of stream cipher. It was

designed to be self-synchronizing, in that even if we get a burst error and drop

a few bits, the system will recover synchronization after one block length. This

is achieved by using our block cipher to encrypt the last nbits of ciphertext,

and then adding one of the output bits to the next plaintext bit.

With decryption, the reverse operation is performed, with ciphertext feeding

in from the right in Figure 5.15. Thus even if we get a burst error and drop a

few bits, as soon as we’ve received enough ciphertext bits to ﬁll up the shift

Cipher feedback is not much used any more. It was designed for use in

military HF radio links which are vulnerable to fading, in the days when

digital electronics were relatively expensive. Now that silicon is cheap, people

use dedicated link layer protocols for synchronization and error correction

rather than trying to combine them with the cryptography.

5.5.6 Message Authentication Code

The next ofﬁcial mode of operation of a block cipher is not used to encipher data,

but to protect its integrity and authenticity. This is the message authentication

code, or MAC. To compute a MAC on a message using a block cipher, we

encrypt it using CBC mode and throw away all the output ciphertext blocks

except the last one; this last block is the MAC. (The intermediate results are

kept secret in order to prevent splicing attacks.)

⊕





shift register 

•

plaintext ciphertext

Figure 5.15: Ciphertext feedback mode (CFB)

164 Chapter 5 ■Cryptography

This construction makes the MAC depend on all the plaintext blocks as

well as on the key. It is secure provided the message length is ﬁxed; Mihir

Bellare, Joe Kilian and Philip Rogaway proved that any attack on a MAC

under these circumstances would give an attack on the underlying block

cipher [147].

If the message length is variable, you have to ensure that a MAC computed

on one string can’t be used as the IV for computing a MAC on a different

string, so that an opponent can’t cheat by getting a MAC on the composition of

the two strings. In order to ﬁx this problem, NIST has standardised CMAC, in

which a variant of the key is xor-ed in before the last encryption [945]. (CMAC

is based on a proposal by Tetsu Iwata and Kaoru Kurosawa [649].)

There are other possible constructions of MACs: a common one is to use a

hash function with a key, which we’ll look at in more detail in section 5.6.2.

5.5.7 Composite Modes of Operation

In applications needing both integrity and privacy, the standard procedure

used to be to ﬁrst calculate a MAC on the message using one key, and then CBC

encrypt it using a different key. (If the same key is used for both encryption

and authentication, then the security of the latter is no longer guaranteed;

cut-and-splice attacks are still possible.)

Recently two further modes of operation have been tackled by NIST that

combine encryption and authentication. The ﬁrst is CCM, which combines

counter-mode encryption with CBC-MAC authentication. The danger to watch

for here is that the counter values used in encryption must not coincide with the

initialisation vector used in the MAC; the standard requires that the formatting

function prevent this [946].

The second combined mode is Galois Counter Mode (GCM), which has just

been approved at the time of writing (2007). This interesting and innovative

mode is designed to be parallelisable so that it can give high throughput

on fast data links with low cost and low latency. As the implementation is

moderately complex, and the algorithm was approved as this book was in

its ﬁnal edit, I don’t include the details here, but refer you instead to the

ofﬁcial speciﬁcation [947]. The telegraphic summary is that the encryption is

performed in a variant of counter mode; the resulting ciphertexts are also

multiplied together with key material and message length information in a

Galois ﬁeld of 2128 elements to get an authenticator tag. The output is thus

a ciphertext of the same length as the plaintext, plus a tag of typically 128

bits. The tag computation uses a universal hash function which comes from

the theory of unconditionally-secure authentication codes; I’ll describe this in

Chapter 13, ‘Nuclear Command and Control’.

5.6 Hash Functions 165

Both CCM, and old-fashioned CBC plus CBC MAC, need a completely new

MAC to be computed on the whole message if any bit of it is changed. How-

ever, the GCM mode of operation has an interesting incremental property: a

new authenticator and ciphertext can be calculated with an amount of effort

proportional to the number of bits that were changed. GCM is an invention of

David McGrew and John Viega of Cisco; their goal was to create an authenti-

cated encryption mode that is highly parallelisable for use in high-performance

network hardware and that only uses one block cipher operation per block of

plaintext, unlike CCM or the old-fashioned CBC plus CBC-MAC [862]. Now

that GCM has been adopted as a standard, we might expect it to become the

most common mode of operation for the encryption of bulk content.

5.6 Hash Functions

In section 5.4.3.1 I showed how the Luby-Rackoff theorem enables us to

construct a block cipher from a hash function. It’s also possible to construct

a hash function from a block cipher. (In fact, we can also construct hash

functions and block ciphers from stream ciphers — so, subject to some caveats

I’ll discuss in the next section, given any one of these three primitives we can

construct the other two.)

The trick is to feed the message blocks one at a time to the key input of

our block cipher, and use it to update a hash value (which starts off at say

H0=0). In order to make this operation non-invertible, we add feedforward:

the (i−1)st hash value is exclusive or’ed with the output of round i. This is

our ﬁnal mode of operation of a block cipher (Figure 5.16).





⊕



•

hi−1

Figure 5.16: Feedforward mode (hash function)

166 Chapter 5 ■Cryptography

5.6.1 Extra Requirements on the Underlying Cipher

The birthday effect makes another appearance here, in that if a hash function h

is built using an nbit block cipher, it is possible to ﬁnd two messages M1= M2

with h(M1)=h(M2) with about 2

n/2effort (hash slightly more than that many

messages Miand look for a match). So a 64 bit block cipher is not adequate, as

the cost of forging a message would be of the order of 232 messages, which is

quite practical. A 128-bit cipher such as AES may be just about adequate, and

in fact the AACS content protection mechanism used in the next generation of

DVDs uses ‘AES-H’, the hash function derived from AES in this way.

The birthday limit is not the only way in which the hash function mode of

operation is more demanding on the underlying block cipher than a mode such

as CBC designed for conﬁdentiality. A good illustration comes from a cipher

called Treyfer which was designed to encrypt data using as little memory as

possible in the 8051 microcontrollers commonly found in consumer electronics

and domestic appliances [1371]. (It takes only 30 bytes of ROM.)

Treyfer ‘scavenges’ its S-box by using 256 bytes from the ROM, which may

be code, or even — to make commercial cloning riskier — contain a copyright

message. At each round, it acts on eight bytes of text with eight bytes of key

by adding a byte of text to a byte of key, passing it through the S-box, adding

it to the next byte and then rotating the result by one bit (see Figure 5.17).

This rotation deals with some of the problems that might arise if the S-box has

uneven randomness across its bitplanes (for example, if it contains ascii text

P0k0k1

<<<1

• • •

Figure 5.17: The basic component of the Treyfer block cipher

5.6 Hash Functions 167

such as a copyright message). Finally, the algorithm makes up for its simple

round structure and probably less than ideal S-box by having a large number

of rounds (32).

Treyfer can in theory be used for conﬁdentiality (although its effective

keylength is only 44 bits). However, the algorithm does have a weakness that

prevents its use in hash functions. It suffers from a ﬁxed-point attack.Given

any input, there is a fair chance we can ﬁnd a key which will leave the input

unchanged. We just have to look to see, for each byte of input, whether the

S-box assumes the output which, when added to the byte on the right, has

the effect of rotating it one bit to the right. If such outputs exist for each

of the input bytes, then it’s easy to choose key values which will leave the data

unchanged after one round, and thus after 32. The probability that we can do

this depends on the S-box3. This means that we can easily ﬁnd collisions if

Treyfer is used as a hash function. In effect, hash functions have to be based

on block ciphers which withstand chosen-key attacks.

5.6.2 Common Hash Functions and Applications

Algorithms similar to Treyfer have been used in hash functions in key manage-

ment protocols in some pay-TV systems, but typically they have a modiﬁcation

to prevent ﬁxed-point attacks, such as a procedure to add in the round number

at each round, or to mix up the bits of the key in some way (a key scheduling

algorithm).

The most commonly used hash functions are all cryptographically suspect.

They are based on variants of a block cipher with a 512 bit key and a block size

of either 128 or 160 bits:

MD4 has three rounds and a 128 bit hash value, and a collision was

found for it in 1998 [394];

MD5 has four rounds and a 128 bit hash value, and a collision was found

for it in 2004 [1315, 1317];

the US Secure Hash Standard has ﬁve rounds and a 160 bit hash value,

and it was shown in 2005 that a collision can be found with a computa-

tional effort of 269 steps rather than the 280 that one would hope given its

block size [1316].

The block ciphers underlying these hash functions are similar: their round

function is a complicated mixture of the register operations available on 32 bit

processors [1125].

3Curiously, an S-box which is a permutation is always vulnerable, while a randomly selected

one isn’t quite so bad. In many cipher designs, S-boxes which are permutations are essential or

at least desirable. Treyfer is an exception.

168 Chapter 5 ■Cryptography

MD5 was broken by Xiaoyun Wang and her colleagues in 2004 [1315, 1317];

collisions can now be found easily, even between strings containing meaningful

text and adhering to message formats such as those used for digital certiﬁcates.

Wang seriously dented SHA the following year, providing an algorithm that

will ﬁnd collisions in only 269 steps [1316]; and at the Crypto 2007 conference,

the view was that ﬁnding a collision should cost about 260. Volunteers were

being recruited for the task. So it appears that soon a collision will be found

and SHA-1 will be declared ‘broken’.

At the time of writing, the US National Institute of Standards and Technology

(NIST) recommends that people use extended block-size versions of SHA, such

as SHA-256 or SHA-512. The draft FIPS 180-3 allows, though discourages, the

original SHA; it speciﬁes SHA-256 and SHA-512, and also supports 224-bit and

384-bit hashes derived from SHA-256 and SHA-512 respectively by changing

the initial values and truncating the output. The NSA speciﬁes the use of SHA-

256 or SHA-382 along with AES in its Suite B of cryptographic algorithms for

defense use. NIST is also organising a competition to ﬁnd a replacement hash

function family [949].

Whether a collision-search algorithm that requires months of work on

hundreds of machines (or a few days on a large botnet) will put any given

application at risk can be a complex question. If bank systems would actually

take a message composed by a customer saying ‘Pay Xthe sum Y’, hash it and

sign it, then a weak hash function could indeed be exploited: a bad man could

ﬁnd two messages ‘Pay Xthe sum Y’ and ‘Pay Xthe sum Z’ that hashed to

the same value, get one signed, and swap it for the other. But bank systems

don’t work like that. They typically use MACs rather than digital signatures

on actual transactions, relying on signatures only in public-key certiﬁcates

that bootstrap key-management protocols; and as the public-key certiﬁcates

are generated by trusted CAs using fairly constrained algorithms, there isn’t

an opportunity to insert one text of a colliding pair. Instead you’d have to

ﬁnd a collision with an externally-given target value, which is a much harder

cryptanalytic task.

Hash functions have many uses. One of them is to compute MACs. A naive

method would be to simply hash the message with a key: MACk(M)=h(k,M).

However the accepted way of doing this, called HMAC, uses an extra step

in which the result of this computation is hashed again. The two hashing

operations are done using variants of the key, derived by exclusive-or’ing

them with two different constants. Thus HMACk(M)=h(k⊕A,h(k⊕B,M)). A

is constructed by repeating the byte 0x36 as often as necessary, and Bsimilarly

from the byte 0x5C. Given a hash function that may be on the weak side, this

is believed to make exploitable collisions harder to ﬁnd [741]. HMAC is now

FIPS 198, being replaced by FIPS 198-1.

5.6 Hash Functions 169

Another use of hash functions is to make commitments that are to be

revealed later. For example, I might wish to timestamp a digital document

in order to establish intellectual priority, but not reveal the contents yet. In

that case, I can submit a hash of the document to a commercial timestamping

service [572]. Later, when I reveal the document, the fact that its hash was

timestamped at a given time establishes that I had written it by then. Again,

an algorithm that generates colliding pairs doesn’t break this, as you have to

have the pair to hand when you do the timestamp. The moral, I suppose, is

that engineers should be clear about whether a given application needs a hash

function that’s strongly collision-resistant.

But even though there may be few applications where the ability to ﬁnd

collisions could enable a bad guy to steal real money today, the existence of a

potential vulnerability can still undermine a system’s value. In 2005, a motorist

accused of speeding in Sydney, Australia, was acquitted after the New South

Wales Roads and Trafﬁc Authority failed to ﬁnd an expert to testify that MD5

was secure. The judge was ‘‘not satisﬁed beyond reasonable doubt that the

photograph [had] not been altered since it was taken’’ and acquitted the

motorist; this ruling was upheld on appeal the following year [964]. So even if

a vulnerability doesn’t present an engineering threat, it can still present a very

real certiﬁcational threat.

Finally, before we go on to discuss asymmetric cryptography, there are

two particular uses of hash functions which need mention: key updating and

autokeying.

Key updating means that two or more principals who share a key pass it

through a one-way hash function at agreed times: Ki=h(Ki−1). The point is

that if an attacker compromises one of their systems and steals the key, he

only gets the current key and is unable to decrypt back trafﬁc. The chain of

compromise is broken by the hash function’s one-wayness. This property is

also known as backward security.

Autokeying means that two or more principals who share a key hash it

at agreed times with the messages they have exchanged since the last key

change: K+1i=h(Ki,Mi1,Mi2,...). The point is that if an attacker compromises

one of their systems and steals the key, then as soon as they exchange a

message which he doesn’t observe or guess, security will be recovered in

that he can no longer decrypt their trafﬁc. Again, the chain of compromise is

broken. This property is known as forward security. It is used, for example, in

EFT payment terminals in Australia [143, 145]. The use of asymmetric crypto

allows a slightly stronger form of forward security, namely that as soon as

a compromised terminal exchanges a message with an uncompromised one

which the opponent doesn’t control, then security can be recovered even if the

message is in plain sight. I’ll describe how this trick works next.

170 Chapter 5 ■Cryptography

5.7 Asymmetric Crypto Primitives

The commonly used building blocks in asymmetric cryptography, that is public

key encryption and digital signature, are based on number theory. I’ll give only

a brief overview here, and look in more detail at some of the mechanisms used

in Part II where I discuss applications. (If you ﬁnd the description assumes too

much mathematics, I’d suggest you skip the following two sections and read

up the material from a cryptography textbook.)

The technique is to make the security of the cipher depend on the difﬁculty of

solving a certain mathematical problem. The two problems which are used in

almost all ﬁelded systems are factorization (used in most commercial systems)

and discrete logarithm (used in many government systems).

5.7.1 Cryptography Based on Factoring

The prime numbers are the positive whole numbers with no proper divisors; that

is, the only numbers that divide a prime number are 1 and the number itself. By

deﬁnition, 1 is not prime; so the primes are {2, 3, 5, 7, 11, ...}. The fundamental

theorem of arithmetic states that each natural number greater than 1 factors into

prime numbers in a way that is unique up to the order of the factors. It is

easy to ﬁnd prime numbers and multiply them together to give a composite

number, but much harder to resolve a composite number into its factors. The

largest composite product of two large random primes to have been factorized

to date was RSA-200, a 663-bit number (200 decimal digits), factored in 2005.

This factorization was done on a number of PCs and took the equivalent of 75

years’ work on a single 2.2GHz machine. It is possible for factoring to be done

surreptitiously, perhaps using a botnet; in 2001, when the state of the art was

factoring 512-bit numbers, such a challenge was set in Simon Singh’s ‘Code

Book’ and solved by ﬁve Swedish students using several hundred computers

to which they had access [24]. By 2007, 512-bit factorization had entered into

mainstream commerce. From 2003, Intuit had protected its Quicken ﬁles with

strong encryption, but left a back door based on a 512-bit RSA key so that they

could offer a key recovery service. Elcomsoft appears to have factored this key

and now offers a competing recovery product.

It is believed that factoring an RSA modulus of 1024 bits would require a

special-purpose machine costing in the range of $10–50m and that would take

a year for each factorization [781]; but I’ve heard of no-one seriously planning

to build such a machine. Many physicists hope that a quantum computer could

be built that would make it easy to factor even large numbers. So, given that

Moore’s law is slowing down and that quantum computers haven’t arrived

yet, we can summarise the state of the art as follows. 1024-bit products of

two random primes are hard to factor and cryptographic systems that rely on

5.7 Asymmetric Crypto Primitives 171

them are at no immediate risk from low-to-medium budget attackers; NIST

expects them to be secure until 2010, while an extrapolation of the history of

factoring records suggests the ﬁrst factorization will be published in 2018. So

risk-averse organisations that want keys to remain secure for many years are

already using 2048-bit numbers.

The algorithm commonly used to do public-key encryption and digital

signatures based on factoring is RSA, named after its inventors Ron Rivest,

Adi Shamir and Len Adleman. It uses Fermat’s (little) theorem, which states that

for all primes pnot dividing a,ap−1≡1(modp) (proof: take the set {1, 2, ...,

p−1}and multiply each of them modulo pby a, then cancel out (p−1)! each

side). Euler’s function φ(n) is the number of positive integers less than nwith

which it has no divisor in common; so if nis the product of two primes pq then

φ(n)=(p−1)(q−1) (the proof is similar).

The encryption key is a modulus Nwhich is hard to factor (take N=pq for

two large randomly chosen primes pand q, say of 1024 bits each) plus a public

exponent ethat has no common factors with either p−1orq−1. The private

key is the factors pand q, which are kept secret. Where Mis the message and

Cis the ciphertext, encryption is deﬁned by

C≡Me(mod N)

Decryption is the reverse operation:

M≡e

√C(mod N)

Whoever knows the private key — the factors pand qof N— can easily cal-

culate e

√C(mod N). As φ(N)=(p−1)(q−1) and ehas no common factors with

φ(N), the key’s owner can ﬁnd a number dsuch that de ≡1(modφ(N)) — she

ﬁnds the value of dseparately modulo p−1andq−1, and combines the

answers. e

√C(mod N) is now computed as Cd(mod N), and decryption works

because of Fermat’s theorem:

Cd≡{Me}d≡Med ≡M1+kφ(N)≡M.Mkφ(N)≡M.1≡M(mod N)

Similarly, the owner of a private key can operate on a message with this to

produce a signature

Sigd(M)≡Md(mod N)

and this signature can be veriﬁed by raising it to the power emod N(thus,

using eand Nas the public signature veriﬁcation key) and checking that the

message Mis recovered:

M≡(Sigd(M))e(mod N)

Neither RSA encryption nor signature is generally safe to use on its own.

The reason is that, as encryption is an algebraic process, it preserves certain

algebraic properties. For example, if we have a relation such as M1M2=M3

172 Chapter 5 ■Cryptography

that holds among plaintexts, then the same relationship will hold among

ciphertexts C1C2=C3and signatures Sig1Sig2=Sig3. This property is known

as a multiplicative homomorphism; a homomorphism is a function that preserves

some mathematical structure. The homomorphic nature of raw RSA means that

it doesn’t meet the random oracle model deﬁnitions of public key encryption

or signature.

Another problem with public-key encryption is that if the plaintexts are

drawn from a small set, such as ‘attack’ or ‘retreat’, and the encryption process

is known to the opponent, then he can precompute possible ciphertexts and

recognise them when they appear. Speciﬁc algorithms also have speciﬁc

vulnerabilities: with RSA, it’s dangerous to use a small exponent eto encrypt

the same message to multiple recipients, as this can lead to an algebraic

attack. To stop the guessing attack, the low-exponent attack and attacks

based on homomorphism, it’s sensible to add in some randomness, and some

redundancy, into a plaintext block before encrypting it. However, there are

good ways and bad ways of doing this.

In fact, crypto theoreticians have wrestled for decades to analyze all the

things that can go wrong with asymmetric cryptography, and to ﬁnd ways to

tidy it up. Shaﬁ Goldwasser and Silvio Micali came up with formal models of

probabilistic encryption in which we add randomness to the encryption process,

and semantic security, which means that an attacker cannot get any information

at all about a plaintext Mthat was encrypted to a ciphertext C,evenifhe

is allowed to request the decryption of any other ciphertext Cnot equal

to C[536]. There are a number of constructions that give provable semantic

security, but they tend to be too ungainly for practical use.

The common real-world solution is optimal asymmetric encryption padding

(OAEP), where we concatenate the message Mwith a random nonce N,and

use a hash function hto combine them:

C1=M⊕h(N)

C2=N⊕h(C1)

In effect, this is a two-round Feistel cipher that uses has its round function.

The result, the combination C1,C2, is then encrypted with RSA and sent. The

recipient then computes Nas C2⊕h(C1) and recovers Mas C1⊕h(N) [148].

(This construction came with a security proof, in which a mistake was sub-

sequently found [1167, 234], sparking a vigorous debate on the value of

mathematical proofs in security engineering [724].) RSA Data Security, which

for years licensed the RSA algorithm, developed a number of public-key

cryptography standards; PKCS #1 describes OAEP [672].

With signatures, things are slightly simpler. In general, it’s often enough

to just hash the message before applying the private key: Sigd=[h(M)]d

(mod N); PKCS #7 describes simple mechanisms for signing a message

5.7 Asymmetric Crypto Primitives 173

digest [680]. However, in some applications one might wish to include further

data in the signature block, such as a timestamp, or some randomness in order

to make side-channel attacks harder.

Many of the things that have gone wrong with real implementations have

to do with error handling. Some errors can affect cryptographic mechanisms

directly. The most spectacular example was when Daniel Bleichenbacher found

a way to break the RSA implementation in SSL v 3.0 by sending suitably chosen

ciphertexts to the victim and observing any resulting error messages. If he

can learn from the target whether a given c, when decrypted as cd(mod n),

corresponds to a PKCS #1 message, then he can use this to decrypt or sign

messages [189]. Other attacks have depended on measuring the precise time

taken to decrypt; I’ll discuss these in the chapter on emission security. Yet

others have involved stack overﬂows, whether by sending the attack code in

as keys, or as padding in poorly-implemented standards. Don’t assume that

the only attacks on your crypto code will be doing cryptanalysis.

5.7.2 Cryptography Based on Discrete Logarithms

While RSA is used in most web browsers in the SSL protocol, and in the SSH

protocol commonly used for remote login to computer systems, there are other

products, and many government systems, which base public key operations on

discrete logarithms. These come in a number of ﬂavors, some using ‘normal’

arithmetic while others use mathematical structures called elliptic curves.I’ll

explain the normal case. The elliptic variants use essentially the same idea but

the implementation is more complex.

Aprimitive root modulo pis a number whose powers generate all the nonzero

numbers mod p; for example, when working modulo 7 we ﬁnd that 52=25

which reduces to 4 (modulo 7), then we can compute 53as 52×5or4×5

which is 20, which reduces to 6 (modulo 7), and so on, as in Figure 5.18:

51=5 (mod7)

52=25≡4(mod7)

53≡4x5 ≡6(mod7)

54≡6x5 ≡2(mod7)

55≡2x5 ≡3(mod7)

56≡3x5 ≡1(mod7)

Figure 5.18: Example of discrete logarithm calculations

Thus 5 is a primitive root modulo 7. This means that given any y, we can

always solve the equation y=5x(mod 7); xis then called the discrete logarithm

of ymodulo 7. Small examples like this can be solved by inspection, but for a

large random prime number p, we do not know how to do this computation.

174 Chapter 5 ■Cryptography

So the mapping f:x→gx(mod p) is a one-way function, with the additional

properties that f(x+y)=f(x)f(y)andf(nx)=f(x)n. In other words, it is a

one-way homomorphism. As such, it can be used to construct digital signature

and public key encryption algorithms.

5.7.2.1 Public Key Encryption — Difﬁe Hellman and ElGamal

To understand how discrete logarithms can be used to build a public-key

encryption algorithm, bear in mind that we want a cryptosystem which does

not need the users to start off with a shared secret key. Consider the following

‘classical’ scenario.

Imagine that Anthony wants to send a secret to Brutus, and the only

communications channel available is an untrustworthy courier (say, a slave

belonging to Caesar). Anthony can take the message, put it in a box, padlock it,

and get the courier to take it to Brutus. Brutus could then put his own padlock

on it too, and have it taken back to Anthony. He in turn would remove his

padlock, and have it taken back to Brutus, who would now at last open it.

Exactly the same can be done using a suitable encryption function that

commutes, that is, has the property that {{M}KA}KB ={{M}KB}KA. Alice can take

the message Mand encrypt it with her key KA to get {M}KA which she sends

to Bob. Bob encrypts it again with his key KB getting {{M}KA}KB. But the

commutativity property means that this is just {{M}KB}KA, so Alice can decrypt

it using her key KA getting {M}KB. She sends this to Bob and he can decrypt

it with KB, ﬁnally recovering the message M. The keys KA and KB might be

long-term keys if this mechanism were to be used as a conventional public-key

encryption system, or they might be transient keys if the goal were to establish

a key with forward secrecy.

How can a suitable commutative encryption be implemented? The one-time

pad does commute, but is not suitable here. Suppose Alice chooses a random

key xA and sends Bob M⊕xA while Bob returns M⊕xB and Alice ﬁnally

sends him M⊕xA ⊕xB, then an attacker can simply exclusive-or these three

messages together; as X⊕X=0forallX, the two values of xA and xB both

cancel our leaving as an answer the plaintext M.

The discrete logarithm problem comes to the rescue. If the discrete log

problem based on a primitive root modulo pis hard, then we can use discrete

exponentiation as our encryption function. For example, Alice encodes her

message as the primitive root g, chooses a random number xA, calculates

gxA modulo pand sends it, together with p, to Bob. Bob likewise chooses

a random number xB and forms gxAxB modulo p, which he passes back to

Alice. Alice can now remove her exponentiation: using Fermat’s theorem, she

calculates gxB =(gxAxB)(p−xA)(mod p) and sends it to Bob. Bob can now remove

his exponentiation, too, and so ﬁnally gets hold of g. The security of this scheme

depends on the difﬁculty of the discrete logarithm problem. In practice, it is

5.7 Asymmetric Crypto Primitives 175

tricky to encode a message to be a primitive root; but there is a much simpler

means of achieving the same effect. The ﬁrst public key encryption scheme

to be published, by Whitﬁeld Difﬁe and Martin Hellman in 1976, has a ﬁxed

primitive root gand uses gxAxB modulo pas the key to a shared-key encryption

system. The values xA and xB can be the private keys of the two parties.

Let’s see how this might provide a public-key encryption system. The prime

pand generator gare common to all users. Alice chooses a secret random

number xA, calculates yA =gxA and publishes it opposite her name in the

company phone book. Bob does the same, choosing a random number xBand

publishing yB =gxB. In order to communicate with Bob, Alice fetches yB from

the phone book, forms yBxA which is just gxAxB, and uses this to encrypt the

message to Bob. On receiving it, Bob looks up Alice’s public key yAand forms

yAxB which is also equal to gxAxB, so he can decrypt her message.

Slightly more work is needed to provide a full solution. Some care is needed

when choosing the parameters pand g; and there are several other details

which depend on whether we want properties such as forward security.

Variants on the Difﬁe-Hellman theme include the US government key exchange

algorithm (KEA) [939], used in network security products such as the Fortezza

card, and the so-called Royal Holloway protocol, which is used by the UK

government [76].

Of course, one of the big problems with public-key systems is how to be

sure that you’ve got a genuine copy of the phone book, and that the entry

you’re interested in isn’t out of date. I’ll discuss that in section 5.7.5.

5.7.2.2 Key Establishment

Mechanisms for providing forward security in such protocols are of indepen-

dent interest, As before, let the prime pand generator gbe common to all

users. Alice chooses a random number RA, calculates gRAand sends it to Bob;

Bob does the same, choosing a random number RBand sending gRBto Alice;

they then both form gRARB, which they use as a session key (Figure 5.19).

A→B:gRA(mod p)

B→A:gRB(mod p)

A→B:{M}gRARB

Figure 5.19: The Difﬁe-Hellman key exchange protocol

Alice and Bob can now use the session key gRARBto encrypt a conversation.

They have managed to create a shared secret ‘out of nothing’. Even if an

opponent had obtained full access to both their machines before this protocol

was started, and thus knew all their stored private keys, then provided some

basic conditions were met (e.g., that their random number generators were

176 Chapter 5 ■Cryptography

not predictable) the opponent could still not eavesdrop on their trafﬁc. This

is the strong version of the forward security property to which I referred in

section 5.6.2. The opponent can’t work forward from knowledge of previous

keys which he might have obtained. Provided that Alice and Bob both destroy

the shared secret after use, they will also have backward security: an opponent

who gets access to their equipment subsequently cannot work backward to

break their old trafﬁc.

But this protocol has a small problem: although Alice and Bob end up with

a session key, neither of them has any idea who they share it with.

Suppose that in our padlock protocol Caesar had just ordered his slave

to bring the box to him instead, and placed his own padlock on it next to

Anthony’s. The slave takes the box back to Anthony, who removes his padlock,

and brings the box back to Caesar who opens it. Caesar can even run two

instances of the protocol, pretending to Anthony that he’s Brutus and to Brutus

that he’s Anthony. One ﬁx is for Anthony and Brutus to apply their seals to

their locks.

With the vanilla Difﬁe-Hellman protocol, the same idea leads to a mid-

dleperson attack. Charlie intercepts Alice’s message to Bob and replies to it; at

the same time, he initiates a key exchange with Bob, pretending to be Alice.

He ends up with a key gRARCwhich he shares with Alice, and another key gRBRC

which he shares with Bob. So long as he continues to sit in the middle of the

network and translate the messages between them, they may have a hard time

detecting that their communications are compromised. The usual solution is

to authenticate transient keys, and there are various possibilities.

In one secure telephone product, the two principals would read out an eight

digit hash of the key they had generated and check that they had the same

value before starting to discuss classiﬁed matters. A more general solution is

for Alice and Bob to sign the messages that they send to each other.

A few other details have to be got right, such as a suitable choice of the

values pand g. There’s some non-trivial mathematics behind this, which is best

left to specialists. There are also many things that can go wrong in implemen-

tations — examples being software that will generate or accept very weak keys

and thus give only the appearance of protection; programs that leak the key by

the amount of time they take to decrypt; and software vulnerabilities leading

to stack overﬂows and other nasties. Nonspecialists implementing public-key

cryptography should consult up-to-date standards documents and/or use

properly accredited toolkits.

5.7.2.3 Digital Signature

Suppose that the base pand the generator gare public values chosen in some

suitable way, and that each user who wishes to sign messages has a private

signing key Xand a public signature veriﬁcation key Y=gX. An ElGamal

5.7 Asymmetric Crypto Primitives 177

signature scheme works as follows. Choose a message key kat random, and

form r=gk(mod p). Now form the signature susing a linear equation in k,r,

the message Mand the private key X. There are a number of equations that

will do; the particular one that happens to be used in ElGamal signatures is

rX +sk =M

So sis computed as s=(M−rX)/k; this is done modulo φ(p). When both

sides are passed through our one-way homomorphism f(x)=gxmod pwe get:

grXgsk ≡gM

Yrrs≡gM

An ElGamal signature on the message Mconsists of the values rand s,and

the recipient can verify it using the above equation.

A few more details need to be ﬁxed up to get a functional digital signature

scheme. As before, bad choices of pand gcan weaken the algorithm. We will

also want to hash the message Musing a hash function so that we can sign

messages of arbitrary length, and so that an opponent can’t use the algorithm’s

algebraic structure to forge signatures on messages that were never signed.

Having attended to these details and applied one or two optimisations, we get

the Digital Signature Algorithm (DSA) which is a US standard and widely used

in government applications.

DSA (also known as DSS, for Digital Signature Standard) assumes a prime

pof typically 1024 bits, a prime qof 160 bits dividing (p−1), an element gof

order qin the integers modulo p, a secret signing key xand a public veriﬁcation

key y=gx. The signature on a message M,Sigx(M), is (r,s)where

r≡(gk(mod p)) (mod q)

s≡(h(M)−xr)/k(mod q)

The hash function used here is SHA1.

DSA is the classic example of a randomized digital signature scheme without

message recovery. The standard has changed somewhat with faster computers,

as variants of the algorithm used to factor large numbers can also be used to

compute discrete logarithms modulo bases of similar size4. Initially the prime

pcould be in the range 512–1024 bits, but this was changed to 1023–1024

bits in 2001 [941]; the proposed third-generation standard will allow primes

pin the range 1024–3072 bits and qin the range 160–256 bits [942]. Further

tweaks to the standard are also foreseeable after a new hash function standard

is adopted.

4Discrete log efforts lag slightly behind, with a record set in 2006 of 440 bits.

178 Chapter 5 ■Cryptography

5.7.3 Special Purpose Primitives

Researchers have discovered a large number of public-key and signature

primitives with special properties. Two that have so far appeared in real

products are threshold cryptography and blind signatures.

Threshold crypto is a mechanism whereby a signing key, or a decryption key,

can be split up among nprincipals so that any kout of ncan sign a message (or

decrypt). For k=nthe construction is easy. With RSA, for example, you can

split up the private key das d=d1+d2+...+dn.Fork<nit’s slightly more

complex (but not much — you use the Lagrange interpolation formula) [382].

Threshold signatures are used in systems where a number of servers process

transactions independently and vote independently on the outcome; they

could also be used to implement business rules such as ‘a check may be signed

by any two of the seven directors’.

Blind signatures are a way of making a signature on a message without

knowing what the message is. For example, if we are using RSA, I can take a

random number R,formReM(mod n), and give it to the signer who computes

(ReM)d=R.Md(mod n). When he gives this back to me, I can divide out R

to get the signature Md. Now you might ask why on earth someone would

want to sign a document without knowing its contents, but there are indeed

applications.

The ﬁrst was in digital cash; a bank might want to be able to issue anonymous

payment tokens to customers, and this has been done by getting it to sign

‘digital coins’ without knowing their serial numbers. In such a system, the

bank might agree to honour for $10 any string Mwith a unique serial number

and a speciﬁed form of redundancy, bearing a signature that veriﬁed as correct

using the public key (e,n). The blind signature protocol shows how a customer

can get a bank to sign a coin without the banker knowing its serial number. The

effect is that the digital cash can be anonymous for the spender. (There are a few

technical details that need to be sorted out, such as how you detect people who

spend the same coin twice; but these are ﬁxable.) Blind signatures and digital

cash were invented by Chaum [285], along with much other supporting digital

privacy technology which I’ll discuss later [284]. They were used brieﬂy in pilot

projects for road tolls in the Netherlands and for electronic purses in Brussels,

but failed to take off on a broader scale because of patent issues and because

neither banks nor governments really want payments to be anonymous: the

anti-money-laundering regulations nowadays restrict anonymous payment

services to rather small amounts. Anonymous digital credentials are now

talked about, for example, in the context of ‘identity management’: the TPM

chip on your PC motherboard might prove something about you (such as your

age) without actually revealing your name.

Researchers continue to suggest new applications for specialist public key

mechanisms. A popular candidate is in online elections, which require a

5.7 Asymmetric Crypto Primitives 179

particular mixture of anonymity and accountability. Voters want to be sure

that their votes have been counted, but it’s also desirable that they should

not be able to prove which way they voted to anybody else; if they can, then

vote-buying and intimidation become easier.

5.7.4 Elliptic Curve Cryptography

Finally, discrete logarithms and their analogues exist in many other mathe-

matical structures; thus for example elliptic curve cryptography uses discrete

logarithms on an elliptic curve — a curve given by an equation like y2=

x3+ax +b. These curves have the property that you can deﬁne an addition

operation on them and use it for cryptography; the algebra gets a bit complex

and a general book like this isn’t the place to set it out. However, elliptic curve

cryptosystems are interesting for two reasons.

First, they give versions of the familiar primitives such as Difﬁe-Hellmann

key exchange and the Digital Signature Algorithm that use less computation,

and also have slightly shorter variables; both can be welcome in constrained

environments such as smartcards and embedded processors. Elliptic curve

cryptography is used, for example, in the rights-management mechanisms of

Windows Media Player, and has been adopted as a standard by the NSA for

use in defense systems.

Second, some elliptic curves have a bilinear pairing which Dan Boneh and

Matt Franklin used to construct cryptosystems where your public key is your

name [207]. Recall that in RSA and Difﬁe-Hellmann, the user chose his private

key and then computed a corresponding public key. In a so-called identity-

based cryptosystem, you choose your identity then go to a central authority that

issues you with a private key corresponding to that identity. There is a global

public key, with which anyone can encrypt a message to your identity; you

can decrypt this using your private key. Earlier, Adi Shamir had discovered

identity-based signature schemes that allow you to sign messages using a private

key so that anyone can verify the signature against your name [1147]. In both

cases, your private key is computed by the central authority using a system-

wide private key known only to itself. Identity-based primitives could have

interesting implications for specialist systems, but in the context of ordinary

public-key and signature systems they achieve much the same result as the

certiﬁcation of public keys, which I’ll discuss next.

5.7.5 Certiﬁcation

Now that we can do public-key encryption and digital signature, we need

some mechanism to bind users to keys. The approach proposed by Difﬁe and

Hellman when they invented digital signatures was to have a directory of the

public keys of a system’s authorized users, like a phone book. A more common

180 Chapter 5 ■Cryptography

solution, due to Loren Kohnfelder, is for a certiﬁcation authority (CA) to sign the

users’ public encryption and/or signature veriﬁcation keys giving certiﬁcates

that contain the user’s name, attributed such as authorizations, and public

keys. The CA might be run by the local system administrator; or it might be a

third party service such as Verisign whose business is to sign public keys after

doing some due diligence about whether they belong to the principals named

in them.

A certiﬁcate might be described symbolically as

CA=SigKS(TS,L,A,KA,VA) (5.1)

where (using the same notation as with Kerberos) TSis the certiﬁcate’s start-

ing date and time, Lis the length of time for which it is valid, A is the user’s

name, KAis her public encryption key, and VAis her public signature veriﬁ-

cation key. In this way, only the administrator’s public signature veriﬁcation

key needs to be communicated to all principals in a trustworthy manner.

Certiﬁcation is hard, for a whole lot of reasons. I’ll discuss different aspects

later — naming in Chapter 6, ‘Distributed Systems’, public-key infrastructures

in Chapter 21, ‘Network Attack and Defense’, and the policy aspects in Part III.

Here I’ll merely point out that the protocol design aspects are much harder

than they look.

One of the ﬁrst proposed public-key protocols was due to Dorothy Denning

and Giovanni Sacco, who in 1982 proposed that two users, say Alice and Bob,

set up a shared key KAB as follows. When Alice ﬁrst wants to communicate

with Bob, she goes to the certiﬁcation authority and gets current copies of

public key certiﬁcates for herself and Bob. She then makes up a key packet

containing a timestamp TA, a session key KAB and a signature, which she

computes on these items using her private signing key. She then encrypts this

whole bundle under Bob’s public key and ships it off to him. Symbolically,

A→B:CA,CB,{TA,KAB,SigKA(TA,KAB)}KB(5.2)

In 1994, Mart´

ın Abadi and Roger Needham pointed out that this protocol

is fatally ﬂawed [2]. Bob, on receiving this message, can masquerade as Alice

for as long as Alice’s timestamp TAremains valid! To see how, suppose that

Bob wants to masquerade as Alice to Charlie. He goes to Sam and gets a

fresh certiﬁcate CCfor Charlie, and then strips off the outer encryption {...}KB

from message 3 in the above protocol. He now re-encrypts the signed key

packet TA,KAB,SigKA(TA,KAB) with Charlie’s public key — which he gets from

CC— and makes up a bogus message 3:

B→C:CA,CC,{TA,KAB,SigKA(TA,KAB)}KC(5.3)

It is quite alarming that such a simple protocol — essentially, a one line

program — should have such a serious ﬂaw remain undetected for so long.

With a normal program of only a few lines of code, you might expect to ﬁnd a

5.7 Asymmetric Crypto Primitives 181

bug in it by looking at it for a minute or two. In fact, public key protocols are

if anything harder to design than protocols that use shared-key encryption,

as they are prone to subtle and pernicious middleperson attacks. This further

motivates the use of formal methods to prove that protocols are correct.

Often, the participants’ names aren’t the most important things the authen-

tication mechanism has to establish. In the STU-III secure telephone used by

the US government and defense contractors, there is a protocol for establishing

transient keys with forward and backward security; to exclude middleperson

attacks, users have a crypto ignition key, a portable electronic device that they

plug into the phone to identify not just their names, but their security clearance

level. In general, textbooks tend to talk about identiﬁcation as the main goal

of authentication and key management protocols; but in real life, it’s usually

authorization that matters. This is more complex, as it starts to introduce

assumptions about the application into the protocol design. (In fact, the NSA

security manual emphasises the importance of always knowing whether there

is an uncleared person in the room. The STU-III design is a natural way of

extending this to electronic communications.)

One serious weakness of relying on public-key certiﬁcates is the difﬁculty of

getting users to understand all their implications and manage them properly,

especially where they are not an exact reimplementation of a familiar manual

control system [357]. There are many other things that can go wrong with

certiﬁcation at the level of systems engineering, which I’ll start to look at in

the next chapter.

5.7.6 The Strength of Asymmetric Cryptographic

Primitives

In order to provide the same level of protection as a symmetric block cipher,

asymmetric cryptographic primitives generally require at least twice the block

length. Elliptic curve systems appear to achieve this bound; a 128-bit elliptic

scheme could be about as hard to break as a 64-bit block cipher with a 64-bit

key; and the only public-key encryption schemes used in the NSA’s Suite B of

military algorithms are 256- and 384-bit elliptic curve systems. The commoner

schemes, based on factoring and discrete log, are less robust because there

are shortcut attack algorithms such as the number ﬁeld sieve that exploit

the fact that some integers are smooth, that is, they have a large number of

small factors. When I wrote the ﬁrst edition of this book in 2000, the number

ﬁeld sieve had been used to attack keys up to 512 bits, a task comparable in

difﬁculty to keysearch on 56-bit DES keys; by the time I rewrote this chapter

for the second edition in 2007, 64-bit symmetric keys had been brute-forced,

and the 663-bit challenge number RSA-200 had been factored. The advance

in factoring has historically been due about equally to better hardware and

better algorithms. I wrote in 2000 that ‘The current consensus is that private

182 Chapter 5 ■Cryptography

keys for RSA and for standard discrete log systems should be at least 1024 bits

long, while 2048 bits gives some useful safety margin’; now in 2007, 1024-bit

RSA is widely believed to give about the same protection as 80-bit symmetric

keys, and designers are starting to move to 2048 bits for keys intended to last

many years. As I mentioned above, an extrapolation of recent factoring results

suggests that it might be a decade before we see a 1024-bit challenge factored —

although with Moore’s law starting to slow down, it might take much longer.

No-one really knows. (However I expect to see 768-bit RSA factored within a

few years.)

There has been much research into quantum computers — devices that per-

form a large number of computations simultaneously using superposed

quantum states. Peter Shor has shown that if a sufﬁciently large quantum

computer can be built, then both factoring and discrete logarithm compu-

tations will become easy [1165]. So far only very small quantum computers

can be built; factoring 15 is about the state of the art in 2007. Many people

are sceptical about whether the technology can be scaled up to threaten real

systems. But if it does, then asymmetric cryptography may have to change

radically. So it is fortunate that many of the things we currently do with asym-

metric mechanisms can also be done with symmetric ones; most authentication

protocols in use could be redesigned to use variants on Kerberos.

5.8 Summary

Many ciphers fail because they’re used improperly, so we need a clear model

of what a cipher does. The random oracle model provides a useful intuition:

we assume that each new value returned by the encryption engine is random

in the sense of being statistically independent of all the different outputs seen

before.

Block ciphers for symmetric key applications can be constructed by the

careful combination of substitutions and permutations; for asymmetric appli-

cations such as public key encryption and digital signature one uses number

theory. In both cases, there is quite a large body of mathematics. Other kinds

of ciphers — stream ciphers and hash functions — can be constructed from

block ciphers by using them in suitable modes of operation. These have

different error propagation, pattern concealment and integrity protection

properties.

The basic properties that the security engineer needs to understand are not

too difﬁcult to grasp, though there are many subtle things that can go wrong.

In particular, it is surprisingly hard to build systems that are robust even

when components fail (or are encouraged to) and where the cryptographic

mechanisms are well integrated with other measures such as access control

and physical security. I’ll return to this repeatedly in later chapters.

Further Reading

The classic papers by Whit Difﬁe and Martin Hellman [385] and by Ron

Rivest, Adi Shamir and Len Adleman [1078] are the closest to required reading

in this subject. The most popular introduction is Bruce Schneier’s Applied

Cryptography [1125] which covers a lot of ground at a level a non-mathematician

can understand, but is slightly dated. Alfred Menezes, Paul van Oorshot and

Scott Vanstone’s Handbook of Applied Cryptography [872] is the closest to a

standard reference book on the mathematical detail. For an appreciation of the

recent history of cryptanalysis, try Mark Stamp and Richard Low’s ‘Applied

Cryptanalysis’ [1214]: this has recent attacks on ﬁelded ciphers such as PKZIP,

RC4, CMEA and MD5.

There are many more specialised references. The bible on differential crypt-

analysis is a book by its inventors Eli Biham and Adi Shamir [170], while a good

short tutorial on linear and differential cryptanalysis was written by Howard

Heys [602]. A textbook by Doug Stinson has another detailed explanation of

linear cryptanalysis [1226]; and the modern theory of block ciphers can be

traced through the papers in the Fast Software Encryption conference series. The

original book on modes of operation is by Carl Meyer and Steve Matyas [880].

Neal Koblitz has a good basic introduction to the mathematics behind public

key cryptography [723]; and the number ﬁeld sieve is described in [780].

184 Chapter 5 ■Cryptography

There’s a shortage of good books on the random oracle model and on

theoretical cryptology in general: all the published texts I’ve seen are very

technical and heavy going. Probably the most regarded source is a book by

Oded Goldreich [535] but this is pitched at the postgraduate maths student. A

less thorough but more readable introduction to randomness and algorithms

is in [564]. Current research at the theoretical end of cryptology is found at the

FOCS, STOC, Crypto, Eurocrypt and Asiacrypt conferences.

Four of the simple block cipher modes of operation (ECB, CBC, OFB and

CFB) date back to FIPS-81; their speciﬁcation was reissued, with CTR mode

added, in 2001 as NIST Special Publication 800-38A [944]. The compound

modes of operation are described in subsequent papers in that series.

The history of cryptology is fascinating, and so many old problems keep on

recurring in modern guises that the security engineer should be familiar with

it. The standard work is Kahn [676]; there are also compilations of historical

articles from Cryptologia [363, 361, 362] as well as several books on the history

of cryptology in World War 2 [296, 677, 836, 1336]. The NSA Museum at Fort

George Meade, Md., is also worth a visit, as is the one at Bletchley Park in

England.

Finally, no chapter that introduces public key encryption would be complete

without a mention that, under the name of ‘non-secret encryption,’ it was ﬁrst

discovered by James Ellis in about 1969. However, as Ellis worked for GCHQ

(Britain’s Government Communications Headquarters, the equivalent of the

NSA) his work remained classiﬁed. The RSA algorithm was then invented by

Clifford Cocks, and also kept secret. This story is told in [427]. One effect of

the secrecy was that their work was not used: although it was motivated by the

expense of Army key distribution, Britain’s Ministry of Defence did not start

building electronic key distribution systems for its main networks until 1992.

It should also be noted that the classiﬁed community did not pre-invent digital

signatures; they remain the achievement of Whit Difﬁe and Martin Hellman.

CHAPTER

Distributed Systems

You know you have a distributed system when the crash of a

computer you’ve never heard of stops you from

getting any work done.

— Leslie Lamport

6.1 Introduction

We’ve seen in the last few chapters how people can authenticate themselves to

systems (and systems can authenticate themselves to each other) using security

protocols; how access controls can be used to manage which principals can

perform what operations in a system; and some of the mechanics of how crypto

can be used to underpin access control in distributed systems. But there’s much

more to building a secure distributed system than just implementing access

controls, protocols and crypto. When systems become large, the scale-up

problems are not linear; there is often a qualitative change in complexity, and

some things that are trivial to deal with in a network of only a few machines

and principals (such as naming) suddenly become a big deal.

Over the last 40 years, computer science researchers have built many

distributed systems and studied issues such as concurrency, failure recovery

and naming. The theory is supplemented by a growing body of experience from

industry, commerce and government. These issues are central to the design

of effective secure systems but are often handled rather badly. I’ve already

described attacks on security protocols that can be seen as concurrency failures.

If we replicate data to make a system fault-tolerant then we may increase the

risk of a compromise of conﬁdentiality. Finally, naming is a particularly

thorny problem. Many governments and organisations are trying to build

185

186 Chapter 6 ■Distributed Systems

larger, ﬂatter namespaces — using identity cards to number citizens and using

RFID to number objects — and yet naming problems undermined attempts

during the 1990s to build useful public key infrastructures.

6.2 Concurrency

Processes are said to be concurrent if they run at the same time, and concurrency

gives rise to a number of well-studied problems. Processes may use old data;

they can make inconsistent updates; the order of updates may or may not

matter; the system might deadlock; the data in different systems might never

converge to consistent values; and when it’s important to make things happen

in the right order, or even to know the exact time, this can be harder than you

might think.

Systems are now rapidly becoming more concurrent. First, the scale of

online business has grown rapidly; Google may have started off with four

machines but now its server farms have hundreds of thousands. Second,

devices are becoming more complex; a luxury car can now contain over forty

different processors. Third, the components are also getting more complex:

the microprocessor in your PC may now have two, or even four, CPU cores,

and will soon have more, while the graphics card, disk controller and other

accessories all have their own processors too. On top of this, virtualization

technologies such as VMware and Xen may turn a handful of real CPUs into

hundreds or even thousands of virtual CPUs.

Programming concurrent systems is hard; and, unfortunately, most of the

textbook examples come from the relatively rareﬁed world of operating system

internals and thread management. But concurrency control is also a security

issue. Like access control, it exists in order to prevent users interfering with

each other, whether accidentally or on purpose. Also, concurrency problems

can occur at many levels in a system, from the hardware right up to the business

environment. In what follows, I provide a number of concrete examples of the

effects of concurrency on security. These are by no means exhaustive.

6.2.1 Using Old Data Versus Paying to Propagate State

I’ve already described two kinds of concurrency problem. First, there are

replay attacks on protocols, where an attacker manages to pass off out-of-

date credentials. Secondly, there are race conditions. I mentioned the ‘mkdir’

vulnerability from Unix, in which a privileged instruction that is executed in

two phases could be attacked halfway through the process by renaming an

object on which it acts. These problems have been around for a long time.

In one of the ﬁrst multiuser operating systems, IBM’s OS/360, an attempt to

6.2 Concurrency 187

open a ﬁle caused it to be read and its permissions checked; if the user was

authorized to access it, it was read again. The user could arrange things so that

the ﬁle was altered in between [774].

These are examples of a time-of-check-to-time-of-use (TOCTTOU) attack. There

are systematic ways of ﬁnding such attacks in ﬁle systems [176], but as more

of our infrastructure becomes concurrent, attacks crop up at other levels

such as system calls in virtualised environments, which may require different

approaches. (I’ll discuss this speciﬁc case in detail in Chapter 18.) They also

appear at the level of business logic. Preventing them isn’t always economical,

as propagating changes in security state can be expensive.

For example, the banking industry manages lists of all hot credit cards

(whether stolen or abused) but there are millions of them worldwide, so it

isn’t possible to keep a complete hot card list in every merchant terminal, and

it would be too expensive to verify all transactions with the bank that issued

the card. Instead, there are multiple levels of stand-in processing. Terminals

are allowed to process transactions up to a certain limit (the ﬂoor limit)ofﬂine;

larger transactions need online veriﬁcation with a local bank, which will know

about all the local hot cards plus foreign cards that are being actively abused;

above another limit there might be a reference to an organization such as VISA

with a larger international list; while the largest transactions might need a

reference to the card issuer. In effect, the only transactions that are checked

immediately before use are those that are local or large.

Credit card systems are interesting as the largest systems that manage

the global propagation of security state — which they do by assuming that

most events are local, of low value, or both. They taught us that revoking

compromised credentials quickly and on a global scale was expensive. In the

1990s, when people started to build infrastructures of public key certiﬁcates to

support everything from web shopping to corporate networks, there was a fear

that biggest cost would be revoking the credentials of principals who changed

address, changed job, had their private key hacked, or got ﬁred. This turned

out not to be the case in general1. Another aspect of the costs of revocation can

be seen in large web services, where it would be expensive to check a user’s

credentials against a database every time she visits any one of the service’s

thousands of machines. A common solution is to use cookies — giving the

user an encrypted credential that her browser automatically presents on each

visit. That way only the key has to be shared between the server farm’s many

machines. However, if revoking users quickly is important to the application,

some other method needs to be found to do this.

1Frauds against web-based banking and shopping services don’t generally involve compromised

certiﬁcates. However, one application where revocation is a problem is the Deparatment of

Defense, which has issued 16 million certiﬁcates to military personnel since 1999 and now has

a list of 10 million revoked certiﬁcates that must be downloaded to all security servers every

day [878].

188 Chapter 6 ■Distributed Systems

6.2.2 Locking to Prevent Inconsistent Updates

When a number of people are working concurrently on a document, they may

use a version control system to ensure that only one person has write access at

any one time to any given part of it. This illustrates the importance of locking

as a way to manage contention for resources such as ﬁlesystems and to reduce

the likelihood of conﬂicting updates. Another mechanism is callback; a server

may keep a list of all those clients which rely on it for security state, and notify

them when the state changes.

Locking and callback also matter in secure distributed systems. Credit cards

again provide an example. If I own a hotel, and a customer presents a credit

card on checkin, I ask the card company for a pre-authorization which records

the fact that I will want to make a debit in the near future; I might register

a claim on ‘up to $500’ of her available credit. If the card is cancelled the

following day, her bank can call me and ask me to contact the police, or to

get her to pay cash. (My bank might or might not have guaranteed me the

money; it all depends on what sort of contract I’ve managed to negotiate

with it.) This is an example of the publish-register-notify model of how to do

robust authorization in distributed systems (of which there’s a more general

description in [105]).

Callback mechanisms don’t provide a universal solution, though. The cre-

dential issuer might not want to run a callback service, and the customer

might object on privacy grounds to the issuer being told all her comings and

goings. Consider passports as an example. In many countries, government

ID is required for many transactions, but governments won’t provide any

guarantee, and most citizens would object if the government kept a record

of every time an ID document was presented. Indeed, one of the frequent

objections to the British government’s proposal for biometric ID cards is that

checking citizens’ ﬁngerprints against a database whenever they show their

ID would create an audit trail of all the places where the card was used.

In general, there is a distinction between those credentials whose use gives

rise to some obligation on the issuer, such as credit cards, and the others, such

as passports. Among the differences is the importance of the order in which

updates are made.

6.2.3 The Order of Updates

If two transactions arrive at the government’s bank account — say a credit of

$500,000 and a debit of $400,000 — then the order in which they are applied

may not matter much. But if they’re arriving at my bank account, the order will

have a huge effect on the outcome! In fact, the problem of deciding the order in

which transactions are applied has no clean solution. It’s closely related to the

problem of how to parallelize a computation, and much of the art of building

6.2 Concurrency 189

efﬁcient distributed systems lies in arranging matters so that processes are

either simple sequential or completely parallel.

The usual algorithm in retail checking account systems is to batch the

transactions overnight and apply all the credits for each account before

applying all the debits. Inputs from devices such as ATMs and check sorters

are ﬁrst batched up into journals before the overnight reconciliation. The

inevitable side-effect of this is that payments which bounce then have to be

reversed out — and in the case of ATM and other transactions where the cash

has already been dispensed, you can end up with customers borrowing money

without authorization. In practice, chains of failed payments terminate, though

in theory this isn’t necessarily so. Some interbank payment mechanisms are

moving to real time gross settlement in which transactions are booked in order

of arrival. The downside here is that the outcome can depend on network

vagaries. Some people thought this would limit the systemic risk that a non-

terminating payment chain might bring down the world’s banking system,

but there is no real agreement on which practice is better. Credit cards operate

a mixture of the two strategies, with credit limits run in real time or near real

time (each authorization reduces the available credit limit) while settlement is

run just as in a checking account. The downside here is that by putting through

a large pre-authorization, a merchant can tie up your card.

The checking-account approach has recently been the subject of research in

the parallel systems community. The idea is that disconnected applications

propose tentative update transactions that are later applied to a master

copy. Various techniques can be used to avoid instability; mechanisms for

tentative update, such as with bank journals, are particularly important [553].

Application-level sanity checks are important; banks know roughly how much

they expect to pay each other each day to settle net payments, and large cash

ﬂows get veriﬁed.

In other systems, the order in which transactions arrive is much less

important. Passports are a good example. Passport issuers only worry about

their creation and expiration dates, not the order in which visas are stamped

on them. (There are exceptions, such as the Arab countries that won’t let you

in if you have an Israeli stamp on your passport, but most pure identiﬁcation

systems are stateless.)

6.2.4 Deadlock

Deadlock is another problem. Things may foul up because two systems are

each waiting for the other to move ﬁrst. A famous exposition of deadlock is

the dining philosophers’ problem in which a number of philosophers are seated

round a table. There is a chopstick between each philosopher, who can only

eat when he can pick up the two chopsticks on either side. Deadlock can follow

if they all try to eat at once and each picks up (say) the chopstick on his right.

190 Chapter 6 ■Distributed Systems

This problem, and the algorithms that can be used to avoid it, are presented in

a classic paper by Dijkstra [388].

This can get horribly complex when you have multiple hierarchies of locks,

and they’re distributed across systems some of which fail (especially where

failures can mean that the locks aren’t reliable). There’s a lot written on the

problem in the distributed systems literature [104]. But it is not just a technical

matter; there are many Catch-22 situations in business processes. So long as

the process is manual, some fudge may be found to get round the catch, but

when it is implemented in software, this option may no longer be available.

Sometimes it isn’t possible to remove the fudge. In a well known business

problem — the battle of the forms — one company issues an order with its

own terms attached, another company accepts it subject to its own terms,

and trading proceeds without any agreement about whose conditions govern

the contract. The matter may only be resolved if something goes wrong

and the two companies end up in court; even then, one company’s terms

might specify an American court while the other’s specify a court in England.

This kind of problem looks set to get worse as trading becomes more electronic.

6.2.5 Non-Convergent State

When designing protocols that update the state of a distributed system, the

‘motherhood and apple pie’ is ACID — that transactions should be atomic,

consistent, isolated and durable. A transaction is atomic if you ‘do it all or not at

all’ — which makes it easier to recover the system after a failure. It is consistent

if some invariant is preserved, such as that the books must still balance. This

is common in banking systems, and is achieved by insisting that each credit

to one account is matched by an equal and opposite debit to another (I’ll

discuss this more in Chapter 10, ‘Banking and Bookkeeping’). Transactions are

isolated if they look the same to each other, that is, are serializable; and they

are durable if once done they can’t be undone.

These properties can be too much, or not enough, or both. On the one hand,

each of them can fail or be attacked in numerous obscure ways; on the other,

it’s often sufﬁcient to design the system to be convergent. This means that, if the

transaction volume were to tail off, then eventually there would be consistent

state throughout [912]. Convergence is usually achieved using semantic tricks

such as timestamps and version numbers; this can often be enough where

transactions get appended to ﬁles rather than overwritten.

However, in real life, you also need ways to survive things that go wrong and

are not completely recoverable. The life of a security or audit manager can be

a constant battle against entropy: apparent deﬁcits (and surpluses) are always

turning up, and sometimes simply can’t be explained. For example, different

national systems have different ideas of which ﬁelds in bank transaction

records are mandatory or optional, so payment gateways often have to

6.2 Concurrency 191

guess data in order to make things work. Sometimes they guess wrong;

and sometimes people see and exploit vulnerabilities which aren’t understood

until much later (if ever). In the end, things get fudged by adding a correction

factor, called something like ‘branch differences’, and setting a target for

keeping it below a certain annual threshold.

Durability is a subject of debate in transaction processing. The advent of

phishing and keylogging attacks has meant that some small proportion of

bank accounts will at any time be under the control of criminals; money gets

moved both from and through them. When an account compromise is detected,

the bank moves to freeze it and to reverse any payments that have recently

been made from it. The phishermen naturally try to move funds through

institutions, or jurisdictions, that don’t do transaction reversal, or do it at best

slowly and grudgingly [55]. This sets up a tension between the recoverability

and thus the resilience of the payment system on the one hand, and transaction

durability and ﬁnality on the other. The solution may lie at the application

level, namely charging customers a premium for irrevocable payments and

letting the market allocate the associated risks to the bank best able to price it.

The battle of the forms mentioned in the above section gives an example of

a distributed non-electronic system that doesn’t converge.

In military systems, there is the further problem of dealing with users who

request some data for which they don’t have a clearance. For example, someone

at a dockyard might ask the destination of a warship that’s actually on a secret

mission carrying arms to Iran. If she isn’t allowed to know this, the system

may conceal the ship’s real destination by making up a cover story. Search may

have to be handled differently from speciﬁc enquiries; the joining-up of

intelligence databases since 9/11 has forced system builders to start sending

clearances along with search queries, otherwise sorting the results became

unmanageable. This all raises difﬁcult engineering problems, with poten-

tially severe conﬂicts between atomicity, consistency, isolation and durability

(not to mention performance), which will be discussed at more length in

Chapter 8, ‘Multilevel Security’.

6.2.6 Secure Time

The ﬁnal kind of concurrency problem with special interest to the security

engineer is the provision of accurate time. As authentication protocols such

as Kerberos can be attacked by inducing an error in the clock, it’s not enough

to simply trust a time source on the network. A few years ago, the worry

was a Cinderella attack: if a security critical program such as a ﬁrewall has a

license with a timelock in it, an attacker might wind your clock forward ‘and

cause your software to turn into a pumpkin’. Things have become more acute

since the arrival of operating systems such as Vista with hardware security

support, and of media players with built-in DRM; the concern now is that

192 Chapter 6 ■Distributed Systems

someone might do a large-scale service-denial attack by convincing millions

of machines that their owners had tampered with the clock, causing their ﬁles

to become inaccessible.

Anyway, there are several possible approaches to the provision of secure

time.

You could furnish every computer with a radio clock, but that can be

expensive, and radio clocks — even GPS — can be jammed if the oppo-

nent is serious.

There are clock synchronization protocols described in the research liter-

ature in which a number of clocks vote in a way that should make clock

failures and network delays apparent. Even though these are designed to

withstand random (rather than malicious) failure, they can no doubt be

hardened by having the messages digitally signed.

You can abandon absolute time and instead use Lamport time in which all

you care about is whether event A happened before event B, rather than

what date it is [766]. Using challenge-response rather than timestamps in

security protocols is an example of this; another is given by timestamp-

ing services that continually hash all documents presented to them into a

running total that’s published, and can thus provide proof that a certain

document existed by a certain date [572].

However, in most applications, you are likely to end up using the network

time protocol (NTP). This has a moderate amount of protection, with clock

voting and authentication of time servers. It is dependable enough for many

purposes.

6.3 Fault Tolerance and Failure Recovery

Failure recovery is often the most important aspect of security engineering,

yet it is one of the most neglected. For many years, most of the research papers

on computer security have dealt with conﬁdentiality, and most of the rest

with authenticity and integrity; availability has been neglected. Yet the actual

expenditures of a typical bank are the other way round. Perhaps a third of all IT

costs go on availability and recovery mechanisms, such as hot standby process-

ing sites and multiply redundant networks; a few percent more get invested

in integrity mechanisms such as internal audit; and an almost insigniﬁcant

amount goes on conﬁdentiality mechanisms such as encryption boxes. As you

read through this book, you’ll see that many other applications, from burglar

alarms through electronic warfare to protecting a company from Internet-based

service denial attacks, are fundamentally about availability. Fault tolerance

and failure recovery are a huge part of the security engineer’s job.

6.3 Fault Tolerance and Failure Recovery 193

Classical fault tolerance is usually based on mechanisms such as logs and

locking, and is greatly complicated when it must withstand malicious attacks

on these mechanisms. Fault tolerance interacts with security in a number of

ways: the failure model, the nature of resilience, the location of redundancy

used to provide it, and defense against service denial attacks. I’ll use the

following deﬁnitions: a fault may cause an error, which is an incorrect state; this

may lead to a failure which is a deviation from the system’s speciﬁed behavior.

The resilience which we build into a system to tolerate faults and recover

from failures will have a number of components, such as fault detection, error

recovery and if necessary failure recovery. The meaning of mean-time-before-

failure (MTBF) and mean-time-to-repair (MTTR) should be obvious.

6.3.1 Failure Models

In order to decide what sort of resilience we need, we must know what sort

of attacks are expected. Much of this will come from an analysis of threats

speciﬁc to our system’s operating environment, but there are some general

issues that bear mentioning.

6.3.1.1 Byzantine Failure

First, the failures with which we are concerned may be normal or Byzantine.

The Byzantine fault model is inspired by the idea that there are ngenerals

defending Byzantium, tof whom have been bribed by the Turks to cause as

much confusion as possible in the command structure. The generals can pass

oral messages by courier, and the couriers are trustworthy, so each general can

exchange conﬁdential and authentic communications with each other general

(we could imagine them encrypting and computing a MAC on each message).

What is the maximum number tof traitors which can be tolerated?

The key observation is that if we have only three generals, say Anthony,

Basil and Charalampos, and Anthony is the traitor, then he can tell Basil ‘let’s

attack’ and Charalampos ‘let’s retreat’. Basil can now say to Charalampos

‘Anthony says let’s attack’, but this doesn’t let Charalampos conclude that

Anthony’s the traitor. It could just as easily have been Basil; Anthony could

have said ‘let’s retreat’ to both of them, but Basil lied when he said ‘Anthony

says let’s attack’.

This beautiful insight is due to Leslie Lamport, Robert Shostak and

Marshall Pease, who proved that the problem has a solution if and only

if n≥3t+1 [767]. Of course, if the generals are able to sign their messages,

then no general dare say different things to two different colleagues. This illus-

trates the power of digital signatures in particular and of end-to-end security

mechanisms in general. Relying on third parties to introduce principals to each

194 Chapter 6 ■Distributed Systems

other or to process transactions between them can give great savings, but if the

third parties ever become untrustworthy then it can impose signiﬁcant costs.

Another lesson is that if a component that fails (or can be induced to fail

by an opponent) gives the wrong answer rather than just no answer, then it’s

much harder to build a resilient system using it. This has recently become a

problem in avionics, leading to an emergency Airworthiness Directive in April

2005 that mandated a software upgrade for the Boeing 777, after one of these

planes suffered a ‘ﬂight control outage’ [762].

6.3.1.2 Interaction with Fault Tolerance

We can constrain the failure rate in a number of ways. The two most obvious are

by using redundancy and fail-stop processors. The latter process error-correction

information along with data, and stop when an inconsistency is detected; for

example, bank transaction processing will typically stop if an out-of-balance

condition is detected after a processing task. The two may be combined; IBM’s

System/88 minicomputer had two disks, two buses and even two CPUs, each

of which would stop if it detected errors; the fail-stop CPUs were built by

having two CPUs on the same card and comparing their outputs. If they

disagreed the output went open-circuit, thus avoiding the Byzantine failure

problem.

In general distributed systems, either redundancy or fail-stop processing can

make a system more resilient, but their side effects are rather different. While

both mechanisms may help protect the integrity of data, a fail-stop processor

may be more vulnerable to service denial attacks, whereas redundancy can

make conﬁdentiality harder to achieve. If I have multiple sites with backup

data, then conﬁdentiality could be broken if any of them gets compromised;

and if I have some data that I have a duty to destroy, perhaps in response to a

court order, then purging it from multiple backup tapes can be a headache.

It is only a slight simpliﬁcation to say that while replication provides

integrity and availability, tamper resistance provides conﬁdentiality too. I’ll

return to this theme later. Indeed, the prevalence of replication in commercial

systems, and of tamper-resistance in military systems, echoes their differing

protection priorities.

However, there are traps for the unwary. In one case in which I was called

on as an expert, my client was arrested while using a credit card in a store,

accused of having a forged card, and beaten up by the police. He was adamant

that the card was genuine. Much later, we got the card examined by VISA

who conﬁrmed that it was indeed genuine. What happened, as well as we

can reconstruct it, was this. Credit cards have two types of redundancy on

the magnetic strip — a simple checksum obtained by combining together

all the bytes on the track using exclusive-or, and a cryptographic checksum

which we’ll describe in detail later in section 10.5.2. The former is there to

6.3 Fault Tolerance and Failure Recovery 195

detect errors, and the latter to detect forgery. It appears that in this particular

case, the merchant’s card reader was out of alignment in such a way as to

cause an even number of bit errors which cancelled each other out by chance

in the simple checksum, while causing the crypto checksum to fail. The result

was a false alarm, and a major disruption in my client’s life.

Redundancy is hard enough to deal with in mechanical systems. For

example, training pilots to handle multi-engine aircraft involves drilling them

on engine failure procedures, ﬁrst in the simulator and then in real aircraft

with an instructor. Novice pilots are in fact more likely to be killed by an

engine failure in a multi-engine plane than in a single; landing in the nearest

ﬁeld is less hazardous for them than coping with suddenly asymmetric thrust.

The same goes for instrument failures; it doesn’t help to have three artiﬁcial

horizons in the cockpit if, under stress, you rely on the one that’s broken. Air-

craft are much simpler than many modern information systems — yet there

are still regular air crashes when pilots fail to manage the redundancy that’s

supposed to keep them safe. All too often, system designers put in multiple

protection mechanisms and hope that things will be ‘all right on the night’.

This might be compared to strapping a 40-hour rookie pilot into a Learjet and

telling him to go play. It really isn’t good enough. Please bear the aircraft

analogy in mind if you have to design systems combining redundancy and

security!

The proper way to do things is to consider all the possible use cases and

abuse cases of a system, think through all the failure modes that can happen

by chance (or be maliciously induced), and work out how all the combinations

of alarms will be dealt with — and how, and by whom. Then write up your

safety and security case and have it evaluated by someone who knows what

they’re doing. I’ll have more to say on this later in the chapter on ‘System

Evaluation and Assurance’.

Even so, large-scale system failures very often show up dependencies that

the planners didn’t think of. For example, Britain suffered a fuel tanker

drivers’ strike in 2001, and some hospitals had to close because of staff

shortages. The government allocated petrol rations to doctors and nurses, but

not to schoolteachers. So schools closed, and nurses had to stay home to look

after their kids, and this closed hospitals too. We are becoming increasingly

dependent on each other, and this makes contingency planning harder.

6.3.2 What Is Resilience For?

When introducing redundancy or other resilience mechanisms into a system,

we need to be very clear about what they’re for. An important consideration

is whether the resilience is contained within a single organization.

In the ﬁrst case, replication can be an internal feature of the server to make

it more trustworthy. AT&T built a system called Rampart in which a number

196 Chapter 6 ■Distributed Systems

of geographically distinct servers can perform a computation separately and

combine their results using threshold decryption and signature [1065]; the idea

is to use it for tasks like key management [1066]. IBM developed a variant on

this idea called Proactive Security, where keys are regularly ﬂushed through

the system, regardless of whether an attack has been reported [597]. The idea

is to recover even from attackers who break into a server and then simply bide

their time until enough other servers have also been compromised. The trick

of building a secure ‘virtual server’ on top of a number of cheap off-the-shelf

machines has turned out to be attractive to people designing certiﬁcation

authority services because it’s possible to have very robust evidence of attacks

on, or mistakes made by, one of the component servers [337]. It also appeals

to a number of navies, as critical resources can be spread around a ship in

multiple PCs and survive most kinds of damage that don’t sink it [489].

But often things are much more complicated. A server may have to protect

itself against malicious clients. A prudent bank, for example, will assume that

some of its customers would cheat it given the chance. Sometimes the problem

is the other way round, in that we have to rely on a number of services, none

of which is completely trustworthy. Since 9/11, for example, international

money-laundering controls have been tightened so that people opening bank

accounts are supposed to provide two different items that give evidence of

their name and address — such as a gas bill and a pay slip. (This causes serious

problems in Africa, where the poor also need banking services as part of their

path out of poverty, but may live in huts that don’t even have addresses, let

alone utilities [55].)

The direction of mistrust has an effect on protocol design. A server faced with

multiple untrustworthy clients, and a client relying on multiple servers that

may be incompetent, unavailable or malicious, will both wish to control the

ﬂow of messages in a protocol in order to contain the effects of service denial.

So a client facing several unreliable servers may wish to use an authentication

protocol such as the Needham-Schroeder protocol I discussed in section 3.7.2;

then the fact that the client can use old server tickets is no longer a bug but

a feature. This idea can be applied to protocol design in general [1043]. It

provides us with another insight into why protocols may fail if the principal

responsible for the design, and the principal who carries the cost of fraud, are

different; and why designing systems for the real world in which everyone

(clients and servers) are unreliable and mutually suspicious, is hard.

At a still higher level, the emphasis might be on security renewability. Pay-TV

is a good example: secret keys and other subscriber management tools are typ-

ically kept in a cheap smartcard rather than in an expensive set-top box, so that

even if all the secret keys are compromised, the operator can recover by mail-

ing new cards out to his subscribers. I’ll discuss in more detail in Chapter 22,

‘Copyright and Privacy Protection’.

6.3 Fault Tolerance and Failure Recovery 197

6.3.3 At What Level Is the Redundancy?

Systems may be made resilient against errors, attacks and equipment failures at

a number of levels. As with access control systems, these become progressively

more complex and less reliable as we go up to higher layers in the system.

Some computers have been built with redundancy at the hardware level,

such as the IBM System/88 I mentioned earlier. From the late 1980’s, these

machines were widely used in transaction processing tasks (eventually ordi-

nary hardware became reliable enough that banks would not pay the premium

in capital cost and development effort to use non-standard hardware). Some

more modern systems achieve the same goal with standard hardware either at

the component level, using redundant arrays of inexpensive disks (‘RAID’ disks)

or at the system level by massively parallel server farms. But none of these

techniques provides a defense against faulty or malicious software, or against

an intruder who exploits such software.

At the next level up, there is process group redundancy. Here, we may run

multiple copies of a system on multiple servers in different locations, and

compare their outputs. This can stop the kind of attack in which the opponent

gets physical access to a machine and subverts it, whether by mechanical

destruction or by inserting unauthorized software, and destroys or alters

data. It can’t defend against attacks by authorized users or damage by bad

authorized software, which could simply order the deletion of a critical ﬁle.

The next level is backup. Here, we typically take a copy of the system (also

known as a checkpoint) at regular intervals. The backup copies are usually

kept on media that can’t be overwritten such as write-protected tapes or

DVDs. We may also keep journals of all the transactions applied between

checkpoints. In general, systems are kept recoverable by a transaction pro-

cessing strategy of logging the incoming data, trying to do the transaction,

logging it again, and then checking to see whether it worked. Whatever the

detail, backup and recovery mechanisms not only enable us to recover from

physical asset destruction; they also ensure that if we do get an attack at

the logical level — such as a time bomb in our software which deletes our

customer database on a speciﬁc date — we have some hope of recovering.

They are not infallible though. The closest that any bank I know of came

to a catastrophic computer failure that would have closed its business was

when its mainframe software got progressively more tangled as time pro-

gressed, and it just wasn’t feasible to roll back processing several weeks and

try again.

Backup is not the same as fallback. A fallback system is typically a less capable

system to which processing reverts when the main system is unavailable. An

example is the use of manual imprinting machines to capture credit card

transactions from the card embossing when electronic terminals fail.

198 Chapter 6 ■Distributed Systems

Fallback systems are an example of redundancy in the application layer —

the highest layer we can put it. We might require that a transaction above

a certain limit be authorized by two members of staff, that an audit trail be

kept of all transactions, and a number of other things. We’ll discuss such

arrangements at greater length in the chapter on banking and bookkeeping.

It is important to realise that these are different mechanisms, which do

different things. Redundant disks won’t protect against a malicious program-

mer who deletes all your account ﬁles, and backups won’t stop him if rather

than just deleting ﬁles he writes code that slowly inserts more and more

errors. Neither will give much protection against attacks on data conﬁden-

tiality. On the other hand, the best encryption in the world won’t help you

if your data processing center burns down. Real world recovery plans and

mechanisms can get ﬁendishly complex and involve a mixture of all of the

above.

The remarks that I made earlier about the difﬁculty of redundancy, and

the absolute need to plan and train for it properly, apply in spades to system

backup. When I was working in banking we reckoned that we could probably

get our backup system working within an hour or so of our main processing

centre being destroyed, but the tests we did were limited by the fact that we

didn’t want to risk processing during business hours. The most impressive

preparations I’ve ever seen were at a UK supermarket, which as a matter

of policy pulls the plug on its main processing centre once a year without

warning the operators. This is the only way they can be sure that the backup

arrangements actually work, and that the secondary processing centre really

cuts in within a minute or so. Bank tellers can keep serving customers for a

few hours with the systems down; but retailers with dead checkout lanes can’t

do that.

6.3.4 Service-Denial Attacks

One of the reasons we want security services to be fault-tolerant is to make

service-denial attacks less attractive, more difﬁcult, or both. These attacks are

often used as part of a larger attack plan. For example, one might swamp a

host to take it temporarily ofﬂine, and then get another machine on the same

LAN (which had already been subverted) to assume its identity for a while.

Another possible attack is to take down a security server to force other servers

to use cached copies of credentials.

A powerful defense against service denial is to prevent the opponent

mounting a selective attack. If principals are anonymous — or at least there

is no name service which will tell the opponent where to attack — then he

may be ineffective. I’ll discuss this further in the context of burglar alarms and

electronic warfare.

6.3 Fault Tolerance and Failure Recovery 199

Where this isn’t possible, and the opponent knows where to attack, then there

are some types of service-denial attacks which can be stopped by redundancy

and resilience mechanisms, and others which can’t. For example, the TCP/IP

protocol has few effective mechanisms for hosts to protect themselves against

various network ﬂooding attacks. An opponent can send a large number

of connection requests and prevent anyone else establishing a connection.

Defense against this kind of attack tends to involve moving your site to a

beeﬁer hosting service with specialist packet-washing hardware — or tracing

and arresting the perpetrator.

Distributed denial-of-service (DDoS) attacks had been known to the research

community as a possibility for some years. They came to public notice when

they were used to bring down Panix, a New York ISP, for several days in 1996.

During the late 1990s they were occasionally used by script kiddies to take over

chat servers. In 2000, colleagues and I suggested dealing with the problem by

server replication [1366], and in 2001 I mentioned them in passing in the ﬁrst

edition of this book. Over the following three years, small-time extortionists

started using DDoS attacks for blackmail. The modus operandi was to assemble

abotnet, a network of compromised PCs used as attack robots, which would

ﬂood a target webserver with packet trafﬁc until its owner paid them to desist.

Typical targets were online bookmakers, and amounts of $10,000–$50,000

were typically demanded to leave them alone. The typical bookie paid up

the ﬁrst time this happened, but when the attacks persisted the ﬁrst solution

was replication: operators moved their websites to hosting services such as

Akamai whose servers are so numerous (and so close to customers) that they

can shrug off anything that the average botnet could throw at them. In the

end, the blackmail problem was solved when the bookmakers met and agreed

not to pay any more blackmail money, and the Russian police were prodded

into arresting the gang responsible.

Finally, where a more vulnerable fallback system exists, a common technique

is to force its use by a service denial attack. The classic example is the use of

smartcards for bank payments in countries in Europe. Smartcards are generally

harder to forge than magnetic strip cards, but perhaps 1% of them fail every

year, thanks to static electricity and worn contacts. Also, foreign tourists still

use magnetic strip cards. So card payment systems have a fallback mode that

uses the magnetic strip. A typical attack nowadays is to use a false terminal, or

a bug inserted into the cable between a genuine terminal and a branch server,

to capture card details, and then write these details to the magnetic stripe of

a card whose chip has been destroyed (connecting 20V across the contacts

does the job nicely). In the same way, burglar alarms that rely on network

connections for the primary response and fall back to alarm bells may be very

vulnerable if the network can be interrupted by an attacker: now that online

alarms are the norm, few people pay attention any more to alarm bells.

200 Chapter 6 ■Distributed Systems

6.4 Naming

Naming is a minor if troublesome aspect of ordinary distributed systems, but

it becomes surprisingly hard in security engineering. The topical example in

the 1990s was the problem of what names to put on public key certiﬁcates. A

certiﬁcate that says simply ‘the person named Ross Anderson is allowed to

administer machine X’ is little use. Before the arrival of Internet search engines,

I was the only Ross Anderson I knew of; now I know of dozens of us. I am

also known by different names to dozens of different systems. Names exist in

contexts, and naming the principals in secure systems is becoming ever more

important and difﬁcult.

Engineers observed then that using more names than you need to causes

unnecessary complexity. For example, A certiﬁcate that simply says ‘the bearer

of this certiﬁcate is allowed to administer machine X’ is a straightforward

bearer token, which we know how to deal with; but once my name is involved,

then presumably I have to present some kind of ID to prove who I am,

and the system acquires a further dependency. Worse, if my ID is compromised

the consequences could be extremely far-reaching.

Since 9/11 the terms of this debate have shifted somewhat, as many

governments have rushed to issue their citizens with ID cards. In order to

justify the expense and hassle, pressure is often put on commercial system

operators to place some reliance on government-issue ID where this was

previously not thought necessary. In the UK, for example, you can no longer

board a domestic ﬂight using just the credit card with which you bought the

ticket, and you have to produce a passport or driving license to cash a check or

order a bank transfer for more than £1000. Such measures cause inconvenience

and introduce new failure modes into all sorts of systems.

No doubt this identity ﬁxation will abate in time, as governments ﬁnd

other things to scare their citizens with. However there is a second reason

that the world is moving towards larger, ﬂatter name spaces: the move from

barcodes (which code a particular product) to RFID tags (which contain a

128-bit unique identiﬁer that code a particular item.) This has ramiﬁcations

well beyond naming — into issues such as the interaction between product

security, supply-chain security and competition policy.

For now, it’s useful to go through what a generation of computer science

researchers have learned about naming in distributed systems.

6.4.1 The Distributed Systems View of Naming

During the last quarter of the twentieth century, the distributed systems

research community ran up against many naming problems. The basic algo-

rithm used to bind names to addresses is known as rendezvous: the principal

6.4 Naming 201

exporting a name advertises it somewhere, and the principal seeking to

import and use it searches for it. Obvious examples include phone books, and

directories in ﬁle systems.

However, the distributed systems community soon realised that naming

can get ﬁendishly complex, and the lessons learned are set out in a classic

article by Needham [958]. I’ll summarize the main points, and look at which

of them apply to secure systems.

1. The function of names is to facilitate sharing. This continues to hold: my

bank account number exists in order to provide a convenient way of

sharing the information that I deposited money last week with the

teller from whom I am trying to withdraw money this week. In gen-

eral, names are needed when the data to be shared is changeable. If I

only ever wished to withdraw exactly the same sum as I’d deposited, a

bearer deposit certiﬁcate would be ﬁne. Conversely, names need not be

shared — or linked — where data will not be; there is no need to link

my bank account number to my telephone number unless I am going to

pay my phone bill from the account.

2. The naming information may not all be in one place, and so resolving names

brings all the general problems of a distributed system. This holds with

a vengeance. A link between a bank account and a phone number

assumes both of them will remain stable. So each system relies on the

other, and an attack on one can affect the other. In the days when

electronic banking was dial-up rather than web based, a bank which

identiﬁed its customers using calling line ID was vulnerable to attacks

on telephone systems (such as tapping into the distribution frame in

an apartment block, hacking a phone company computer, or brib-

ing a phone company employee). Nowadays some banks are using

two-channel authorization to combat phishing — if you order a pay-

ment online you get a text message on your mobile phone saying ‘if

you want to pay $X to account Y, please enter the following four digit

code into your browser’. This is a bit tougher, as mobile phone trafﬁc

is encrypted — but one weak link to watch for is the binding between

the customer and his phone number. If you let just anyone notify you of

a customer phone number change, you’ll be in trouble.

3. It is bad to assume that only so many names will be needed. The shortage of

IP addresses, which motivated the development of IP version 6 (IPv6),

is well enough discussed. What is less well known is that the most

expensive upgrade which the credit card industry ever had to make

was not Y2K remediation, but the move from thirteen digit credit card

numbers to sixteen. Issuers originally assumed that 13 digits would

be enough, but the system ended up with tens of thousands of banks

(many with dozens of products) so a six digit bank identiﬁcation number

202 Chapter 6 ■Distributed Systems

(BIN number) was needed. Some card issuers have millions of

customers, so a nine digit account number is the norm. And there’s also

acheck digit (a one-digit linear combination of the other digits which is

appended to detect errors).

4. Global names buy you less than you think. For example, the 128-bit address

in IPv6 can in theory enable every object in the universe to have a

unique name. However, for us to do business, a local name at my end

must be resolved into this unique name and back into a local name at

your end. Invoking a unique name in the middle may not buy us any-

thing; it may even get in the way if the unique naming service takes

time, costs money, or occasionally fails (as it surely will). In fact, the

name service itself will usually have to be a distributed system, of

the same scale (and security level) as the system we’re trying to pro-

tect. So we can expect no silver bullets from this quarter. One reason

the banking industry was wary of initiatives such as SET that would

have given each customer a public key certiﬁcate on a key with which

they could sign payment instructions was that banks already have per-

fectly good names for their customers (account numbers). Adding an

extra name has the potential to add extra costs and failure modes.

5. Names imply commitments, so keep the scheme ﬂexible enough to cope with

organizational changes. This sound principle was ignored in the design

of a UK government’s key management system for secure email [76].

There, principals’ private keys are generated by encrypting their names

under departmental master keys. So the frequent reorganizations meant

that the security infrastructure would have to be rebuilt each time —

and that money would have had to be spent solving many secondary

problems such as how people would access old material.

6. Names may double as access tickets, or capabilities. We have already seen

a number of examples of this in Chapters 2 and 3. In general, it’s a bad

idea to assume that today’s name won’t be tomorrow’s password or

capability — remember the Utrecht fraud we discussed in section 3.5.

(This is one of the arguments for making all names public keys — ‘keys

speak in cyberspace’ in Carl Ellison’s phrase — but we’ve already noted

the difﬁculties of linking keys with names.)

I’ve given a number of examples of how things go wrong when a name

starts being used as a password. But sometimes the roles of name and

password are ambiguous. In order to get entry to a car park I used to

use at the university, I had to speak my surname and parking badge

number into a microphone at the barrier. So if I say, ‘Anderson, 123’,

which of these is the password? In fact it was ‘Anderson’, as

anyone can walk through the car park and note down valid badge

6.4 Naming 203

numbers from the parking permits on the car windscreens. Another

example, from medical informatics, is a large database of medical

records kept by the UK government for research, where the name of the

patient has been replaced by their postcode and date of birth. Yet access

to many medical services requires the patient to give just these two

items to the receptionist to prove who they are. I will have more to say

on this later.

7. Things are made much simpler if an incorrect name is obvious. In standard

distributed systems, this enables us to take a liberal attitude to cacheing.

In payment systems, credit card numbers may be accepted while a ter-

minal is ofﬂine so long as the credit card number appears valid (i.e., the

last digit is a proper check digit of the ﬁrst ﬁfteen) and it is not on the

hot card list. Certiﬁcates provide a higher-quality implementation of

the same basic concept.

It’s important where the name is checked. The credit card check digit

algorithm is deployed at the point of sale, so it is public. A further

check — the card veriﬁcation value on the magnetic strip — is computed

with secret keys but can be checked at the issuing bank, the acquir-

ing bank or even at a network switch (if one trusts these third parties

with the keys). This is more expensive, and still vulnerable to network

outages.

8. Consistency is hard, and is often fudged. If directories are replicated, then

you may ﬁnd yourself unable to read, or to write, depending on whether too

many or too few directories are available. Naming consistency causes prob-

lems for e-commerce in a number of ways, of which perhaps the most

notorious is the bar code system. Although this is simple enough in

theory — with a unique numerical code for each product — in prac-

tice it can be a nightmare, as different manufacturers, distributors and

retailers attach quite different descriptions to the bar codes in their

databases. Thus a search for products by ‘Kellogg’s’ will throw up

quite different results depending on whether or not an apostrophe is

inserted, and this can cause great confusion in the supply chain. Pro-

posals to ﬁx this problem can be surprisingly complicated [618]. There

are also the issues of convergence discussed above; data might not be

consistent across a system, even in theory. There are also the prob-

lems of timeliness, such as whether a product has been recalled.

Now, many ﬁrms propose moving to RFID chips that contain a glob-

ally unique number: an item code rather than a product code. This may

move name resolution upstream; rather than the shop’s

computer recognising that the customer has presented a packet of

vitamin C at the checkout, it may go to the manufacturer to ﬁnd this

204 Chapter 6 ■Distributed Systems

out. Manufacturers push for this on safety grounds; they can then be

sure that the product hasn’t passed its sell-by date and has not been

recalled. But this also increases their power over the supply chain;

they can detect and stop gray-market trading that would otherwise

undermine their ability to charge different prices in different towns.

9. Don’t get too smart. Phone numbers are much more robust than computer

addresses. Amen to that — but it’s too often ignored by secure system

designers. Bank account numbers are much easier to deal with than

the public-key certiﬁcates which were once believed both necessary

and sufﬁcient to secure credit card payments online. I’ll discuss spe-

ciﬁc problems of public key infrastructures in section 21.4.5.7.

10. Some names are bound early, others not; and in general it is a bad thing to

bind early if you can avoid it. A prudent programmer will normally avoid

coding absolute addresses or ﬁlenames as that would make it hard to

upgrade or replace a machine. He will prefer to leave this to a conﬁgu-

ration ﬁle or an external service such as DNS. (This is another reason

not to put addresses in names.) It is true that secure systems often

want stable and accountable names as any third-party service used

for last minute resolution could be a point of attack. However, design-

ers should read the story of Netgear, who got their home routers to ﬁnd

out the time using the Network Time Protocol from a server

at the University of Wisconsin-Madison. Their product was successful;

the university was swamped with hundreds of thousands of packets a

second. Netgear ended up paying them $375,000 to maintain the time

service for three years. Shortly afterwards, D-Link repeated the same

mistake [304].

So Needham’s ten principles for distributed naming apply fairly directly to

distributed secure systems.

6.4.2 What Else Goes Wrong

Needham’s principles, although very useful, are not sufﬁcient. They were

designed for a world in which naming systems could be designed and

imposed at the system owner’s convenience. When we move from distributed

systems in the abstract to the reality of modern web-based (and interlinked)

service industries, there is still more to say.

6.4.2.1 Naming and Identity

The most obvious difference is that the principals in security protocols may be

known by many different kinds of name — a bank account number, a company

registration number, a personal name plus a date of birth or a postal address,

6.4 Naming 205

a telephone number, a passport number, a health service patient number, or a

userid on a computer system.

As I mentioned in the introductory chapter, a common mistake is to confuse

naming with identity. Identity is when two different names (or instances of the

same name) correspond to the same principal (this is known in the distributed

systems literature as an indirect name or symbolic link). The classic example

comes from the registration of title to real estate. It is very common that

someone who wishes to sell a house uses a different name than they did at

the time it was purchased: they might have changed their name on marriage,

or after a criminal conviction. Changes in name usage are also common. For

example, the DE Bell of the Bell-LaPadula system2wrote his name ‘D. Elliot

Bell’ in 1973 on that paper; but he was always known as David, which is how

he now writes his name too. A land-registration system must cope with a lot

of identity issues like this.

The classic example of identity failure leading to compromise is check fraud.

Suppose I steal a high-value check made out to Mr Elliott Bell. I then open an

account in that name and cash it; banking law in both the USA and the UK

absolves the bank of liability so long as it pays the check into an account of the

same name. The modern procedure of asking people who open bank accounts

for two proofs of address, such as utility bills, probably makes the bad guys’

job easier; there are hundreds of utility companies, many of which provide

electronic copies of bills that are easy to alter. The pre-9/11 system, of taking

up personal references, may well have been better.

Moving to verifying government-issue photo-ID on account opening adds

to the mix statements such as ‘The Elliott Bell who owns bank account number

12345678 is the Elliott James Bell with passport number 98765432 and date

of birth 3/4/56’. This may be seen as a symbolic link between two separate

systems — the bank’s and the passport ofﬁce’s. Note that the latter part of

this ‘identity’ encapsulates a further statement, which might be something

like ‘The US passport ofﬁce’s ﬁle number 98765432 corresponds to the entry

in birth register for 3/4/56 of one Elliott James Bell’. In general, names may

involve several steps of recursion, and this gives attackers a choice of targets.

For example, a lot of passport fraud is pre-issue fraud: the bad guys apply

for passports in the names of genuine citizens who haven’t applied for a

passport already and for whom copies of birth certiﬁcates are easy enough to

obtain. Postmortem applications are also common. Linden Labs, the operators

of Second Life, introduced in late 2007 a scheme whereby you prove you’re

over 18 by providing the driver’s license number or social security number

of someone who is. Now a web search quickly pulls up such data for many

people, such as the rapper Tupac Amaru Shakur; and yes, Linden Labs did

accept Mr Shakur’s license number — even through the license is expired, and

2I’ll discuss this in Chapter 8, ‘Multilevel Secure Systems’.

206 Chapter 6 ■Distributed Systems

he’s dead. Indeed, someone else managed to verify their age using Mohammed

Atta’s driver’s license [389].

6.4.2.2 Cultural Assumptions

The assumptions that underlie names often change from one country to

another. In the English-speaking world, people may generally use as many

names as they please; a name is simply what you are known by. But some

countries forbid the use of aliases, and others require them to be registered.

This can lead to some interesting scams: in at least one case, a British citizen

evaded pursuit by foreign tax authorities by changing his name. On a less

exalted plane, women who pursue academic careers and change their name

on marriage may wish to retain their former name for professional use, which

means that the name on their scientiﬁc papers is different from their name on

the payroll. This caused a row at my university which introduced a uniﬁed ID

card system, keyed to payroll names, without support for aliases.

In general, many of the really intractable problems arise when an attempt is

made to unify two local naming systems which turn out to have incompatible

assumptions. As electronics invades everyday life more and more, and systems

become linked up, conﬂicts can propagate and have unexpected remote effects.

For example, one of the lady professors in the dispute over our university card

was also a trustee of the British Library, which issues its own admission tickets

on the basis of the name on the holder’s home university library card.

Even human naming conventions are not uniform. Russians are known by

a forename, a patronymic and a surname; Icelanders have no surname but are

known instead by a given name followed by a patronymic if they are male

and a matronymic if they are female. This causes problems when they travel.

When US immigration comes across ‘Maria Trosttad´

ottir’ and learns that

‘Trosttad´

ottir’ isn’t a surname or even a patronymic, their standard practice

was to compel her to adopt as a surname a patronymic (say, ‘Carlsson’ if her

father was called Carl). This causes unnecessary offence.

The biggest cultural divide is often thought to be that between the English

speaking countries (where identity cards were long considered to be unaccept-

able on privacy grounds3) and the countries conquered by Napoleon or by the

Soviets, where identity cards are the norm.

There are further subtleties. I know Germans who have refused to believe

that a country could function at all without a proper system of population reg-

istration and ID cards, yet admit they are asked for their ID card only rarely (for

example, to open a bank account or get married). Their card number can’t be

used as a name, because it is a document number and changes every time a new

card is issued. A Swiss hotelier may be happy to register a German guest on

3unless they’re called drivers’ licences or health service cards!

6.4 Naming 207

sight of an ID card rather than a credit card, but if he discovers some damage

after a German guest has checked out, he may be out of luck. And the British

passport ofﬁce will issue a citizen with more than one passport at the same

time, if he says he needs them to make business trips to (say) Cuba and the

USA; so our Swiss hotelier, ﬁnding that a British guest has just left without

paying, can’t rely on the passport number to have him stopped at the airport.

There are many other hidden assumptions about the relationship between

governments and people’s names, and they vary from one country to another

in ways which can cause unexpected security failures.

6.4.2.3 Semantic Content of Names

Another hazard arises on changing from one type of name to another without

adequate background research. A bank got sued after they moved from storing

customer data by account number to storing it by name and address. They

wanted to target junk mail more accurately, so they wrote a program to link

up all the accounts operated by each of their customers. The effect for one poor

customer was that the bank statement for the account he maintained for his

mistress got sent to his wife, who divorced him.

Sometimes naming is simple, but sometimes it merely appears to be. For

example, when I got a monthly ticket for the local swimming baths, the cashier

simply took the top card off a pile, swiped it through a reader to tell the system

it was now live, and gave it to me. I had been assigned a random name — the

serial number on the card. Many US road toll systems work in much the same

way. Sometimes a random, anonymous name can add commercial value. In

Hong Kong, toll tokens for the Aberdeen tunnel could be bought for cash, or at

a discount in the form of a reﬁllable card. In the run-up to the transfer of power

from Britain to Beijing, many people preferred to pay extra for the less traceable

version as they were worried about surveillance by the new government.

Semantics of names can change. I once got a hardware store loyalty card with

a random account number (and no credit checks). I was offered the chance to

change this into a bank card after the store was taken over by the supermarket

and the supermarket started a bank. (This appears to have ignored money

laundering regulations that all new bank customers must be subjected to due

diligence.)

Assigning bank account numbers to customers might have seemed unprob-

lematic — but as the above examples show, systems may start to construct

assumptions about relationships between names that are misleading and

dangerous.

6.4.2.4 Uniqueness of Names

Human names evolved when we lived in small communities. We started off

with just forenames, but by the late Middle Ages the growth of travel led

208 Chapter 6 ■Distributed Systems

governments to bully people into adopting surnames. That process took a

century or so, and was linked with the introduction of paper into Europe

as a lower-cost and more tamper-resistant replacement for parchment; paper

enabled the badges, seals and other bearer tokens, which people had previ-

ously used for road tolls and the like, to be replaced with letters that mentioned

their names.

The mass movement of people, business and administration to the Internet

in the decade after 1995 has been too fast to allow any such social adaptation.

There are now many more people (and systems) online than we are used

to dealing with. As I remarked at the beginning of this section, I used to

be the only Ross Anderson I knew of, but thanks to search engines, I now

know dozens of us. Some of us work in ﬁelds I’ve also worked in, such

as software engineering and electric power distribution; the fact that I’m

www.ross-anderson.com and ross.anderson@iee.org is down to luck — I

got there ﬁrst. (Even so, rjanderson@iee.org is somebody else.) So even

the combination of a relatively rare name and a specialized profession is

still ambiguous. Another way of putting this is that ‘traditional usernames,

although old-school-geeky, don’t scale well to the modern Internet’ [21].

Sometimes system designers are tempted to solve the uniqueness problem

by just giving everything and everyone a number. This is very common in

transaction processing, but it can lead to interesting failures if you don’t put

the uniqueness in the right place. A UK bank wanted to send £20m overseas,

but the operator typed in £10m by mistake. To correct the error, a second

payment of £10m was ordered. However, the sending bank’s system took

the transaction sequence number from the paperwork used to authorise it.

Two payments were sent to SWIFT with the same date, payee, amount and

sequence number — so the second was discarded as a duplicate [218].

6.4.2.5 Stability of Names and Addresses

Many names include some kind of address, yet addresses change. About a

quarter of Cambridge phone book addresses change every year; with email,

the turnover is probably higher. A project to develop a directory of people who

use encrypted email, together with their keys, found that the main cause of

changed entries was changes of email address [67]. (Some people had assumed

it would be the loss or theft of keys; the contribution from this source was

precisely zero.)

A serious problem could arise with IPv6. The security community assumes

that v6 IP addresses will be stable, so that public key certiﬁcates can bind

principals of various kinds to them. All sorts of mechanisms have been

proposed to map real world names, addresses and even document content

indelibly and eternally on to 128 bit strings (see, for example, [573]). The data

communications community, on the other hand, assumes that IPv6 addresses

6.4 Naming 209

will change regularly. The more signiﬁcant bits will change to accommodate

more efﬁcient routing algorithms, while the less signiﬁcant bits will be used to

manage local networks. These assumptions can’t both be right.

Distributed systems pioneers considered it a bad thing to put addresses in

names [912]. But in general, there can be multiple layers of abstraction with

some of the address information at each layer forming part of the name at the

layer above. Also, whether a namespace is better ﬂat depends on the applica-

tion. Often people end up with different names at the departmental and orga-

nizational level (such as rja14@cam.ac.uk and ross.anderson@cl.cam.ac.uk

in my own case). So a clean demarcation between names and addresses is not

always possible.

Authorizations have many (but not all) of the properties of addresses. Kent’s

Law tells designers that if a credential contains a list of what it may be used

for, then the more things are on this list the shorter its period of usefulness. A

similar problem besets systems where names are composite. For example, some

online businesses recognize me by the combination of email address and credit

card number. This is clearly bad practice. Quite apart from the fact that I have

several email addresses, I have several credit cards. The one I use will depend

on which of them is currently offering the best service or the biggest bribes.

There are many good reasons to use pseudonyms. It’s certainly sensible

for children and young people to use online names that aren’t easily linkable

to their real names. This is often advocated as a child-protection measure,

although the number of children abducted and murdered by strangers in

developed countries remains happily low and stable at about 1 per 10,000,000

population per year. A more serious reason is that when you go for your ﬁrst

job on leaving college aged 22, or for a CEO’s job at 45, you don’t want Google

to turn up all your teenage rants. Many people also change email addresses

from time to time to escape spam; I give a different email address to every

website where I shop. Of course, there are police and other agencies that

would prefer people not to use pseudonyms, and this takes us into the whole

question of traceability online, which I’ll discuss in Part II.

6.4.2.6 Adding Social Context to Naming

The rapid growth recently of social network sites such as Facebook points to

a more human and scaleable way of managing naming. Facebook does not

give me a visible username: I use my own name, and build my context by

having links to a few dozen friends. (Although each proﬁle does have a unique

number, this does not appear in the page itself, just in URLs.) This ﬁxes the

uniqueness problem — Facebook can have as many Ross Andersons as care

to turn up — and the stability problem (though at the cost of locking me into

Facebook if I try to use it for everything).

210 Chapter 6 ■Distributed Systems

Distributed systems folks had argued for some time that no naming system

can be simultaneously globally unique, decentralized, and human-meaningful.

It can only have two of those attributes (Zooko’s triangle) [21]. In the past,

engineers tanded to look for naming systems that were unique and meaningful,

like URLs, or unique and decentralised, as with public-key certiﬁcates4. The

innovation from sites like Facebook is to show on a really large scale that

naming doesn’t have to be unique at all. We can use social context to build

systems that are both decentralised and meaningful — which is just what our

brains evolved to cope with.

6.4.2.7 Restrictions on the Use of Names

The interaction between naming and society brings us to a further problem:

some names may be used only in restricted circumstances. This may be laid

down by law, as with the US Social Security Number (SSN) and its equivalents

in many European countries. Sometimes it is a matter of marketing. I would

rather not give out my residential address (or my home phone number) when

shopping online, and I avoid businesses that demand them.

Restricted naming systems interact in unexpected ways. For example, it’s

fairly common for hospitals to use a patient number as an index to medical

record databases, as this may allow researchers to use pseudonymous records

for some limited purposes without much further processing. This causes

problems when a merger of health maintenance organizations, or a new policy

directive in a national health service, forces the hospital to introduce uniform

names. In the UK, for example, the merger of two records databases — one of

which used patient names while the other was pseudonymous — has raised

the prospect of legal challenges to processing on privacy grounds.

Finally, when we come to law and policy, the deﬁnition of a name turns

out to be unexpectedly tricky. Regulations that allow police to collect com-

munications data — that is, a record of who called whom and when — are

often very much more lax than the regulations governing phone tapping; in

many countries, police can get this data just by asking the phone company.

There was an acrimonious public debate in the UK about whether this enables

them to harvest the URLs which people use to fetch web pages. URLs often

have embedded in them data such as the parameters passed to search engines.

Clearly there are policemen who would like a list of everyone who hit a URL

such as http://www.google.com/search?q=cannabis+cultivation+UK; just as

clearly, many people would consider such large-scale trawling to be an unac-

ceptable invasion of privacy. The police argued that if they were limited to

4Carl Ellison, Butler Lampson and Ron Rivest went so far as to propose the SPKI/SDSI certiﬁcate

system in which naming would be relative, rather than ﬁxed with respect to central authority.

The PGP web of trust worked informally in the same way.

6.5 Summary 211

monitoring IP addresses, they could have difﬁculties tracing criminals who

use transient IP addresses. In the end, Parliament resolved the debate when it

passed the Regulation of Investigatory Powers Act in 2000: the police just get

the identity of the machine under the laxer regime for communications data.

6.4.3 Types of Name

The complexity of naming appears at all levels — organisational, technical

and political. I noted in the introduction that names can refer not just to

persons (and machines acting on their behalf), but also to organizations, roles

(‘the ofﬁcer of the watch’), groups, and compound constructions: principal in

role — Alice as manager; delegation — Alice for Bob; conjunction — Alice and

Bob. Conjunction often expresses implicit access rules: ‘Alice acting as branch

manager plus Bob as a member of the group of branch accountants’.

That’s only the beginning. Names also apply to services (such as NFS, or

a public key infrastructure) and channels (which might mean wires, ports,

or crypto keys). The same name might refer to different roles: ‘Alice as a

computer game player’ ought to have less privilege than ‘Alice the system

administrator’. The usual abstraction used in the security literature is to treat

them as different principals. This all means that there’s no easy mapping

between names and principals.

Finally, there are functional tensions which come from the underlying

business processes rather than from system design. Businesses mainly want

to get paid, while governments want to identify people uniquely. In effect,

business wants a credit card number while government wants a passport

number. Building systems which try to be both — as many governments are

trying to encourage — is a tar-pit. There are many semantic differences. You

can show your passport to a million people, if you wish, but you had better

not try that with a credit card. Banks want to open accounts for anyone who

turns up with some money; governments want them to verify people’s identity

carefully in order to discourage money laundering. The list is a long one.

6.5 Summary

Many secure distributed systems have incurred huge costs, or developed

serious vulnerabilities, because their designers ignored the basic lessons of

how to build (and how not to build) distributed systems. Most of these lessons

are still valid, and there are more to add.

A large number of security breaches are concurrency failures of one kind or

another; systems use old data, make updates inconsistently or in the wrong

order, or assume that data are consistent when they aren’t and can’t be.

Knowing the right time is harder than it seems.

212 Chapter 6 ■Distributed Systems

Fault tolerance and failure recovery are critical. Providing the ability to

recover from security failures, and random physical disasters, is the main

purpose of the protection budget for many organisations. At a more technical

level, there are signiﬁcant interactions between protection and resilience mech-

anisms. Byzantine failure — where defective processes conspire, rather than

failing randomly — is an issue, and interacts with our choice of cryptographic

tools. There are many different ﬂavors of redundancy, and we have to use the

right combination. We need to protect not just against failures and attempted

manipulation, but also against deliberate attempts to deny service which may

often be part of larger attack plans.

Many problems also arise from trying to make a name do too much, or

making assumptions about it which don’t hold outside of one particular

system, or culture, or jurisdiction. For example, it should be possible to revoke

a user’s access to a system by cancelling their user name without getting sued

on account of other functions being revoked. The simplest solution is often to

assign each principal a unique identiﬁer used for no other purpose, such as a

bank account number or a system logon name. But many problems arise when

merging two systems that use naming schemes that are incompatible for some

reason. Sometimes this merging can even happen by accident — an example

being when two systems use a common combination such as ‘name plus date

of birth’ to track individuals, but in different ways.

Research Problems

In the research community, secure distributed systems tend to have been dis-

cussed as a side issue by experts on communications protocols and operating

systems, rather than as a discipline in its own right. So it is a relatively open

ﬁeld, and one still holds much promise.

There are many technical issues which I’ve touched on in this chapter, such

as how we design secure time protocols and the complexities of naming. But

perhaps the most important research problem is to work out how to design

systems that are resilient in the face of malice, that degrade gracefully, and

whose security can be recovered simply once the attack is past. This may mean

revisiting the deﬁnition of convergent applications. Under what conditions

can one recover neatly from corrupt security state?

What lessons do we need to learn from the onset of phishing and keylogging

attacks on electronic banking, which mean that at any given time a small (but

nonzero) proportion of customer accounts will be under criminal control?

Do we have to rework recovery (which in its classic form explores how

to rebuild databases from backup tapes) into resilience, and if so how do

we handle the tensions with the classic notions of atomicity, consistency,

Further Reading

There are many books on distributed systems; I’ve found Mullender [912] to

be helpful and thought-provoking for graduate students, while the textbook

we recommend to our undergraduates by Bacon [104] is also worth reading.

Geraint Price has a survey of the literature on the interaction between fault

tolerance and security [1043]. The research literature on concurrency, such as

the SIGMOD conferences, has occasional gems. There is also a 2003 report

from the U.S. National Research Council, ‘Who Goes There? Authentication

Through the Lens of Privacy’ which discusses the tradeoffs between authentica-

tion and privacy, and how they tend to scale poorly [710].

CHAPTER

Economics

The great fortunes of the information age lie in the hands of

companies that have established proprietary

architectures that are used by a

large installed base of

locked-in customers.

— Carl Shapiro and Hal Varian

There are two things I am sure of after all these years: there is

a growing societal need for high assurance software, and

market forces are never going to provide it.

— Earl Boebert

If you try to buck the markets, then the markets will buck you.

— Margaret Thatcher

7.1 Introduction

The economics of information security has recently become a thriving and

fast-moving discipline. We started to realise round about 2000 that many

security system failures weren’t due to technical errors so much as to wrong

incentives: the classic case is where the people who guard a system are not

the people who suffer when it fails. Indeed, security mechanisms are often

designed quite deliberately to shift liability, which often leads to trouble.

Economics has always been important to engineering, at the raw level of

cost accounting; a good engineer was one who could build a bridge safely

with a thousand tons of concrete when everyone else used two thousand

tons. But the perverse incentives that arise in complex systems with multiple

owners make economic questions both more important and more subtle

215

216 Chapter 7 ■Economics

for the security engineer. Truly global-scale systems like the Internet arise

from the actions of millions of independent principals with divergent interests;

we hope that reasonable global outcomes will result from selﬁsh local actions.

In general, people won’t do something unless they have an incentive to.

Markets are often the best guide we have to what sort of mechanisms work, or

fail. Markets also fail; the computer industry has been dogged by monopolies

since its earliest days. The reasons for this are now understood, and their

interaction with security is starting to be. When someone asks ‘Why is Microsoft

software insecure?’ we can now give a principled answer rather than simply

cursing Redmond as a form of bad weather.

The new ﬁeld of security economics provides valuable insights not just

into ‘security’ topics such as privacy, bugs, spam, and phishing, but into

more general areas such as system dependability. For example, what’s the

optimal balance of effort by programmers and testers? (For the answer, see

section 7.5.1 below.) It also enables us to analyse the policy problems that

security technology increasingly throws up — on issues like digital rights

management. Where protection mechanisms are used by the system designer

to control the owner of a machine, rather than to protect her against outside

enemies, questions of competition policy and consumer rights follow, which

economics provides the language to discuss. There are also questions of the

balance between public and private action. Network insecurity is somewhat

like air pollution or congestion, in that people who connect insecure machines

to the Internet do not bear the full consequences of their actions. So how much

of the protection effort can (or should) be left to individuals, and how much

should be borne by vendors, regulators or the police?

7.2 Classical Economics

Modern economics is an enormous ﬁeld covering many different aspects of

human behaviour. The parts of it that (so far) have found application in security

are largely drawn from microeconomics and game theory. I’ll discuss game

theory in the next section; here, I’ll give a helicopter tour of the most relevant

ideas from microeconomics. The object of the exercise is not to provide a tutorial

on economics, or even on information economics — for that, I recommend you

read Carl Shapiro and Hal Varian’s book ‘Information Rules’ [1159] — but to

familiarise you with the essential terminology and ideas, so we can move on

to discuss security economics.

The modern subject started in the 18th century when the industrial revolu-

tion and growing trade changed the world, and people wanted to understand

why. In 1776, Adam Smith’s classic ‘The Wealth of Nations’ [1192] provided a

ﬁrst draft: he explained how rational self-interest in a free market economy

leads to economic wellbeing. Specialisation leads to productivity gains at

all levels from a small factory to international trade, and the self-interested

7.2 Classical Economics 217

striving of many individuals and ﬁrms drives progress, as people must pro-

duce something others value to survive in a competitive market. In his famous

phrase, ‘It is not from the benevolence of the butcher, the brewer, or the baker,

that we can expect our dinner, but from their regard to their own interest’.

These ideas were reﬁned by nineteenth-century economists; David Ricardo

clariﬁed and strengthened Smith’s arguments in favour of free trade, Stanley

Jevons, L´

eon Walras and Carl Menger built detailed models of supply and

demand, and by the end of the century Alfred Marshall had combined models

of supply and demand in markets for goods, labour and capital into an

overarching ‘classical’ model in which, at equilibrium, all the excess proﬁts

would be competed away and the economy would be functioning efﬁciently.

By 1948, Kenneth Arrow and G´

erard Debreu had put this on a rigorous

mathematical foundation by proving that markets give efﬁcient outcomes,

subject to certain conditions. Much of the interest in economics — especially

to the computer industry, and to security folks in particular — comes from the

circumstances in which these conditions aren’t met, giving rise to monopolies

and other problems.

7.2.1 Monopoly

A rapid way into the subject is to consider a simple textbook case of monopoly.

Suppose we have a market for apartments in a university town, and the

students have different incomes. We might have one rich student able to pay

$4000 a month, maybe 300 people willing to pay $2000 a month, and (to give

us round numbers) 1000 prepared to pay $1000 a month. That gives us the

demand curve shown in Figure 7.1 below.

price

supply

demand

apartments

1000800

$1400 pm

$4000 pm

$1000 pm

Figure 7.1: The market for apartments

218 Chapter 7 ■Economics

So if there are 1000 apartments being let by many competing landlords,

the market-clearing price will be at the intersection of the demand curve

with the vertical supply curve, namely $1000. But suppose the market is

rigged — say the landlords have set up a cartel, or the university makes

its students rent through a tied agency. For simplicity let’s assume a single

monopolist landlord. He examines the demand curve, and notices that if he

rents out only 800 apartments, he can get $1400 per month for each of them.

Now 800 times $1400 is $1,120,000 per month, which is more than the million

dollars a month he’ll make from the market price at $1000. (Economists would

say that his ‘revenue box’ is CBFO rather than EDGO.) So he sets an artiﬁcially

high price, and 200 apartments remain empty.

This is clearly inefﬁcient, and the Italian economist Vilfredo Pareto invented

a neat way to formalise this. A Pareto improvement is any change that would

make some people better off without making anyone else worse off, and an

allocation is Pareto efﬁcient if there isn’t any Pareto improvement available.

Here, the allocation is not efﬁcient, as the monopolist could rent out one empty

apartment to anyone at a lower price, making both him and them better off.

Now Pareto efﬁciency is a rather weak criterion; both perfect communism

(everyone gets the same income) and perfect dictatorship (the President gets

all the income) are Pareto-efﬁcient. In neither case can you make anyone better

off without making someone worse off! Yet the simple monopoly described

here is not efﬁcient even in this very weak sense.

So what can the monopolist do? There is one possibility — if he can charge

everyone a different price, then he can set each student’s rent at exactly what

they are prepared to pay. We call such a landlord a discriminating monopolist;he

charges the rich student exactly $4000, and so on down to the 1000th student

whom he charges exactly $1000. The same students get apartments as before,

yet almost all of them are worse off. The rich student loses $3000, money that

he was prepared to pay but previously didn’t have to; economists refer to this

money he saved as surplus. In effect, the discriminating monopolist manages

to extract all the consumer surplus.

Merchants have tried to price-discriminate since antiquity. The carpet seller

in Damascus who offers to ‘make a very special price, just for you’ is playing

this game, as is Microsoft in offering seven different versions of Vista at

different price points, and an airline in selling ﬁrst, business and cattle class

seats. The extent to which ﬁrms can do this depends on a number of factors,

principally their market power and the amount of information they have.

Market power is a measure of how close a merchant is to being a monopolist;

under monopoly the merchant is a price setter while under perfect competition

he simply has to accept whatever price the market establishes (he is a price

taker). Technology tends to increase market power while reducing the cost of

information about customers at the same time, and this combination is one

of the main factors eroding privacy in the modern world.

7.2 Classical Economics 219

7.2.2 Public Goods

A second type of market failure occurs when everyone gets the same quantity

of some good, whether they want it or not. Classic examples are air quality,

national defense and scientiﬁc research. Economists call these public goods,and

the formal deﬁnition is that they are goods which are non-rivalrous (my using

them doesn’t mean there’s less available for you) and non-excludable (there’s

no practical way to exclude people from consuming them). Uncoordinated

markets are generally unable to provide public goods in socially optimal

quantities.

Public goods may be supplied by governments directly, as in the case of

national defense, or by using indirect mechanisms to coordinate markets. The

classic example is laws on patents and copyrights to encourage people to

produce inventions, literary works and musical compositions by giving them

a temporary monopoly — by making the goods in question excludable for a

limited period of time. Very often, public goods are provided by some mix

of public and private action; scientiﬁc research is done in universities that

get some public subsidy, earn some income from student fees, and get some

research contracts from industry (where industry may get patents on the useful

inventions while the underlying scientiﬁc research gets published for all to

use). The mix can be controversial; the debate on global warming sets people

who want direct government action in the form of a ‘carbon tax’ (which would

be simple and easy to enforce) against others who want a ‘cap and trade’

system whereby ﬁrms and countries can trade licenses to emit carbon (which

in a perfect world would cause emission reductions by the ﬁrms who could do

so most cheaply, but which might well be more open to abuse and evasion).

The importance of this for us is that many aspects of security are public

goods. I do not have an anti-aircraft gun on the roof of my house; air-defense

threats come from a small number of actors, and are most efﬁciently dealt with

by government action. So what about Internet security? Certainly there are

strong externalities involved, and people who connect insecure machines to the

Internet end up dumping costs on others, just like people who burn polluting

coal ﬁres. So what should we do about it? One might imagine a government

tax on vulnerabilities, with rewards paid to researchers who discover them

and larger ﬁnes imposed on the ﬁrms whose software contained them. Again,

one of the early papers on security economics suggested a vulnerability cap-

and-trade system; vendors who could not be bothered to make their software

secure could buy permits from other vendors who were making the effort

to tighten up their products [256]. (Both arrangements would be resisted by

the free software community!) But is air pollution the right analogy — or air

defense?

Threats such as viruses and spam used to come from a large number of

small actors, but since about 2004 we’ve seen a lot of consolidation as malware

220 Chapter 7 ■Economics

writers and users have become commercial. By 2007, the number of serious

spammers had dropped to the point that ISPs see signiﬁcant ﬂuctuations in

overall spam volumes as the big spammers run particular campaigns — there

is no law of large numbers operating any more [305]. This suggests a different

and perhaps more centralised strategy. If our air-defense threat in 1987 was

mainly the Russian airforce, and our cyber-defense threat in 2007 is mainly

from a small number of Russian gangs, and they are imposing large costs

on US and European Internet users and companies, then state action may be

needed now as it was then. Instead of telling us to buy anti-virus software,

our governments could be putting pressure on the Russians to round up and

jail their cyber-gangsters. I’ll discuss this in greater detail in Part III; for now,

it should be clear that concepts such as ‘monopoly’ and ‘public goods’ are

important to the security engineer — and indeed to everyone who works in

IT. Just think of the two operating systems that dominate the world’s desktops

and server farms: Windows is a monopoly, while the common Unix systems

(Linux and OpenBSD) are public goods maintained by volunteers. Why should

this be so? Why are markets for information goods and services so odd?

7.3 Information Economics

One of the insights from the nineteenth-century economists Jevons and Menger

is that the price of a good, at equilibrium, is the marginal cost of production.

When coal cost nine shillings a ton in 1870, that didn’t mean that every mine

dug coal at this price, merely that the marginal producers — those who were

only just managing to stay in business — could sell at that price. If the price

went down, these mines would close; if it went up, other, even more marginal

mines, would open. That’s how supply responded to changes in demand.

7.3.1 The Price of Information

So in a competitive equilibrium, the price of information should be its marginal

cost of production. But that is almost zero! This explains why there is so much

information available for free in the Internet; zero is its fair price. If two

or more suppliers compete to offer an operating system, or a map, or an

encyclopaedia, that they can duplicate for no cost, then the incentive will be

for them to keep on cutting their prices without limit. This is what happened

with encyclopaedias; the Britannica used to cost $1,600 for 32 volumes; then

Microsoft brought out Encarta for $49.95, forcing Britannica to produce a

cheap CD edition; and now we have Wikipedia for free [1159]. One ﬁrm after

another has had to move to a business model in which the goods are given

away free, and the money comes from advertising or in some parallel market.

7.3 Information Economics 221

Linux companies give away an operating system, and make their money from

support; many Linux developers give their time free to the project while at

college, as their contribution strengthens their CV and helps them get a good

job when they graduate.

Many other industries with high ﬁxed costs and low marginal costs moved

to an advertising or service model; think terrestrial TV. Others have moved in

this direction: most newspapers made most of their money from advertising,

and so have had little difﬁculty moving to free online editions plus paid paper

editions, all putting the lucrative ads in front of eyeballs. Yet other industries,

such as airlines and hotels, tended instead to become monopolists who try to

dominate particular routes or areas and to charge different prices to different

customers.

So what other characteristics of the information goods and services indus-

tries are particularly important?

1. There are often network externalities, whereby the value of a network

grows more than linearly in the number of users. For example, the more

people used fax machines in the 1980s, the more useful they became,

until round about 1985 fax machines suddenly took off; every business

needed one. Much the same happened with email in about 1995. Network

effects also apply to services more generally: anyone wanting to auction

some goods will usually go to the largest auction house, as it will attract

more bidders. They also apply to software: ﬁrms develop software for

Windows so they will have access to more users than they would if they

developed for Linux or Mac, and users for their part prefer Windows

because there’s more software for it. (This is called a two-sided market.)

2. There is often technical lock-in stemming from interoperability. Once a

software ﬁrm is committed to using Windows as a platform for its prod-

uct, it can be expensive to change; for users, too, changing platforms can

be expensive. They have to buy new software, convert ﬁles (if they can),

and retrain themselves.

These features separately can lead to industries with dominant ﬁrms;

together, they are even more likely to. If users simply want to be compat-

ible with other users (and software vendors) then they will logically buy from

the vendor they expect to win the biggest market share.

7.3.2 The Value of Lock-In

There is an interesting result, due to Shapiro and Varian: that the value of

a software company is the total lock-in (due to both technical and network

effects) of all its customers [1159]. To see how this might work, consider a ﬁrm

with 100 staff each using Ofﬁce, for which it has paid $500 per copy. It could

222 Chapter 7 ■Economics

save this $50,000 by moving to OpenOfﬁce, so if the costs of installing this

product, retraining its staff, converting ﬁles and so on — in other words the

total switching costs — were less than $50,000, it would switch. But if the costs

of switching were more than $50,000, then Microsoft would put up its prices.

Technical lock-in existed before, but the move to the digital economy has

made it much more signiﬁcant. If you own a Volvo car, you are locked in to

Volvo spares and accessories; but when you’re fed up with it you can always

trade it in for a Mercedes. But if you own an Apple Mac, you’ll have Mac

software, a Mac printer, and quite possibly thousands of music tracks that

you’ve ripped to iTunes. You’d also have to learn to use different commands

and interfaces. Moving to Windows would be much more painful than just

shelling out $700 for a new laptop. You’d have to retrain yourself; you’d have to

throw away Ofﬁce for Mac and buy Ofﬁce for Windows. And if you’d bought

a lot of music tracks from the iTunes music store, it could be more painful still

(you’d probably decide to keep your iPod with your new Windows machine

rather than moving to a Windows music player, even though the iPod works

better with the Mac). This shows why lock-in can be so durable; although each

piece of equipment — be it a Mac laptop, an iPod, or a printer — wears out,

the lock-in persists in the complementary relationship between them. And this

doesn’t just apply to PC platforms, but to ISPs; commercial software systems

such as databases; equipment such as telephone exchanges; and various online

services.

This is why so much effort gets expended in standards wars and antitrust

suits. It’s also why so many security mechanisms now aim at controlling

compatibility. In such cases, the likely hackers are not malicious outsiders, but

the owners of the equipment, or new ﬁrms trying to challenge the incumbent

by making compatible products. The issues are made more complex by the fact

that innovation is often incremental, and products succeed when new ﬁrms

ﬁnd killer applications for them [607]. The PC, for example, was designed by

IBM as a machine to run spreadsheets; if they had locked it down to this

application alone, then a massive opportunity would have been lost. Indeed,

the fact that the IBM PC was more open than the Apple Mac was a factor in its

becoming the dominant desktop platform.

So the law in many countries gives companies a right to reverse-engineer

their competitors’ products for compatibility [1110]. More and more, security

mechanisms are being used to try to circumvent that law: incumbents try to

lock down their platforms using cryptography and tamper-resistance so that

even if competitors have the legal right to try to reverse engineer them, they

are not always going to succeed in practice. Many businesses are seeing brutal

power struggles for control of the supply chain; for example, mobile phone

makers’ attempts to introduce sophisticated DRM into their handsets were frus-

trated by network operators determined to prevent the handset makers from

7.4 Game Theory 223

establishing business relationships directly with their customers. These

struggles set the scene in which more and more security products succeed

or fail.

7.3.3 Asymmetric Information

Another of the ways in which markets can fail, beyond monopoly and public

goods, is when some principals know more than others. The study of asymmetric

information was kicked off by a famous paper in 1970 on the ‘market for

lemons’ [19], for which George Akerlof won a Nobel prize. It presents the

following simple yet profound insight: suppose that there are 100 used cars

for sale in a town: 50 well-maintained cars worth $2000 each, and 50 ‘lemons’

worth $1000. The sellers know which is which, but the buyers don’t. What is

the market price of a used car? You might think $1500; but at that price no

good cars will be offered for sale. So the market price will be close to $1000.

This is one reason poor security products predominate. When users can’t tell

good from bad, they might as well buy a cheap antivirus product for $10 as a

better one for $20, and we may expect a race to the bottom on price.

A further distinction can be drawn between hidden information and hidden

action. For example, Volvo has a reputation for building safe cars that survive

accidents well, yet it is well known that Volvo drivers have more accidents.

Is this because people who know they’re bad drivers buy Volvos so they’re

less likely to get killed, or because people in Volvos drive faster? The ﬁrst is

the hidden-information case, also known as adverse selection, and the second

is the hidden-action case, also known as moral hazard. Both effects are important

in security, and both may combine in speciﬁc cases. (In the case of drivers, there

seems to be a growing consensus that people adjust their driving behaviour

to keep their risk exposure to the level with which they are comfortable.

This also explains why mandatory seat-belt laws tend not to save lives

overall, merely to move fatalities from vehicle occupants to pedestrians and

cyclists [10].)

Asymmetric information explains many market failures in the real world,

from low prices in used-car markets to the difﬁculty that older people have

in getting insurance on reasonable terms (people who know they’re sick will

tend to buy more of it, making it uneconomic for the healthy). It tends to lead

to surveillance or rationing.

7.4 Game Theory

There are really just two ways to get something you want if you can’t ﬁnd

or make it yourself. You either make something useful and trade it; or you

224 Chapter 7 ■Economics

take what you need, by force, by the ballot box or whatever. Choices between

cooperation and conﬂict are made every day at all sorts of levels, by both

humans and animals.

The main tool we can use to study and analyse them is game theory, which

I will deﬁne as ‘the study of problems of cooperation and conﬂict among

independent decision makers’. We’re interested in games of strategy rather

than games of chance, and we’re less interested in games of perfect informa-

tion (such as chess) than in games of imperfect information, which can be

much more interesting. We try to get to the core of games by abstracting

away much of the detail. For example, consider the school playground game

of ‘matching pennies’: Alice and Bob toss coins and reveal them simultane-

ously, upon which Alice gets Bob’s penny if they’re different and Bob gets

Alice’s penny if they’re the same. I’ll write this as shown in Figure 7.2:

Bob

Alice

H T

H−1,1 1,−1

T1,-1 −1,1

Figure 7.2: Matching pennies

Each entry in the table shows ﬁrst Alice’s outcome and then Bob’s outcome.

Thus if the coins fall (H,H) Alice loses a penny and Bob gains a penny. This is

an example of a zero-sum game: Alice’s gain is Bob’s loss.

Often we can solve a game quickly by writing out a payoff matrix like this.

Here’s an example (Figure 7.3):

Bob

Alice

Left Right

Top 1,2 0,1

Bottom 2,1 1,0

Figure 7.3: Dominant strategy equilibrium

In this game, no matter what Bob plays, Alice is better off playing ‘Bottom’;

and no matter what Alice plays, Bob is better off playing ‘Left’. Each player

has a dominant strategy — an optimal choice regardless of what the other does.

So Alice’s strategy should be a constant ‘Bottom’ and Bob’s a constant ‘Left’.

(A strategy in game theory is just an algorithm that takes a game state and

outputs a move.) We call this a dominant strategy equilibrium.

7.4 Game Theory 225

Another example is shown in Figure 7.4:

Bob

Alice

Left Right

Top 2,1 0,0

Bottom 0,0 1,2

Figure 7.4: Nash equilibrium

Here each player’s optimal strategy depends on what the other player does,

or (perhaps more accurately) what they think the other player will do. We

say that two strategies are in Nash equilibrium when Alice’s choice is optimal

given Bob’s, and vice versa. Here there are two symmetric Nash equilibria, at

top left and bottom right. You can think of them as being like local optima

while a dominant strategy equilibrium is a global optimum.

7.4.1 The Prisoners’ Dilemma

We’re now ready to look at a famous problem posed in 1950 by John Nash, and

for which he won the Nobel. It applies to many situations from international

trade negotiations to free-riding in peer-to-peer ﬁle-sharing systems to coop-

eration between hunting animals, and Nash ﬁrst studied it in the context of US

and USSR defense spending; his employer, the Rand corporation, was paid

to think about possible strategies in nuclear war. However, Nash presented it

using the following simple example.

Two prisoners are arrested on suspicion of planning a bank robbery. The

police interview them separately and tell each of them the following: ‘‘If neither

of you confesses you’ll each get a year for carrying a concealed ﬁrearm without

a permit. If one of you confesses, he’ll go free and the other will get 6 years for

conspiracy to rob. If both of you confess, you will each get three years.’’

What should the prisoners do? Let’s write the game out formally, as shown

in Figure 7.5:

Benjy

Alﬁe

Confess Deny

Confess −3,−30,−6

Deny −6,0 −1,−1

Figure 7.5: The prisoners’ dilemma

When Alﬁe looks at this table, he will reason as follows: ‘If Benjy’s going

to confess then I should too as then I get 3 years rather than 6; and if he’s

226 Chapter 7 ■Economics

going to deny then I should still confess as that way I walk rather than doing

a year’. Benjy will reason similarly. The two of them will each confess, and get

three years each. This is not just a Nash equilibrium; it’s a dominant strategy

equilibrium. Each prisoner should logically confess regardless of what the

other does.

But hang on, you say, if they had agreed to keep quiet then they’ll get a

year each, which is a better outcome for them! In fact the strategy (deny,deny)

is Pareto efﬁcient, while the dominant strategy equilibrium is not. (That’s one

reason it’s useful to have concepts like ‘Pareto efﬁcient’ and ‘dominant strategy

equilibrium’ rather than just arguing over ‘best’.)

So what’s the solution? Well, so long as the game is going to be played once

only, and this is the only game in town, there isn’t a solution. Both prisoners

will logically confess and get three years. We can only change this state of

affairs if somehow we can change the game itself. There are many possibilities:

there can be laws of various kinds from international treaties on trade to

the gangster’s omert´

a. In practice, a prisoner’s dilemma game is changed by

altering the rules or the context so as to turn it into another game where the

equilibrium is more efﬁcient.

7.4.2 Evolutionary Games

An important class of problems can be solved where the game is played

repeatedly — if Alﬁe and Benjy are career criminals who expect to be dealing

with each other again and again. Then of course there can be an incentive for

them to cooperate. There are at least two ways of modelling this.

In the 1970s, Bob Axelrod started thinking about how people might play

many rounds of prisoners’ dilemma. He set up a series of competitions to

which people could submit programs, and these programs played each other

repeatedly in tournaments. He found that one of the best strategies overall

was tit-for-tat, which is simply that you cooperate in round one, and at each

subsequent round you do to your opponent what he or she did in the previous

round [99]. It began to be realised that strategy evolution could explain a lot.

For example, in the presence of noise, players tend to get locked into (defect,

defect) whenever one player’s cooperative behaviour is misread by the other

as defection. So in this case it helps to ‘forgive’ the other player from time

to time.

Simultaneously, a parallel approach was opened up by John Maynard

Smith and George Price [848]. They considered what would happen if you had

a mixed population of aggressive and docile individuals, ‘hawks’ and ‘doves’,

with the behaviour that doves cooperate; hawks take food from doves; and

hawks ﬁght, with a risk of death. Suppose the value of the food at each

7.4 Game Theory 227

interaction is Vand the risk of death in a hawk ﬁght per encounter is C. Then

the payoff matrix looks like Figure 7.6:

Hawk Dove

Hawk V−C

2,V−C

2V,0

Dove 0, VV

2,V

Figure 7.6: The hawk-dove game

Here, if V>C, the whole population will become hawk, as that’s the

dominant strategy, but if C>V(ﬁghting is too expensive) then there is an

equilibrium where the probability pthat a bird is a hawk sets the hawk payoff

and the dove payoff equal, that is

pV−C

2+(1 −p)V=(1 −p)V

which is solved by p=V/C. In other words, you can have aggressive and

docile individuals coexisting in a population, and the proportion of aggressive

individuals will at equilibrium be a function of the costs of aggression; the more

dangerous it is, the fewer such individuals there will be. Of course, the costs

can change over time, and diversity is a good thing in evolutionary terms as

a society with a minority of combative individuals may be at an advantage

when war breaks out. Again, it takes generations for a society to move to

equilibrium. Perhaps our current incidence of aggression is too high because it

reﬂects conditions in the Dark Ages, or even on the African highveld 500,000

years ago1.

This neat insight, along with Bob Axelrod’s simulation methodology for

tackling problems that don’t have a neat algebraic solution, got many people

from moral philosophers to students of animal behaviour interested in evolu-

tionary game theory. They give deep insights into how cooperation evolved.

It turns out that many primates have an inbuilt sense of fairness and punish

individuals who are seen to be cheating — the instinct for vengeance is part of

the mechanism to enforce sociality. Fairness can operate in a number of differ-

ent ways at different levels. For example, the philosopher Brian Skyrms found

that doves can get a better result against hawks if they can recognise each other

and interact preferentially, giving a model for how social movements such as

freemasons and maybe even some religions establish themselves [1188].

1A number of leading anthropologists believe that, until recent times, tribal warfare was endemic

among human societies [777].

228 Chapter 7 ■Economics

Of course, the basic idea behind tit-for-tat goes back a long way. The Old

Testament has ‘An eye for an eye’ and the New Testament ‘Do unto others

as you’d have them do unto you’; the latter formulation is, of course, more

fault-tolerant, and versions of it can be found in Aristotle, in Confucius and

elsewhere. More recently, Thomas Hobbes used primitive prisoners’-dilemma-

style arguments in the seventeenth century to justify the existence of a state

without the Divine Right of Kings.

The applications of evolutionary game theory keep on growing. Since 9/11,

for example, there has been interest in whether hawk-dove games explain the

ability of fundamentalists to take over discourse in religions at a time of stress.

From the economists’ viewpoint, evolutionary games explain why cartel-like

behaviour can appear in industries even where there are no secret deals being

done in smoke-ﬁlled rooms. For example, if there are three airlines operating

a proﬁtable route, and one lowers its prices to compete for volume, the others

may well respond by cutting prices even more sharply to punish it and make

the route unproﬁtable, in the hope that the discounts will be discontinued and

everyone can go back to gouging the customer. And there are some interesting

applications in security, too, which I’ll come to later.

7.5 The Economics of Security and Dependability

Economists used to be well aware of the interaction between economics and

security; rich nations could afford big armies. But nowadays a web search on

‘economics’ and ‘security’ turns up relatively few articles. The main reason

is that, after 1945, economists drifted apart from people working on strategic

studies; nuclear weapons were thought to decouple national survival from

economic power [839]. A secondary factor may have been that the USA

confronted the USSR over security, but Japan and the EU over trade. It has

been left to the information security world to re-establish the connection.

One of the observations that rekindled interest in security economics came

from banking. In the USA, banks are generally liable for the costs of card

fraud; when a customer disputes a transaction, the bank must either show

she is trying to cheat it, or refund her money. In the UK, banks generally got

away with claiming that their systems were ‘secure’, and telling customers

who complained that they must be mistaken or lying. ‘Lucky bankers,’ you

might think; yet UK banks spent more on security and suffered more fraud.

This was probably a moral-hazard effect: UK bank staff knew that customer

complaints would not be taken seriously, so they became lazy and careless,

leading to an epidemic of fraud [33, 34].

Another was that people were not spending as much money on anti-virus

software as the vendors might have hoped. Now a typical virus payload then

was a service-denial attack on Microsoft; and while a rational consumer might

7.5 The Economics of Security and Dependability 229

spend $20 to stop a virus trashing her hard disk, she will be less likely to do so

just to protect a wealthy corporation [1290]. There are many other examples,

such as hospital systems bought by medical directors and administrators that

look after their interests but don’t protect patient privacy. The picture that

started to emerge was of system security failing because the people guarding

a system were not the people who suffered the costs of failure, or of particular

types of failure. Sometimes, as we’ll see, security mechanisms are used to

dump risks on others, and if you are one of these others you’d be better off

with an insecure system. Put differently, security is often not a scalar, but a

power relationship; the principals who control what it means in a given system

often use it to advance their own interests.

This was the initial insight. But once we started studying security economics

seriously, we found that there’s a lot more to it than that.

7.5.1 Weakest Link, or Sum of Efforts?

The late Jack Hirshleifer, the founder of conﬂict theory, told the story of

Anarchia, an island whose ﬂood defences were constructed by individual

families who each maintained a section of the ﬂood wall. The island’s ﬂood

defence thus depended on the weakest link, that is, the laziest family. He

compared this with a city whose defences against missile attack depend on

the single best defensive shot [609]. Another example of best-shot is medieval

warfare, where there was on occasion a single combat between the two armies’

champions. Hal Varian extended this model to three cases of interest to

the dependability of information systems — where performance depends

on the minimum effort, the best effort, or the sum-of-efforts [1292]. This

last case, the sum-of-efforts, is the modern model for warfare: we pay our

taxes and the government hires soldiers. It’s a lot more efﬁcient than best-shot

(where most people will free-ride behind the heroes), and that in turn is miles

better than weakest-link (where everyone will free-ride behind the laziest).

Program correctness can depend on minimum effort (the most careless

programmer introducing a vulnerability) while software vulnerability testing

may depend on the sum of everyone’s efforts. Security may also depend on

the best effort — the actions taken by an individual champion such as a

security architect. As more agents are added, systems become more reliable

in the total-effort case but less reliable in the weakest-link case. What are the

implications? Well, software companies should hire more software testers and

fewer (but more competent) programmers.

7.5.2 Managing the Patching Cycle

There has been much debate about ‘open source security’, and more generally

whether actively seeking and disclosing vulnerabilities is socially desirable.

230 Chapter 7 ■Economics

It’s a debate that has ﬂared up again and again; as we saw in the preface, the

Victorians agonised over whether it was socially responsible to publish books

about lockpicking, and eventually concluded that it was [1257]. People have

worried more recently about the online availability of (for example) the US

Army Improvised Munitions Handbook [1271]; is the risk of helping terrorists

sufﬁcient to justify online censorship?

Security economics provides both a theoretical and a quantitative framework

for discussing some issues of this kind. I showed in 2002 that, under standard

assumptions of reliability growth, open systems and proprietary systems are

just as secure as each other; opening up a system helps the attackers and defend-

ers equally [54]. Thus the open-security question will often be an empirical one,

turning on the extent to which a given real system follows the standard model.

In 2004, Eric Rescorla argued that for software with many latent vulnerabil-

ities, removing one bug makes little difference to the likelihood of an attacker

ﬁnding another one later. Since exploits are often based on vulnerabilities

inferred from patches, he argued against disclosure and frequent patching

unless the same vulnerabilities are likely to be rediscovered [1071]. Ashish

Arora and others responded with data showing that public disclosure made

vendors respond with ﬁxes more quickly; attacks increased to begin with,

but reported vulnerabilities declined over time [88]. In 2006, Andy Ozment

and Stuart Schechter found that the rate at which unique vulnerabilities were

disclosed for the core OpenBSD operating system has decreased over a six-year

period [998]. These results support the current system of responsible disclo-

sure whereby people who discover vulnerabilities report them to CERT, which

reports them on to vendors, and publicises them once patches are shipped.

This is by no means all that there is to say about the economics of depend-

ability. There are tensions between vendors and their customers over the

frequency and timing of patch release; issues with complementers; difﬁculties

with metrics; companies such as iDefense and TippingPoint that buy and sell

information on vulnerabilities; and even concerns that intelligence agencies

with privileged access to bug reports use them for zero-day exploits against

other countries’ systems. I’ll come back to all this in Part III.

7.5.3 Why Is Windows So Insecure?

The micromanagement of the patching cycle begs a deeper question: why

are there so many bugs in the ﬁrst place? In particular, why is Windows

so insecure, despite Microsoft’s dominant market position? It’s possible to

write much better software, and there are ﬁelds such as defense and healthcare

where a serious effort is made to produce dependable systems. Why do

we not see a comparable effort made with commodity platforms, especially

since Microsoft has no real competitors?

7.5 The Economics of Security and Dependability 231

To be honest, Microsoft’s software security is improving. Windows 95

was dreadful, Windows 98 slightly better, and the improvement’s continued

through NT, XP and Vista. But the attackers are getting better too, and the

protection in Vista isn’t all for the user’s beneﬁt. As Peter Gutmann points

out, enormous effort has gone into protecting premium video content, and

almost no effort into protecting users’ credit card numbers [570]. The same

pattern has also been seen in other platform products, from the old IBM

mainframe operating systems through telephone exchange switches to the

Symbian operating system for mobile phones. Products are insecure at ﬁrst,

and although they improve over time, many of the new security features are

for the vendor’s beneﬁt as much as the user’s.

By now, you should not ﬁnd this surprising. The combination of high ﬁxed

and low marginal costs, network effects and technical lock-in makes platform

markets particularly likely to be dominated by single vendors, who stand to

gain vast fortunes if they can win the race to dominate the market. In such

a race, the notorious Microsoft philosophy of the 1990s — ‘ship it Tuesday

and get it right by version 3’ — is perfectly rational behaviour. In such a race,

the platform vendor must appeal at least as much to complementers — to the

software companies who decide whether to write applications for its platform

or for someone else’s. Security gets in the way of applications, and it tends

to be a lemons market anyway. So the rational vendor will enable (indeed

encourage) all applications to run as root, until his position is secure. Then he

will add more security — but there will still be a strong incentive to engineer

it in such a way as to maximise customer lock-in, or to appeal to complementers

in new markets such as digital media.

From the viewpoint of the consumer, markets with lock-in are often ‘bargains

then rip-offs’. You buy a nice new printer for $39.95, then ﬁnd to your disgust

after just a few months that you need two new printer cartridges for $19.95

each. You wonder whether you’d not be better off just buying a new printer.

From the viewpoint of the application developer, markets with standards races

based on lock-in look a bit like this. At ﬁrst it’s really easy to write code for

them; later on, once you’re committed, there are many more hoops to jump

through. From the viewpoint of the poor consumer, they could be described

as ‘poor security, then security for someone else’.

Sometimes it can be worse than that. When racing to establish a dominant

position, vendors are likely to engineer their products so that the cost of

managing such security as there is falls on the user, rather than on the

application developers. A classic example is SSL/TLS encryption. This was

adopted in the mid-1990s as Microsoft and Netscape battled for dominance of

the browser market. As I discussed in Chapter 2, SSL leaves it up to the user

to assess the certiﬁcate offered by a web site and decide whether to trust it;

and this has turned out to facilitate all kinds of phishing and other attacks. Yet

dumping the compliance costs on the user made perfect sense at the time, and

232 Chapter 7 ■Economics

competing protocols such as SET, that would have required heavier investment

by banks and merchants, were allowed to wither on the vine [357]. The world

has ended up not just with a quite insecure system of Internet payments,

but with widespread liability dumping that makes progress towards a better

system difﬁcult. Too much of the infrastructure has weakest-link rather than

sum-of-efforts security.

7.5.4 Economics of Privacy

The big conundrum with privacy is that people say that they value privacy,

yet act otherwise. If you stop people in the street and ask them their views,

about a third say they are privacy fundamentalists and will never hand over

their personal information to marketers or anyone else; about a third say they

don’t care; and about a third are in the middle, saying they’d take a pragmatic

view of the risks and beneﬁts of any disclosure. However, the behavior that

people exhibit via their shopping behavior — both online and ofﬂine — is

quite different; the great majority of people pay little heed to privacy, and will

give away the most sensitive information for little beneﬁt. Privacy-enhancing

technologies have been offered for sale by various ﬁrms, yet most have failed

in the marketplace. Why should this be?

Privacy is one aspect of information security that interested economists

before 2000. In 1978, Richard Posner deﬁned privacy in terms of secrecy [1035],

and the following year extended this to seclusion [1036]. In 1980, Jack

Hirshleifer published a seminal paper in which he argued that rather than

being about withdrawing from society, privacy was a means of organising

society, arising from evolved territorial behavior; internalised respect for prop-

erty is what allows autonomy to persist in society. These privacy debates in

the 1970s led in Europe to generic data-protection laws, while the USA limited

itself to a few sector-speciﬁc laws such as HIPAA. Economists’ appetite for

work on privacy was further whetted recently by the Internet, the dotcom

boom, and the exploding trade in personal information about online shoppers.

An early modern view of privacy can be found in a 1996 paper by Hal Varian

who analysed privacy in terms of information markets [1289]. Consumers

want to not be annoyed by irrelevant marketing calls while marketers do

not want to waste effort. Yet both are frustrated, because of search costs,

externalities and other factors. Varian suggested giving consumers rights in

information about themselves, and letting them lease it to marketers with the

proviso that it not be resold without permission.

The recent proliferation of complex, information-intensive business models

demanded a broader approach. Andrew Odlyzko argued in 2003 that privacy

erosion is a consequence of the desire to charge different prices for similar

services [981]. Technology is simultaneously increasing both the incentives

and the opportunities for price discrimination. Companies can mine online

7.5 The Economics of Security and Dependability 233

purchases and interactions for data revealing individuals’ willingness to

pay. From airline yield-management systems to complex and ever-changing

software and telecommunications prices, differential pricing is economically

efﬁcient — but increasingly resented. Peter Swire argued that we should

measure the costs of privacy intrusion more broadly [1237]. If a telesales

operator calls 100 prospects, sells three of them insurance, and annoys 80,

then the conventional analysis considers only the beneﬁt to the three and to

the insurer. However, persistent annoyance causes millions of people to go

ex-directory, to not answer the phone during dinner, or to screen calls through

an answering machine. The long-run societal harm can be considerable.

Several empirical studies have backed this up by examining people’s privacy

valuations.

My own view on this is that it simply takes time for the public to assimilate

the privacy risks. For thirty years or so, IT policy folks have been agonising

about the death of privacy, but this remained a geek interest until recently.

The signiﬁcance is now starting to percolate down to sophisticated people

like stock-market investors: Alessandro Acquisti and others have found that

the stock price of companies reporting a security or privacy breach is likely

to fall [8, 265]. It’s only when tabloid newspapers and talk-radio shows give

lots of coverage to stories of ordinary people who’ve suffered real harm as a

result of ‘identity theft’ and phishing that the average voter will start to sit

up and take notice. There are some early signs that this is starting to happen

(for example in the growing number of requests that privacy experts like me

get to appear on radio and TV shows). But another behavioural economist,

George Loewnstein, points out that people are more sensitive to large changes

in their circumstances rather than to small ones: they will get concerned if

things suddenly get worse, but not if they get worse gradually. They also

become habituated surprisingly easily to bad circumstances that they don’t

believe they can change.

It may be of particular interest that, in late 2007, the British government

suffered spectacular embarrassment when it lost the electronic tax records

on all the nation’s children and their families — including bank account

details — leading to a personal apology in Parliament from the Prime Minister

and massive media coverage of subsequent privacy breaches. I’ll discuss this

in more detail in section 9.4; the privacy economist’s interest will be in whether

this changes public attitudes in any measurable way over time, and whether

attitudes stay changed.

7.5.5 Economics of DRM

Rights-management technologies have also come in for economic scrutiny.

Hal Varian pointed out in 2002 that DRM and similar mechanisms were also

about tying, bundling and price discrimination; and that their unfettered use

234 Chapter 7 ■Economics

could damage competition [1291]. I wrote an FAQ on ‘Trusted Computing’

in 2003, followed by a research paper, in which I pointed out the potential

for competitive abuse of rights management mechanisms; for example, by

transferring control of user data from the owner of the machine on which it is

stored to the creator of the ﬁle in which it is stored, the potential for lock-in

is hugely increased [53]. Think of the example above, in which a law ﬁrm

of 100 fee earners each has a PC on which they install Ofﬁce for $500. The

$50,000 they pay Microsoft is roughly equal to the total costs of switching to

(say) OpenOfﬁce, including training, converting ﬁles and so on. However, if

control of the ﬁles moves to its thousands of customers, and the ﬁrm now

has to contact each customer and request a digital certiﬁcate in order to

migrate the ﬁle, then clearly the switching costs have increased — so you can

expect the cost of Ofﬁce to increase too, over time.

There are some interesting angles on the debate about rights management in

music too. In 2004, Felix Oberholzer and Koleman Strumpf published a now-

famous paper, in which they examined how music downloads and record

sales were correlated [978]. They showed that downloads do not do signiﬁcant

harm to the music industry. Even in the most pessimistic interpretation,

ﬁve thousand downloads are needed to displace a single album sale, while

high-selling albums actually beneﬁt from ﬁle sharing. This research was

hotly disputed by music-industry spokesmen at the time, but has since been

conﬁrmed by Canadian government research that found a positive correlation

between downloading and CD sales among peer-to-peer system users, and no

correlation among the population as a whole [28].

In January 2005, Hal Varian made a controversial prediction [1293]: that

stronger DRM would help system vendors more than the music industry,

because the computer industry is more concentrated (with only three seri-

ous suppliers of DRM platforms — Microsoft, Sony, and the dominant ﬁrm,

Apple). The content industry scoffed, but by the end of that year music pub-

lishers were protesting that Apple was getting too large a share of the cash

from online music sales. As power in the supply chain moved from the music

majors to the platform vendors, so power in the music industry appears to

be shifting from the majors to the independents, just as airline deregulation

favoured aircraft makers and low-cost airlines. This is a striking demonstration

of the predictive power of economic analysis. By ﬁghting a non-existent threat,

the record industry helped the computer industry forge a weapon that may be

its undoing.

7.6 Summary

Many systems fail because the incentives are wrong, rather than because of

some technical design mistake. As a result, the security engineer needs to

Further Reading

The best initial introduction to information economics is Shapiro and Var-

ian’s ‘Information Rules’ which remains remarkably fresh and accurate for a

book written ten years ago [1159]. I generally recommend that students read

this ﬁrst. For those who want to go on to do research in the subject, I then

suggest Hal Varian’s textbook ‘Intermediate Microeconomics’ which covers the

material from an academic viewpoint, with fewer case histories and more

mathematics [1284].

The current research in security economics is published mostly at the Work-

shop on the Economics of Information Security (WEIS), which has been held

annually since 2002; details of WEIS, and other relevant events, can be found

at [58]. There is a current (2007) survey of the ﬁeld, that I wrote with Tyler

Moore, at [72]. There are two books of collected research papers [257, 548], and

a popular account by Bruce Schneier [1129]; I also maintain an Economics and

Security Resource Page at http://www.cl.cam.ac.uk/ rja14/econsec.html.

Two other relevant papers, which are in press as this book goes to print,

are a report I’m writing with Rainer B¨

ohme, Richard Clayton and Tyler

Moore on security economics in the European internal market [62], and an

OECD report by Michel van Eeten and colleagues that reports extensive inter-

views with information security stakeholders about the incentives they face in

practice [420].

A number of economists study related areas. I mentioned Jack Hirshleifer’s

conﬂict theory; a number of his papers are available in a book [610]. Another

really important strand is the economics of crime, which was kick-started

by Gary Becker [138], and has recently been popularised by Steve Levitt and

236 Chapter 7 ■Economics

Stephen Dubner’s ‘Freakonomics’ [787]. Much of this analyses volume crime

by deprived young males, an issue to which I’ll return in Chapter 11; but some

scholars have also studied organised crime [392, 473]. As computer crime is

increasingly driven by the proﬁt motive rather than by ego and bragging

rights, we can expect economic analyses to be ever more useful.

PART

In this second part of the book, I describe a large

number of applications of secure systems, many of

which introduce particular protection concepts or

technologies.

There are three broad themes. Chapters 8–10 look

at conventional computer security issues, and by dis-

cussing what one is trying to do and how it’s done

in different environments — the military, banks and

healthcare — we introduce security policy models

which set out the protection concepts that real systems

try to implement. We introduce our ﬁrst detailed case

studies. An example is the worldwide network of auto-

matic teller machines, which illustrates many of the

problems of transferring familiar protection properties

from a bank branch to a global distributed environment

using cryptography.

Chapters 10–18 look at the hardware and system

engineering aspects of information security. This in-

cludes biometrics, the design of various tokens such

as smartcards, and the whole panoply of hardware

security mechanisms from physical locks through chip-

level tamper-resistance and emission security to secu-

rity printing and seals. New applications that illustrate

the technologies are described, ranging from elec-

tronic warfare and nuclear weapon control through

burglar alarms, truck speed limiters and prepayment

gas meters. We end up with a chapter on the security of

application programming interfaces, where hardware

and software security meet.

238 Part II

The third theme is attacks on networks, and on highly-networked sys-

tems. We start off with electronic and information warfare in Chapter 19, as

these activities give some of the more extreme examples and show how far

techniques of denial, deception and exploitation can be taken by a resourceful

opponent under severe operational pressure. It also gives a view of surveillance

and intrusion from the intelligence agencies’ point of view, and introduces

concepts such as anonymity and trafﬁc analysis. We then study the lessons

of history by examining frauds on phone systems, and on applications that

rely on them, in Chapter 20. This sets the scene for a discussion in Chapter 21

of attacks on computer networks and defensive technologies ranging from

ﬁrewalls to protocols such as SSH, TLS, IPSEC and Bluetooth. Chapter 22

deals with copyright and DRM, and related topics such as information hiding.

Finally, in Chapter 23 I present four vignettes of the ‘bleeding edge’ of security

research in 2007: computer games; web applications (such as auctions, search

and social networking); privacy technology (from email anonymity and web

proxies to forensics countermeasures); and elections.

One reason for this ordering is to give the chapters a logical progression.

Thus, for example, I discuss frauds against magnetic stripe bank cards before

going on to describe the smartcards which may replace them and the pay-TV

systems which actually use smartcards today.

Sometimes a neat linear ordering isn’t possible as a particular technology has

evolved through a number of iterations over several applications. In such cases

I try to distill what I know into a case history. To keep the book manageable

for readers who use it primarily as a reference rather than as a textbook, I have

put the more technical material towards the end of each chapter or section: if

you get lost at a ﬁrst reading then do just skip to the next section and carry on.

CHAPTER

Multilevel Security

Most high assurance work has been done in the area of kinetic devices

and infernal machines that are controlled by stupid robots. As

information processing technology becomes more important

to society, these concerns spread to areas

previously thought inherently harmless,

like operating systems.

— Earl Boebert

I brief;

you leak;

he/she commits a criminal offence

by divulging classiﬁed information.

— British Civil Service Verb

They constantly try to escape

From the darkness outside and within

By dreaming of systems so perfect that no one will need to be good

— TS Eliot

8.1 Introduction

I mentioned in the introduction that military database systems, which can

hold information at a number of different levels of classiﬁcation (Conﬁdential,

Secret, Top Secret, ...), have to ensure that data can only be read by a principal

whose level is at least as high as the data’s classiﬁcation. The policies they

implement are known as multilevel secure or alternatively as mandatory access

control or MAC.

239

240 Chapter 8 ■Multilevel Security

Multilevel secure systems are important because:

1. a huge amount of research has been done on them, thanks to military

funding for computer science in the USA. So the military model of protec-

tion has been worked out in much more detail than any other, and it gives

us a lot of examples of the second-order and even third-order effects of

implementing a security policy rigorously;

2. although multilevel concepts were originally developed to support con-

ﬁdentiality in military systems, many commercial systems now use mul-

tilevel integrity policies. For example, telecomms operators want their

billing system to be able to see what’s happening in their switching sys-

tem, but not affect it;

3. recently, products such as Microsoft Vista and Red Hat Linux have started

to incorporate mandatory access control mechanisms, and they have also

appeared in disguise in digital rights management systems. For example,

Red Hat uses SELinux mechanisms developed by the NSA to isolate dif-

ferent servers running on a machine — so that even if your web server is

hacked, it doesn’t necessarily follow that your DNS server gets taken over

too. Vista has a multilevel integrity policy under which Internet Explorer

runs by default at ‘Low’ — which means that even if it gets taken over,

the attacker should not be able to change system ﬁles, or anything else

with a higher integrity level. These mechanisms are still largely invisible

to the domestic computer user, but their professional use is increasing;

4. multilevel conﬁdentiality ideas are often applied in environments where

they’re ineffective or even harmful, because of the huge vested inter-

ests and momentum behind them. This can be a contributory factor in

the failure of large system projects, especially in the public sector.

Sir Isiah Berlin famously described thinkers as either foxes or hedgehogs:

a fox knows many little things, while a hedgehog knows one big thing. The

multilevel philosophy is the hedgehog approach to security engineering.

8.2 What Is a Security Policy Model?

Where a top-down approach to security engineering is possible, it will typi-

cally take the form of threat model — security policy — security mechanisms. The

critical, and often neglected, part of this process is the security policy.

By a security policy, we mean a document that expresses clearly and

concisely what the protection mechanisms are to achieve. It is driven by our

understanding of threats, and in turn drives our system design. It will often

take the form of statements about which users may access which data. It

plays the same role in specifying the system’s protection requirements, and

8.2 What Is a Security Policy Model? 241

evaluating whether they have been met, that the system speciﬁcation does

for general functionality. Indeed, a security policy may be part of a system

speciﬁcation, and like the speciﬁcation its primary function is to communicate.

Many organizations use the phrase ‘security policy’ to mean a collection of

vapid statements. Figure 8.1 gives a simple example:

Megacorp Inc security policy

1. This policy is approved by Management.

2. All staff shall obey this security policy.

3. Data shall be available only to those with a ‘need-to-know’.

4. All breaches of this policy shall be reported at once to Security.

Figure 8.1: A typical corporate information security policy

This sort of wafﬂe is very common but is useless to the security engineer.

Its ﬁrst failing is it dodges the central issue, namely ‘Who determines ‘‘need-

to-know’’ and how?’ Second, it mixes statements at a number of different

levels (organizational approval of a policy should logically not be part of the

policy itself). Third, there is a mechanism but it’s implied rather than explicit:

‘staff shall obey’ — but what does this mean they actually have to do? Must the

obedience be enforced by the system, or are users ‘on their honour’? Fourth,

how are breaches to be detected and who has a speciﬁc duty to report them?

We must do better than this. In fact, because the term ‘security policy’ is

widely abused to mean a collection of managerialist platitudes, there are three

more precise terms which have come into use to describe the speciﬁcation of

protection requirements.

Asecurity policy model is a succinct statement of the protection properties

which a system, or generic type of system, must have. Its key points can

typically be written down in a page or less. It is the document in which

the protection goals of the system are agreed with an entire community, or

with the top management of a customer. It may also be the basis of formal

mathematical analysis.

Asecurity target is a more detailed description of the protection mechanisms

that a speciﬁc implementation provides, and how they relate to a list of control

objectives (some but not all of which are typically derived from the policy

model). The security target forms the basis for testing and evaluation of a

product.

Aprotection proﬁle is like a security target but expressed in an implementa-

tion-independent way to enable comparable evaluations across products and

versions. This can involve the use of a semi-formal language, or at least of

suitable security jargon. A protection proﬁle is a requirement for products that

are to be evaluated under the Common Criteria [935]. (I discuss the Common

242 Chapter 8 ■Multilevel Security

Criteria in Part III; they are associated with a scheme used by many govern-

ments for mutual recognition of security evaluations of defense information

systems.)

When I don’t have to be so precise, I may use the phrase ‘security policy’ to

refer to either a security policy model or a security target. I will never use it

to refer to a collection of platitudes.

Sometimes, we are confronted with a completely new application and have

to design a security policy model from scratch. More commonly, there already

exists a model; we just have to choose the right one, and develop it into a

security target. Neither of these steps is easy. Indeed one of the purposes of this

section of the book is to provide a number of security policy models, describe

them in the context of real systems, and examine the engineering mechanisms

(and associated constraints) which a security target can use to meet them.

Finally, you may come across a third usage of the phrase ‘security pol-

icy’ — as a list of speciﬁc conﬁguration settings for some protection product.

We will refer to this as conﬁguration management, or occasionally as trusted

conﬁguration management, in what follows.

8.3 The Bell-LaPadula Security Policy Model

The classic example of a security policy model was proposed by Bell and

LaPadula in 1973, in response to US Air Force concerns over the security of

time-sharing mainframe systems1. By the early 1970’s, people had realised that

the protection offered by many commercial operating systems was poor, and

was not getting any better. As soon as one operating system bug was ﬁxed,

some other vulnerability would be discovered. (Modern reliability growth

models can quantify this and conﬁrm that the pessimism was justiﬁed; I

discuss them further in section 26.2.4.) There was the constant worry that even

unskilled users would discover loopholes and use them opportunistically;

there was also a keen and growing awareness of the threat from malicious

code. (Viruses were not invented until the following decade; the 70’s concern

was about Trojans.) There was a serious scare when it was discovered that

the Pentagon’s World Wide Military Command and Control System was

vulnerable to Trojan Horse attacks; this had the effect of restricting its use

to people with a ‘Top Secret’ clearance, which was inconvenient. Finally,

academic and industrial researchers were coming up with some interesting

new ideas on protection, which I discuss below.

A study by James Anderson led the US government to conclude that a

secure system should do one or two things well; and that these protection

1This built on the work of a number of other researchers: see section 9.2.1 below for a sketch of

the technical history.

8.3 The Bell-LaPadula Security Policy Model 243

properties should be enforced by mechanisms which were simple enough to

verify and that would change only rarely [29]. It introduced the concept of a

reference monitor — a component of the operating system which would mediate

access control decisions and be small enough to be subject to analysis and

tests, the completeness of which could be assured. In modern parlance, such

components — together with their associated operating procedures — make

up the Trusted Computing Base (TCB). More formally, the TCB is deﬁned as the

set of components (hardware, software, human, ...) whose correct functioning

is sufﬁcient to ensure that the security policy is enforced, or, more vividly,

whose failure could cause a breach of the security policy. The Anderson

report’s goal was to make the security policy simple enough for the TCB to be

amenable to careful veriﬁcation.

But what are these core security properties that should be enforced above

all others?

8.3.1 Classiﬁcations and Clearances

The Second World War, and the Cold War which followed, led NATO

governments to move to a common protective marking scheme for labelling

the sensitivity of documents. Classiﬁcations are labels, which run upwards from

Unclassiﬁed through Conﬁdential,Secret and Top Secret (see Figure 8.2.). The

details change from time to time. The original idea was that information

whose compromise could cost lives was marked ‘Secret’ while information

whose compromise could cost many lives was ‘Top Secret’. Government

employees have clearances depending on the care with which they’ve been

vetted; in the USA, for example, a ‘Secret’ clearance involves checking FBI

ﬁngerprint ﬁles, while ‘Top Secret’ also involves background checks for the

previous ﬁve to ﬁfteen years’ employment [379].

The access control policy was simple: an ofﬁcial could read a document

only if his clearance was at least as high as the document’s classiﬁcation. So

an ofﬁcial cleared to ‘Top Secret’ could read a ‘Secret’ document, but not vice

versa. The effect is that information may only ﬂow upwards, from conﬁdential

to secret to top secret, but it may never ﬂow downwards unless an authorized

person takes a deliberate decision to declassify it.

There are also document handling rules; thus a ‘Conﬁdential’ document

might be kept in a locked ﬁling cabinet in an ordinary government ofﬁce,

TOP SECRET

SECRET

CONFIDENTIAL

UNCLASSIFIED

Figure 8.2: Multilevel security

244 Chapter 8 ■Multilevel Security

while higher levels may require safes of an approved type, guarded rooms

with control over photocopiers, and so on. (The NSA security manual [952]

gives a summary of the procedures used with ‘top secret’ intelligence data.)

The system rapidly became more complicated. The damage criteria for

classifying documents were expanded from possible military consequences

to economic harm and even political embarrassment. The UK has an extra

level, ‘Restricted’, between ‘Unclassiﬁed’ and ‘Conﬁdential’; the USA used

to have this too but abolished it after the Freedom of Information Act was

introduced. America now has two more speciﬁc markings: ‘For Ofﬁcial Use

only’ (FOUO) refers to unclassiﬁed data that can’t be released under FOIA,

while ‘Unclassiﬁed but Sensitive’ includes FOUO plus material which might be

released in response to a FOIA request. In the UK, ‘Restricted’ information is in

practice shared freely, but marking everything ‘Restricted’ allows journalists

and others involved in leaks to be prosecuted under Ofﬁcial Secrets law.

(Its other main practical effect is that an unclassiﬁed US document which

is sent across the Atlantic automatically becomes ‘Restricted’ in the UK and

then ‘Conﬁdential’ when shipped back to the USA. American military system

builders complain that the UK policy breaks the US classiﬁcation scheme!)

There is also a system of codewords whereby information, especially at

Secret and above, can be further restricted. For example, information which

might reveal intelligence sources or methods — such as the identities of agents

or decrypts of foreign government trafﬁc — is typically classiﬁed ‘Top Secret

Special Compartmented Intelligence’ or TS/SCI, which means that so-called

need to know restrictions are imposed as well, with one or more codewords

attached to a ﬁle. Some of the codewords relate to a particular military

operation or intelligence source and are available only to a group of named

users. To read a document, a user must have all the codewords that are

attached to it. A classiﬁcation label, plus a set of codewords, makes up a

security category or (if there’s at least one codeword) a compartment,whichis

a set of records with the same access control policy. I discuss compartmentation

in more detail in the chapter on multilateral security.

There are also descriptors,caveats and IDO markings. Descriptors are words

such as ‘Management’, ‘Budget’, and ‘Appointments’: they do not invoke any

special handling requirements, so we can deal with a ﬁle marked ‘Conﬁden-

tial — Management’ as if it were simply marked ‘Conﬁdential’. Caveats are

warnings such as ‘UK Eyes Only’, or the US equivalent, ‘NOFORN’; there are

also International Defence Organisation markings such as NATO. The lack of

obvious differences between codewords, descriptors, caveats and IDO mark-

ing is one of the things that can make the system confusing. A more detailed

explanation can be found in [1051].

The ﬁnal generic comment about access control doctrine is that allowing

upward-only ﬂow of information also models what happens in wiretapping.

In the old days, tapping someone’s telephone meant adding a physical wire

8.3 The Bell-LaPadula Security Policy Model 245

at the exchange; nowadays, it’s all done in the telephone exchange software

and the effect is somewhat like making the target calls into conference calls

with an extra participant. The usual security requirement is that the target

of investigation should not know he is being wiretapped, so the third party

should be silent — and its very presence must remain unknown to the target.

For example, now that wiretaps are implemented as silent conference calls,

care has to be taken to ensure that the charge for the conference call facility goes

to the wiretapper, not to the target. Wiretapping requires an information ﬂow

policy in which the ‘High’ principal can see ‘Low’ data, but a ‘Low’ principal

can’t tell whether ‘High’ is reading any data at all, let alone what data.

8.3.2 Information Flow Control

It was in this context of the classiﬁcation of government data that the Bell-

LaPadula or BLP model of computer security was formulated in 1973 [146]. It

is also known as multilevel security and systems which implement it are often

called multilevel secure or MLS systems. Their basic property is that information

cannot ﬂow downwards.

More formally, the Bell-LaPadula model enforces two properties:

The simple security property: no process may read data at a higher level.

This is also known as no read up (NRU);

The *-property: no process may write data to a lower level. This is also

known as no write down (NWD).

The *-property was Bell and LaPadula’s critical innovation. It was driven

by the fear of attacks using malicious code. An uncleared user might write a

Trojan and leave it around where a system administrator cleared to ‘Secret’

might execute it; it could then copy itself into the ‘Secret’ part of the system,

read the data there and try to signal it down somehow. It’s also quite possible

that an enemy agent could get a job at a commercial software house and embed

some code in a product which would look for secret documents to copy. If it

could then write them down to where its creator could read it, the security

policy would have been violated. Information might also be leaked as a result

of a bug, if applications could write down.

Vulnerabilities such as malicious and buggy code are assumed to be given. It

is also assumed that most staff are careless, and some are dishonest; extensive

operational security measures have long been used, especially in defence

environments, to prevent people leaking paper documents. (When I worked

in defense avionics as a youngster, all copies of circuit diagrams, blueprints

etc were numbered and had to be accounted for.) So there was a pre-existing

culture that security policy was enforced independently of user actions; the

move to computers didn’t change this. It had to be clariﬁed, which is what Bell-

LaPadula does: the security policy must be enforced not just independently of

246 Chapter 8 ■Multilevel Security

users’ direct actions, but of their indirect actions (such as the actions taken by

programs they run).

So we must prevent programs running at ‘Secret’ from writing to ﬁles at

‘Unclassiﬁed’, or more generally prevent any process at High from signalling

to any object (or subject) at Low. In general, when systems enforce a security

policy independently of user actions, they are described as having mandatory

access control, as opposed to the discretionary access control in systems like Unix

where users can take their own access decisions about their ﬁles.

The Bell-LaPadula model makes it relatively straightforward to verify claims

about the protection provided by a design. Given both the simple security

property (no read up), and the star property (no write down), various results

can be proved about the machine states which can be reached from a given

starting state, and this simpliﬁes formal analysis. There are some elaborations,

such as a trusted subject — a principal who is allowed to declassify ﬁles. To keep

things simple, we’ll ignore this; we’ll also ignore the possibility of incompatible

security levels for the time being, and return to them in the next chapter; and

ﬁnally, in order to simplify matters still further, we will assume from now

on that the system has only two levels, High and Low (unless there is some

particular reason to name individual compartments).

Multilevel security can be implemented in a number of ways. The original

idea was to implement a reference monitor by beeﬁng up the part of an

operating system which supervises all operating system calls and checks

access permissions to decide whether the call can be serviced or not. However

in practice things get much more complex as it’s often hard to build systems

whose trusted computing base is substantially less than the whole operating

system kernel (plus quite a number of its utilities).

Another approach that has been gaining ground as hardware has got cheaper

and faster is to replicate systems. This replication was often physical in the

1990s, and since about 2005 it may use virtual machines; some promising

recent work builds on virtualization products such as VMware and Xen to

provide multiple systems at different security levels on the same PC. One

might, for example, have one database running at Low and another at High,

on separate instances of Windows XP, with a pump that constantly copies

information from Low up to High, all running on VMware on top of SELinux.

I’ll discuss pumps in more detail later.

8.3.3 The Standard Criticisms of Bell-LaPadula

The introduction of BLP caused a lot of excitement: here was a straightforward

security policy which was clear to the intuitive understanding yet still allowed

people to prove theorems. But John McLean showed that the BLP rules

were not in themselves enough. He introduced System Z, deﬁned as a BLP

system with the added feature that a user can ask the system administrator to

8.3 The Bell-LaPadula Security Policy Model 247

temporarily declassify any ﬁle from High to Low. In this way, Low users can

read any High ﬁle without breaking the BLP assumptions.

Bell’s argument was that System Z cheats by doing something the model

doesn’t allow (changing labels isn’t a valid operation on the state), and

McLean’s argument was that it didn’t explicitly tell him so. The issue is dealt

with by introducing a tranquility property. The strong tranquility property says

that security labels never change during system operation, while the weak

tranquility property says that labels never change in such a way as to violate

a deﬁned security policy.

The motivation for the weak property is that in a real system we often

want to observe the principle of least privilege and start off a process at the

uncleared level, even if the owner of the process were cleared to ‘Top Secret’.

If she then accesses a conﬁdential email, her session is automatically upgraded

to ‘Conﬁdential’; and in general, her process is upgraded each time it accesses

data at a higher level (this is known as the high water mark principle). As

subjects are usually an abstraction of the memory management sub-system

and ﬁle handles, rather than processes, this means that state changes when

access rights change, rather than when data actually moves.

The practical implication is that a process acquires the security labels of all

the ﬁles it reads, and these become the default label set of every ﬁle that it writes.

So a process which has read ﬁles at ‘Secret’ and ‘Crypto’ will thereafter create

ﬁles marked (at least) ‘Secret Crypto’. This will include temporary copies made

of other ﬁles. If it then reads a ﬁle at ‘Top Secret Daffodil’ then all ﬁles it creates

after that will be labelled ‘Top Secret Crypto Daffodil’, and it will not be able to

write to any temporary ﬁles at ‘Secret Crypto’. The effect this has on applica-

tions is one of the serious complexities of multilevel security; most application

software needs to be rewritten (or at least modiﬁed) to run on MLS platforms.

Read-time changes in security level introduce the problem that access to

resources can be revoked at any time, including in the middle of a transaction.

Now the revocation problem is generally unsolvable in modern operating

systems, at least in any complete form, which means that the applications

have to cope somehow. Unless you invest some care and effort, you can easily

ﬁnd that everything ends up in the highest compartment — or that the system

fragments into thousands of tiny compartments that don’t communicate at all

with each other. I’ll discuss this in more detail in the next chapter.

Another problem with BLP, and indeed with all mandatory access control

systems, is that separating users and processes is relatively straightforward; the

hard part is when some controlled interaction is needed. Most real applications

need some kind of ‘trusted subject’ that can break the security policy; an

example is a trusted word processor that helps an intelligence analyst scrub a

Top Secret document when she’s editing it down to Secret [861]. BLP is silent

on how the system should protect such an application. Does it become part of

the Trusted Computing Base? I’ll discuss this in more detail below.

248 Chapter 8 ■Multilevel Security

Finally it’s worth noting that even with the high-water-mark reﬁnement,

BLP still doesn’t deal with the creation or destruction of subjects or objects

(which is one of the hard problems of building a real MLS system).

8.3.4 Alternative Formulations

Multilevel security properties have been expressed in several other ways.

The ﬁrst multilevel security policy was a version of high water mark

written in 1967–8 for the ADEPT-50, a mandatory access control system

developed for the IBM S/360 mainframe [1334]. This used triples of level,

compartment and group, with the groups being ﬁles, users, terminals and jobs.

As programs (rather than processes) were subjects, it was vulnerable to Trojan

horse compromises, and it was more complex than need be. Nonetheless, it

laid the foundation for BLP, and also led to the current IBM S/390 mainframe

hardware security architecture [632].

Shortly after that, a number of teams produced primitive versions of the

lattice model, which I’ll discuss in more detail in the next chapter. These also

made a signiﬁcant contribution to the Bell-LaPadula work, as did engineers

working on Multics. Multics had started as an MIT project in 1965 and

developed into a Honeywell product; it became the template for the ‘trusted

systems’ speciﬁed in the Orange Book, being the inspirational example of the

B2 level operating system. The evaluation that was carried out on it by Paul

Karger and Roger Schell was hugely inﬂuential and was the ﬁrst appearance of

the idea that malware could be hidden in the compiler [693] — which led to Ken

Thompson’s famous paper ‘On Trusting Trust’ ten years later. Multics itself

developed into a system called SCOMP that I’ll discuss in section 8.4.1 below.

Noninterference was introduced by Joseph Goguen and Jose Meseguer in

1982 [532]. In a system with this property, High’s actions have no effect on

what Low can see. Nondeducibility is less restrictive and was introduced by

David Sutherland in 1986 [1233]. Here the idea is to try and prove that Low

cannot deduce anything with 100 percent certainty about High’s input.

Low users can see High actions, just not understand them; a more formal

deﬁnition is that any legal string of high level inputs is compatible with every

string of low level events. So for every trace Low can see, there’s a similar

trace that didn’t involve High input. But different low-level event streams may

require changes to high-level outputs or reordering of high-level/low-level

event sequences.

The motive for nondeducibility is to ﬁnd a model that can deal with

applications such as a LAN on which there are machines at both Low and

High, with the High machines encrypting their LAN trafﬁc. (Quite a lot else is

needed to do this right, from padding the High trafﬁc with nulls so that Low

users can’t do trafﬁc analysis, and even ensuring that the packets are the same

size — see [1096] for an early example of such a system.)

8.3 The Bell-LaPadula Security Policy Model 249

Nondeducibility has historical importance since it was the ﬁrst nondeter-

ministic version of Goguen and Messeguer’s ideas. But it is hopelessly weak.

There’s nothing to stop Low making deductions about High input with 99%

certainty. There’s also a whole lot of problems when we are trying to prove

results about databases, and have to take into account any information which

can be inferred from data structures (such as from partial views of data with

redundancy) as well as considering the traces of executing programs. I’ll

discuss these problems further in the next chapter.

Improved models include Generalized Noninterference and restrictiveness. The

former is the requirement that if one alters a high level input event in a legal

sequence of system events, the resulting sequence can be made legal by, at

most, altering one or more subsequent high-level output events. The latter

adds a further restriction on the part of the trace where the alteration of the

high-level outputs can take place. This is needed for technical reasons to ensure

that two systems satisfying the restrictiveness property can be composed into

a third which also does. See [864] which explains these issues.

The Harrison-Ruzzo-Ullman model tackles the problem of how to deal with

the creation and deletion of ﬁles, an issue on which BLP is silent. It operates on

access matrices and veriﬁes whether there is a sequence of instructions which

causes an access right to leak to somewhere it was initially not present [584].

This is more expressive than BLP, but is more complex and thus less tractable

as an aid to veriﬁcation.

John Woodward proposed a Compartmented Mode Workstation (CMW) policy,

which attempted to model the classiﬁcation of information using ﬂoating

labels, as opposed to the ﬁxed labels associated with BLP [1357, 552]. It was

ultimately unsuccessful, because labels tend to either ﬂoat up too far too

fast (if done correctly), or they ﬂoat up more slowly (but don’t block all the

opportunities for malicious information ﬂow). However, CMW ideas have

led to real products — albeit products that provide separation more than

information sharing.

The type enforcement model, due to Earl Boebert and Dick Kain [198], assigns

subjects to domains and objects to types, with matrices deﬁning permitted

domain-domain and domain-type interactions. This is used in a popular and

important mandatory access control system, SELinux, which simpliﬁes it by

putting both subjects and objects in types and having a matrix of allowed

type pairs [813]. In effect this is a second access-control matrix; in addition

to having a user ID and group ID, each process has a security ID. The Linux

Security Modules framework provides pluggable security modules with rules

operating on SIDs.

Type enforcement was later extended by Badger and others to Domain and

Type Enforcement [106]. They introduced their own language for conﬁguration

(DTEL), and implicit typing of ﬁles based on pathname; for example, all

objects in a given subdirectory may be declared to be in a given domain. TE

250 Chapter 8 ■Multilevel Security

and DTE are more general than simple MLS policies such as BLP, as they

start to deal with integrity as well as conﬁdentiality concerns. One of their

early uses, starting in the LOCK system, was to enforce trusted pipelines: the

idea is to conﬁne a set of trusted processes in a pipeline so that each can

only talk to previous stage and the next stage. This can be used to assemble

guards and ﬁrewalls which cannot be bypassed unless at least two stages are

compromised [963]. Type-enforcement mechanisms are used, for example, in

the Sidewinder ﬁrewall. A further advantage of type enforcement mechanisms

is that they can be aware of code versus data, and privileges can be bound to

code; in consequence the tranquility problem can be dealt with at execute time

rather than as data are read. This can make things much more tractable.

The downside of the greater ﬂexibility and expressiveness of TE/DTE is

that it is not always straightforward to implement BLP, because of the state

explosion; when writing a security policy you have to consider all the possible

interactions between different types. (For this reason, SELinux also implements

a simple MLS policy. I’ll discuss SELinux in more detail below.)

Finally, a policy model getting much attention from researchers in recent

years is role-based access control (RBAC), introduced by David Ferraiolo and

Richard Kuhn [466, 467]. This provides a more general framework for manda-

tory access control than BLP in which access decisions don’t depend on users’

names but on the functions which they are currently performing within the

organization. Transactions which may be performed by holders of a given role

are speciﬁed, then mechanisms for granting membership of a role (including

delegation). Roles, or groups, had for years been the mechanism used in

practice in organizations such as banks to manage access control; the RBAC

model starts to formalize this. It can be used to give ﬁner-grained control,

for example by granting different access rights to ‘Ross as Professor’, ‘Ross as

member of the Planning and Resources Committee’ and ‘Ross reading private

email’. Implementations vary; the banking systems of twenty years ago kept

the controls in middleware, and some modern RBAC products do control

at the application layer where it’s easy to bypass. SELinux builds it on top of

TE, so that users are mapped to roles at login time, roles are authorized for

domains and domains are given permissions to types. On such a platform,

RBAC can usefully deal with integrity issues as well as conﬁdentiality, by

allowing role membership to be revised when certain programs are invoked.

Thus, for example, a process calling untrusted software that had been down-

loaded from the net might lose the role membership required to write to

sensitive system ﬁles.

8.3.5 The Biba Model and Vista

The incorporation into Windows Vista of a multilevel integrity model has

revived interest in a security model devised in 1975 by Ken Biba [168], which

8.3 The Bell-LaPadula Security Policy Model 251

textbooks often refer to as ‘Bell-LaPadula upside down’. The Biba model deals

with integrity alone and ignores conﬁdentiality. The key observation is that

conﬁdentiality and integrity are in some sense dual concepts — conﬁdentiality

is a constraint on who can read a message, while integrity is a constraint on

who can write or alter it.

As a concrete application, an electronic medical device such as an ECG

may have two separate modes: calibration and use. Calibration data must be

protected from corruption by normal users, who will therefore be able to read

it but not write to it; when a normal user resets the device, it will lose its

current user state (i.e., any patient data in memory) but the calibration will

remain unchanged.

To model such a system, we can use a multilevel integrity policy with the

rules that we can read data at higher levels (i.e., a user process can read

the calibration data) and write to lower levels (i.e., a calibration process can

write to a buffer in a user process); but we must never read down or write

up, as either could allow High integrity objects to become contaminated

with Low — that is potentially unreliable — data. The Biba model is often

formulated in terms of the low water mark principle, which is the dual of the high

water mark principle discussed above: the integrity of an object is the lowest

level of all the objects that contributed to its creation.

This was the ﬁrst formal model of integrity. A surprisingly large number of

real systems work along Biba lines. For example, the passenger information

system in a railroad may get information from the signalling system, but

certainly shouldn’t be able to affect it (other than through a trusted interface,

such as one of the control staff). However, few of the people who build such

systems are aware of the Biba model or what it might teach them.

Vista marks ﬁle objects with an integrity level, which can be Low, Medium,

High or System, and implements a default policy of NoWriteUp. Critical Vista

ﬁles are at System and other objects are at Medium by default — except for

Internet Explorer which is at Low. The effect is that things downloaded using

IE can read most ﬁles in a Vista system, but cannot write them. The idea is to

limit the damage that can be done by viruses and other malware. I’ll describe

Vista’s mechanisms in more detail below.

An interesting precursor to Vista was LOMAC, a Linux extension that

implemented a low water mark policy [494]. It provided two levels — high

and low integrity — with system ﬁles at High and the network at Low. As

soon as a program (such as a daemon) received trafﬁc from the network, it

was automatically downgraded to Low. Thus even if the trafﬁc contains an

attack that forks a root shell, this shell could not write to the password ﬁle as a

normal root shell would. As one might expect, a number of system tasks (such

as logging) became tricky and required trusted code.

As you might expect, Biba has the same fundamental problems as Bell-

LaPadula. It cannot accommodate real-world operation very well without

252 Chapter 8 ■Multilevel Security

numerous exceptions. For example, a real system will usually require ‘trusted’

subjects that can override the security model, but Biba on its own fails to

provide effective mechanisms to protect and conﬁne them; and in general it

doesn’t work so well with modern software environments. In the end, Vista

dropped the NoReadDown restriction and did not end up using its integrity

model to protect the base system from users.

Biba also cannot express many real integrity goals, like assured pipelines.

In fact, the Type Enforcement model was introduced by Boebert and Kain as

an alternative to Biba. It is unfortunate that Vista didn’t incorporate TE.

I will consider more complex models when I discuss banking and book-

keeping systems in Chapter 10; these are more complex in that they retain

security state in the form of dual control mechanisms, audit trails and so on.

8.4 Historical Examples of MLS Systems

Following some research products in the late 1970’s (such as KSOS [166],

a kernelised secure version of Unix), products that implemented multilevel

security policies started arriving in dribs and drabs in the early 1980’s. By

about 1988, a number of companies started implementing MLS versions of

their operating systems. MLS concepts were extended to all sorts of products.

8.4.1 SCOMP

One of the most important products was the secure communications proces-

sor (SCOMP), a derivative of Multics launched in 1983 [491]. This was a

no-expense-spared implementation of what the US Department of Defense

believed it wanted for handling messaging at multiple levels of classiﬁcation.

It had formally veriﬁed hardware and software, with a minimal kernel and

four rings of protection (rather than Multics’ seven) to keep things simple. Its

operating system, STOP, used these rings to maintain up to 32 separate com-

partments, and to allow appropriate one-way information ﬂows between them.

SCOMP was used in applications such as military mail guards. These are

specialised ﬁrewalls which typically allow mail to pass from Low to High

but not vice versa [369]. (In general, a device which does this is known as

adata diode.) SCOMP’s successor, XTS-300, supported C2G, the Command

and Control Guard. This was used in the time phased force deployment

data (TPFDD) system whose function was to plan US troop movements

and associated logistics. Military plans are developed as TPFDDs at a high

classiﬁcation level, and then distributed at the appropriate times as commands

to lower levels for implementation. (The issue of how high information is

deliberately downgraded raises a number of issues, some of which I’ll deal

8.4 Historical Examples of MLS Systems 253

with below. In the case of TPFDD, the guard examines the content of each

record before deciding whether to release it.)

SCOMP’s most signiﬁcant contribution was to serve as a model for the

Orange Book [375] — the US Trusted Computer Systems Evaluation Criteria.

This was the ﬁrst systematic set of standards for secure computer systems,

being introduced in 1985 and ﬁnally retired in December 2000. The Orange

Book was enormously inﬂuential not just in the USA but among allied powers;

countries such as the UK, Germany, and Canada based their own national

standards on it, until these national standards were ﬁnally subsumed into the

Common Criteria [935].

The Orange Book allowed systems to be evaluated at a number of levels

with A1 being the highest, and moving downwards through B3, B2, B1 and

C2 to C1. SCOMP was the ﬁrst system to be rated A1. It was also extensively

documented in the open literature. Being ﬁrst, and being fairly public, it set

the standard for the next generation of military systems. This standard has

rarely been met since; in fact, the XTS-300 was only evaluated to B3 (the formal

proofs of correctness required for an A1 evaluation were dropped).

8.4.2 Blacker

Blacker was a series of encryption devices designed to incorporate MLS

technology. Previously, encryption devices were built with separate processors

for the ciphertext, or Black, end and the cleartext, or Red, end. Various possible

failures can be prevented if one can coordinate the Red and Black processing.

One can also make the device simpler, and provide greater operational

ﬂexibility: the device isn’t limited to separating two logical networks, but can

provide encryption and integrity assurance selectively, and interact in useful

ways with routers. But then a high level of assurance is required that the ‘Red’

data won’t leak out via the ‘Black’.

Blacker entered service in 1989, and the main lesson learned from it was

the extreme difﬁculty of accommodating administrative trafﬁc within a model

of classiﬁcation levels [1335]. As late as 1994, it was the only communications

security device with an A1 evaluation [161]. So it too had an effect on later

systems. It was not widely used though, and its successor (the Motorola

Network Encryption System), had only a B2 evaluation.

8.4.3 MLS Unix and Compartmented Mode Workstations

MLS versions of Unix started to appear in the late 1980’s, such as AT&T’s Sys-

tem V/MLS [27]. This added security levels and labels, initially by using some

of the bits in the group id record and later by using this to point to a more elab-

orate structure. This enabled MLS properties to be introduced with minimal

changes to the system kernel. Other products of this kind included SecureWare

254 Chapter 8 ■Multilevel Security

(and its derivatives, such as SCO and HP VirtualVault), and Addamax. By the

time of writing (2007), Sun’s Solaris has emerged as the clear market leader,

being the platform of choice for high-assurance server systems and for many

clients as well. Trusted Solaris 8 gave way to Solaris trusted Extensions 10,

which has now been folded into Solaris, so that every copy of Solaris contains

MLS mechanisms, for those knowledgeable enough to use them.

Comparted Mode Workstations (CMWs) are an example of MLS clients. They

allow data at different levels to be viewed and modiﬁed at the same time

by a human operator, and ensure that labels attached to the information

are updated appropriately. The initial demand came from the intelligence

community, whose analysts may have access to ‘Top Secret’ data, such as

decrypts and agent reports, and produce reports at the ‘Secret’ level for users

such as political leaders and ofﬁcers in the ﬁeld. As these reports are vulnerable

to capture, they must not contain any information which would compromise

intelligence sources and methods.

CMWs allow an analyst to view the ‘Top Secret’ data in one window,

compose a report in another, and have mechanisms to prevent the accidental

copying of the former into the latter (i.e., cut-and-paste works from ‘Secret’ to

‘Top Secret’ but not vice versa). CMWs have proved useful in operations, logis-

tics and drug enforcement as well [631]. For the engineering issues involved in

doing mandatory access control in windowing systems, see [437, 438] which

describe a prototype for Trusted X, a system implementing MLS but not infor-

mation labelling. It runs one instance of X Windows per sensitivity level, and

has a small amount of trusted code which allows users to cut and paste from

a lower level to a higher one. For the speciﬁc architectural issues with Sun’s

CMW product, see [451].

8.4.4 The NRL Pump

It was soon realised that simple mail guards and crypto boxes were too

restrictive, as many more networked services were developed besides mail.

Traditional MLS mechanisms (such as blind write-ups and periodic read-

downs) are inefﬁcient for real-time services.

The US Naval Research Laboratory (NRL) therefore developed the Pump —a

one-way data transfer device (a data diode) to allow secure one-way infor-

mation ﬂow (Figure 8.3). The main problem is that while sending data from

Low to High is easy, the need for assured transmission reliability means

that acknowledgement messages must be sent back from High to Low. The

Pump limits the bandwidth of possible backward leakage using a number of

mechanisms such as using buffering and randomizing the timing of acknowl-

edgements [685, 687, 688]. The attraction of this approach is that one can build

MLS systems by using pumps to connect separate systems at different security

levels. As these systems don’t process data at more than one level, they can be

8.4 Historical Examples of MLS Systems 255

HIGH

PUMP

LOW

Figure 8.3: The NRL pump

built from cheap commercial-off-the-shelf (COTS) components [689]. As the

cost of hardware falls, this is often the preferred option where it’s possible.

The pump’s story is told in [691].

The Australian government developed a product called Starlight that uses

pump-type technology married with a keyboard switch to provide a nice

MLS-type windowing system (albeit without any visible labels) using a bit

of trusted hardware which connects the keyboard and mouse with High and

Low systems [30]. There is no trusted software. It’s been integrated with the

NRL Pump [689]. A number of semi-commercial data diode products have

also been introduced.

8.4.5 Logistics Systems

Military stores, like government documents, can have different classiﬁcation

levels. Some signals intelligence equipment is ‘Top Secret’, while things

like jet fuel and bootlaces are not; but even such simple commodities may

become ‘Secret’ when their quantities or movements might leak information

about tactical intentions. There are also some peculiarities: for example, an

inertial navigation system classiﬁed ‘Conﬁdential’ in the peacetime inventory

might contain a laser gyro platform classiﬁed ‘Secret’ (thus security levels are

nonmonotonic).

The systems needed to manage all this seem to be hard to build, as MLS

logistics projects in both the USA and UK have ended up as expensive

disasters. In the UK, the Royal Air Force’s Logistics Information Technology

System (LITS) was a 10 year (1989–99), £500m project to provide a single

stores management system for the RAF’s 80 bases [932]. It was designed to

operate on two levels: ‘Restricted’ for the jet fuel and boot polish, and ‘Secret’

for special stores such as nuclear bombs. It was initially implemented as two

256 Chapter 8 ■Multilevel Security

separate database systems connected by a pump to enforce the MLS property.

The project became a classic tale of escalating costs driven by creeping

requirements changes. One of these changes was the easing of classiﬁcation

rules with the end of the Cold War. As a result, it was found that almost

all the ‘Secret’ information was now static (e.g., operating manuals for air-

drop nuclear bombs which are now kept in strategic stockpiles rather than at

airbases). In order to save money, the ‘Secret’ information is now kept on a CD

and locked up in a safe.

Logistics systems often have application security features too. The clas-

sic example is that ordnance control systems alert users who are about to

breach safety rules by putting explosives and detonators in the same truck or

magazine [910].

8.4.6 Sybard Suite

Most governments’ information security agencies have been unable to resist

user demands to run standard applications (such as MS Ofﬁce) which are

not available for multilevel secure platforms. One response was the ‘Purple

Penelope’ software, from Qinetiq in the UK, now sold as Sybard Suite. This

puts an MLS wrapper round a Windows workstation, implementing the high

water mark version of BLP. It displays in the background the current security

level of the device and upgrades it when necessary as more sensitive resources

are read. It ensures that the resulting work product is labelled correctly.

Rather than preventing users from downgrading, as a classical BLP system

might do, it allows them to assign any security label they like to their output.

However, if this involves a downgrade, the user must conﬁrm the release of

the data using a trusted path interface, thus ensuring no Trojan Horse or virus

can release anything completely unnoticed. Of course, a really clever malicious

program can piggy-back classiﬁed material on stuff that the user does wish to

release, so there are other tricks to make that harder. There is also an audit

trail to provide a record of all downgrades, so that errors and attacks (whether

by users, or by malware) can be traced after the fact [1032]. The security policy

was described to me by one of its authors as ‘we accept that we can’t stop

people leaking the order of battle to the Guardian newspaper if they really

want to; we just want to make sure we arrest the right person for it.’

8.4.7 Wiretap Systems

One of the large applications of MLS is in wiretapping systems. Commu-

nications intelligence is generally fragile; once a target knows his trafﬁc is

being read he can usually do something to frustrate it. Traditional wiretap

kit, based on ‘loop extenders’ spliced into the line, could often be detected by

competent targets; modern digital systems try to avoid these problems, and

8.5 Future MLS Systems 257

provide a multilevel model in which multiple agencies at different levels can

monitor a target, and each other; the police might be tapping a drug dealer,

and an anti-corruption unit watching the police, and so on. Wiretaps are

commonly implemented as conference calls with a silent third party, and the

main protection goal is to eliminate any covert channels that might disclose

the existence of surveillance. This is not always met. For a survey, see [1161],

which also points out that the pure MLS security policy is insufﬁcient: suspects

can confuse wiretapping equipment by introducing bogus signalling tones.

The policy should thus have included resistance against online tampering.

Another secondary protection goal should have been to protect against

software tampering. In a recent notorious case, a wiretap was discovered on

the mobile phones of the Greek Prime Minister and his senior colleagues; this

involved unauthorised software in the mobile phone company’s switchgear

that abused the lawful intercept facility. It was detected when the buggers’

modiﬁcations caused some text messages not to be delivered [1042]. The

phone company was ﬁned 76 million Euros (almost $100m). Perhaps phone

companies will be less willing to report unauthorized wiretaps in future.

8.5 Future MLS Systems

In the ﬁrst edition of this book, I wrote that the MLS industry’s attempts to

market its products as platforms for ﬁrewalls, web servers and other exposed

systems were failing because ‘the BLP controls do not provide enough of

a protection beneﬁt in many commercial environments to justify their large

development costs, and widely ﬁelded products are often better because of

the evolution that results from large-scale user feedback’. I also noted research

on using mandatory access controls to accommodate both conﬁdentiality and

integrity in environments such as smartcards [692], and to provide real-time

performance guarantees to prevent service denial attacks [889]. I ventured that

‘perhaps the real future of multilevel systems is not in conﬁdentiality, but

integrity’.

The last seven years appear to have proved this right.

8.5.1 Vista

Multilevel integrity is coming to the mass market in Vista. As I already

mentioned, Vista essentially uses the Biba model. All processes do, and all

securable objects (including directories, ﬁles and registry keys) may, have an

integrity-level label. File objects are labelled at ‘Medium’ by default, while

Internet Explorer (and everything downloaded using it) is labelled ‘Low’. User

action is therefore needed to upgrade downloaded content before it can modify

258 Chapter 8 ■Multilevel Security

existing ﬁles. This may not be a panacea: it may become so routine a demand

from all installed software that users will be trained to meekly upgrade viruses

too on request. And it must be borne in mind that much of the spyware infesting

the average home PC was installed there deliberately (albeit carelessly and with

incomplete knowledge of the consequences) after visiting some commercial

website. This overlap between desired and undesired software sets a limit on

how much can be achieved against downloaded malware. We will have to

wait and see.

It is also possible to implement a crude BLP policy using Vista, as you can

also set ‘NoReadUp’ and ‘NoExecuteUp’ policies. These are not installed as

default; the reason appears to be that Microsoft was principally concerned

about malware installing itself in the system and then hiding. Keeping the

browser ‘Low’ makes installation harder, and allowing all processes (even

Low ones) to inspect the rest of the system makes hiding harder. But it does

mean that malware running at Low can steal all your data; so some users

might care to set ‘NoReadUp’ for sensitive directories. No doubt this will

break a number of applications, so a cautious user might care to have separate

accounts for web browsing, email and sensitive projects. This is all discussed

by Joanna Rutkowska in [1099]; she also describes some interesting potential

attacks based on virtualization. A further problem is that Vista, in protected

mode, does still write to high-integrity parts of the registry, even though

Microsoft says it shouldn’t [555].

In passing, it’s also worth mentioning rights management, whether of the

classical DRM kind or the more recent IRM (Information Rights Management)

variety, as a case of mandatory access control. Vista, for example, tries to

ensure that no high deﬁnition video content is ever available to an untrusted

process. I will discuss it in more detail later, but for now I’ll just remark that

many of the things that go wrong with multilevel systems might also become

vulnerabilities in, or impediments to the use of, rights-management systems.

Conversely, the efforts expended by opponents of rights management in trying

to hack the Vista DRM mechanisms may also open up holes in its integrity

protection.

8.5.2 Linux

The case of SELinux and Red Hat is somewhat similar to Vista in that the

immediate goal of the new mandatory access control mechanisms is also

to limit the effects of a compromise. SELinux [813] is based on the Flask

security architecture [1209], which separates the policy from the enforcement

mechanism; a security context contains all of the security attributes associated

with a subject or object in Flask, where one of those attributes includes

the Type Enforcement type attribute. A security identiﬁer is a handle to a

security context, mapped by the security server. It has a security server where

8.5 Future MLS Systems 259

policy decisions are made, this resides in-kernel since Linux has a monolithic

kernel and the designers did not want to require a kernel-userspace call

for security decisions (especially as some occur on critical paths where the

kernel is holding locks) [557]). The server provides a general security API to

the rest of the kernel, with the security model hidden behind that API. The

server internally implements RBAC, TE, and MLS (or to be precise, a general

constraints engine that can express MLS or any other model you like). SELinux

is included in a number of Linux distributions, and Red Hat’s use is typical.

There its function is to separate various services. Thus an attacker who takes

over your web server does not thereby acquire your DNS server as well.

Suse Linux has taken a different path to the same goal. It uses AppArmor,

a monitoring mechanism maintained by Novell, which keeps a list of all the

paths each protected application uses and prevents it accessing any new ones.

It is claimed to be easier to use than the SELinux model; but operating-system

experts distrust it as it relies on pathnames as the basis for its decisions. In

consequence, it has ambiguous and mutable identiﬁers; no system view of

subjects and objects; no uniform abstraction for handling non-ﬁle objects; and

no useful information for runtime ﬁles (such as /tmp). By forcing policy to

be written in terms of individual objects and ﬁlesystem layout rather than

security equivalence classes, it makes policy harder to analyze. However, in

practice, with either AppArmor or SELinux, you instrument the code you plan

to protect, watch for some months what it does, and work out a policy that

allows it to do just what it needs. Even so, after you have ﬁelded it, you will

still have to observe and act on bug reports for a year or so. Modern software

components tend to be so complex that ﬁguring out what access they need is

an empirical and iterative process2.

It’s also worth bearing in mind that simple integrity controls merely stop

malware taking over the machine — they don’t stop it infecting a Low com-

partment and using that as a springboard from which to spread elsewhere.

Integrity protection is not the only use of SELinux. At present there is

considerable excitement about it in some sections of government, who are

excited at the prospect of ‘cross department access to data ... and a common

trust infrastructure for shared services’ (UK Cabinet Ofﬁce) and allowing ‘users

to access multiple independent sessions at varying classiﬁcation levels’ (US

Coast Guard) [505]. Replacing multiple terminals with single ones, and moving

from proprietary systems to open ones, is attractive for many reasons — and

providing simple separation between multiple terminal emulators or browsers

running on the same PC is straightforward. However, traditional MAC might

not be the only way to do it.

2Indeed, some of the mandatory access control mechanisms promised in Vista — such as remote

attestation — did not ship in the ﬁrst version, and there have been many papers from folks

at Microsoft Research on ways of managing access control and security policy in complex

middleware. Draw your own conclusions.

260 Chapter 8 ■Multilevel Security

8.5.3 Virtualization

Another technological approach is virtualization. Products such as VMware

and Xen are being used to provide multiple virtual machines at different

levels. Indeed, the NSA has produced a hardened version of VMware, called

NetTop, which is optimised for running several Windows virtual machines

on top of an SELinux platform. This holds out the prospect of giving the

users what they want — computers that have the look and feel of ordinary

windows boxes — while simultaneously giving the security folks what they

want, namely high-assurance separation between material at different levels

of classiﬁcation. So far, there is little information available on NetTop, but it

appears to do separation rather than sharing.

A current limit is the sheer technical complexity of modern PCs; it’s very

difﬁcult to ﬁnd out what things like graphics cards actually do, and thus to

get high assurance that they don’t contain huge covert channels. It can also

be quite difﬁcult to ensure that a device such as a microphone or camera is

really connected to the Secret virtual machine rather than the Unclassiﬁed

one. However, given the effort being put by Microsoft into assurance for

high-deﬁnition video content, there’s at least the prospect that some COTS

machines might eventually offer reasonable assurance on I/O eventually.

The next question must be whether mandatory access control for conﬁden-

tiality, as opposed to integrity, will make its way out of the government sector

and into the corporate world. The simplest application might be for a company

to provide its employees with separate virtual laptops for corporate and home

use (whether with virtualisation or with mandatory access controls). From the

engineering viewpoint, virtualization might preferable, as it’s not clear that

corporate security managers will want much information ﬂow between the

two virtual laptops: ﬂows from ‘home’ to ‘work’ could introduce malware

while ﬂow from ‘work’ to ‘home’ could leak corporate secrets. From the busi-

ness viewpoint, it’s less clear that virtualization will take off. Many corporates

would rather pretend that company laptops don’t get used for anything else,

and as the software industry generally charges per virtual machine rather than

per machine, there could be nontrivial costs involved. I expect most companies

will continue to ignore the problem and just ﬁre people whose machines cause

visible trouble.

The hardest problem is often managing the interfaces between levels, as

people usually end up having to get material from one level to another in

order to get their work done. If the information ﬂows are limited and easy

to model, as with the Pump and the CMW, well and good; the way forward

may well be Pump or CMW functionality also hosted on virtual machines.

(Virtualisation per se doesn’t give you CMW — you need a trusted client for

that, or an app running on a trusted server — but it’s possible to envisage a

trusted thin client plus VMs at two different levels all running on the same box.

8.6 What Goes Wrong 261

So virtualization should probably be seen as complementary to mandatory

access control, rather than a competitor.

But many things can go wrong, as I will discuss in the next session.

8.5.4 Embedded Systems

There are more and more ﬁelded systems which implement some variant of the

Biba model. As well as the medical-device and railroad signalling applications

already mentioned, there are utilities. In an electricity utility, for example,

operational systems such as power dispatching should not be affected by any

others. The metering systems can be observed by, but not inﬂuenced by, the

billing system. Both billing and power dispatching feed information into fraud

detection, and at the end of the chain the executive information systems can

observe everything while having no direct effect on operations. These one-way

information ﬂows can be implemented using mandatory access controls and

there are signs that, given growing concerns about the vulnerability of critical

infrastructure, some utilities are starting to look at SELinux.

There are many military embedded systems too. The primitive mail guards

of 20 years ago have by now been supplanted by guards that pass not just

email but chat, web services and streaming media, often based on SELinux;

an example is described in [478]. There are many more esoteric applications:

for example, some US radars won’t display the velocity of a US aircraft whose

performance is classiﬁed, unless the operator has the appropriate clearance.

(This has always struck me as overkill, as he can just use a stopwatch.)

Anyway, it’s now clear that many of the lessons learned in the early

multilevel systems go across to a number of applications of much wider

interest. So do a number of the failure modes, which I’ll now discuss.

8.6 What Goes Wrong

As I’ve frequently pointed out, engineers learn more from the systems that fail

than from those that succeed, and MLS systems have certainly been an effective

teacher. The large effort expended in building systems to follow a simple policy

with a high level of assurance has led to the elucidation of many second- and

third-order consequences of information ﬂow controls. I’ll start with the more

theoretical and work through to the business and engineering end.

8.6.1 Composability

Consider a simple device that accepts two ‘High’ inputs H1and H2; mul-

tiplexes them; encrypts them by xor’ing them with a one-time pad (i.e., a

random generator); outputs the other copy of the pad on H3; and outputs the

262 Chapter 8 ■Multilevel Security

H1

H2

XOR

RAND H3

XOR

•



L

Figure 8.4: Insecure composition of secure systems with feedback

ciphertext, which being encrypted with a cipher system giving perfect secrecy,

is considered to be low (output L), as in Figure 8.4.

In isolation, this device is provably secure. However, if feedback is permitted,

then the output from H3can be fed back into H2, with the result that the high

input H1now appears at the low output L. Timing inconsistencies can also

lead to the composition of two secure systems being insecure (see for example

McCullough [854]). Simple information ﬂow doesn’t compose; neither does

noninterference or nondeducibility.

In general, the problem of how to compose two or more secure components

into a secure system is hard, even at the relatively uncluttered level of proving

results about ideal components. Most of the low-level problems arise when

some sort of feedback is introduced into the system; without it, composition

can be achieved under a number of formal models [865]. However, in real

life, feedback is pervasive, and composition of security properties can be

made even harder by detailed interface issues, feature interactions and so on.

For example, one system might produce data at such a rate as to perform a

service-denial attack on another. (I’ll discuss some of the interface problems

with reference monitors in detail in Chapter 18, ‘API Attacks’.)

Finally, the composition of secure components or systems is very often

frustrated by higher-level incompatibilities. Components might have been

designed in accordance with two different security policies, or designed

according to requirements that are inconsistent or even incompatible. This is

bad enough for different variants on the BLP theme but even worse when one

of the policies is of a non-BLP type, as we will encounter in the following two

chapters. Composability is a long-standing and very serious problem with

trustworthy systems; a good recent survey is the ﬁnal report of the CHATS

project [963].

8.6.2 The Cascade Problem

An example of the difﬁculty of composing multilevel secure systems is given

by the cascade problem (Figure 8.5). After the Orange book introduced a series

8.6 What Goes Wrong 263

Top Secret

Secret Secret

Unclassified

Figure 8.5: The cascade problem

of graduated evaluation levels, this led to rules about the number of levels

which a system can span [379]. For example, a system evaluated to B3 was

in general allowed to process information at Unclassiﬁed, Conﬁdential and

Secret, or at Conﬁdential, Secret and Top Secret; there was no system permitted

to process Unclassiﬁed and Top Secret data simultaneously [379].

As the diagram shows, it is straightforward to connect together two B3

systems in such a way that this security policy is broken. The ﬁrst system

connects together Unclassiﬁed, Conﬁdential and Secret, and its Conﬁdential

and Secret levels communicate with the second system which also processes

Top Secret information. (The problem’s discussed in more detail in [622].)

This illustrates another kind of danger which formal models of security (and

practical implementations) must take into account.

8.6.3 Covert Channels

One of the reasons why these span limits are imposed on multilevel systems

emerges from a famous — and extensively studied — problem: the covert chan-

nel. First pointed out by Lampson in 1973 [768], a covert channel is a mechanism

that was not designed for communication but which can nonetheless be abused

to allow information to be communicated down from High to Low.

264 Chapter 8 ■Multilevel Security

A typical covert channel arises when a high process can signal to a low

one by affecting some shared resource. For example, it could position the

disk head at the outside of the drive at time tito signal that the i-th bit in

a Top Secret ﬁle was a 1, and position it at the inside to signal that the bit

was a 0.

All systems with shared resources must ﬁnd a balance between covert

channel capacity, resource utilization, and fairness. If a machine is shared

between high and low, and resources are not allocated in ﬁxed slices, then the

high process can signal by ﬁlling up the disk drive, or by using a lot of CPU or

bus cycles (some people call the former case a storage channel and the latter a

timing channel, though in practice they can often be converted into each other).

There are many others such as sequential process IDs, shared ﬁle locks and last

access times on ﬁles — reimplementing all of these in a multilevel secure way

is an enormous task. Various strategies have been adopted to minimize their

bandwidth; for example, we can arrange that the scheduler assigns a ﬁxed

disk quota to each level, and reads the boot sector each time control is passed

downwards; and we might also allocate a ﬁxed proportion of the available time

slices to processes at each level, and change these proportions infrequently.

Each change might allow one or more bits to be signalled, but such strategies

can enormously reduce the available bandwidth. (A more complex multilevel

design, which uses local schedulers at each level plus a global scheduler to

maintain overall consistency, is described in [686].)

It is also possible to limit the covert channel capacity by introducing noise.

Some machines have had randomised system clocks for this purpose. But

some covert channel capacity almost always remains. (Techniques to analyze

the trade-offs between covert channel capacity and system performance are

discussed in [554].)

Many covert channels occur at the application layer, and are a real concern

to security engineers (especially as they are often overlooked). An example

from social care is a UK proposal to create a national database of all children,

for child-protection and welfare purposes, containing a list of all professionals

with which each child has contact. Now it may be innocuous that child X

is registered with family doctor Y, but the fact of a child’s registration with

a social work department is not innocuous at all — it’s well known to be

stigmatizing. For example, teachers will have lower expectations of children

whom they know to have been in contact with social workers. So it is quite

reasonable for parents (and children) to want to keep any record of such

contact private [66].

A more subtle example is that in general personal health information derived

from visits to genitourinary medicine clinics is High in the sense that it can’t

be shared with the patient’s normal doctor and thus appear in their normal

medical record (Low) unless the patient consents. In one case, a woman’s visit

to a GUM clinic leaked when the insurer failed to recall her for a smear test

8.6 What Goes Wrong 265

which her normal doctor knew was due [886]. The insurer knew that a smear

test had been done already by the clinic, and didn’t want to pay twice.

Another case of general interest arises in multilevel integrity systems such

as banking and utility billing, where a programmer who has inserted Trojan

code in a bookkeeping system can turn off the billing to an account by a

certain pattern of behavior (in a phone system he might call three numbers in

succession, for example). Code review is the only real way to block such attacks,

though balancing controls can also help in the speciﬁc case of bookkeeping.

The highest-bandwidth covert channel of which I’m aware is also a feature

of a speciﬁc application. It occurs in large early warning radar systems,

where High — the radar processor — controls hundreds of antenna elements

that illuminate Low — the target — with high speed pulse trains that are

modulated with pseudorandom noise to make jamming harder. In this case,

the radar code must be trusted as the covert channel bandwidth is many

megabits per second.

The best that developers have been able to do consistently with BLP

conﬁdentiality protection in regular time-sharing operating systems is to limit

it to 1 bit per second or so. (That is a DoD target [376], and techniques for doing

a systematic analysis may be found in Kemmerer [706].) One bit per second

may be tolerable in an environment where we wish to prevent large TS/SCI

ﬁles — such as satellite photographs — leaking down from TS/SCI users to

‘Secret’ users. It is much less than the rate at which malicious code might hide

data in outgoing trafﬁc that would be approved by a guard. However, it is

inadequate if we want to prevent the leakage of a cryptographic key. This is

one of the reasons for the military doctrine of doing crypto in special purpose

hardware rather than in software.

8.6.4 The Threat from Viruses

The vast majority of viruses are found on mass-market products such as

PCs. However, the defense computer community was shocked when Cohen

used viruses to penetrate multilevel secure systems easily in 1983. In his ﬁrst

experiment, a ﬁle virus which took only eight hours to write managed to

penetrate a system previously believed to be multilevel secure [311].

There are a number of ways in which viruses and other malicious code

can be used to perform such attacks. If the reference monitor (or other TCB

components) can be corrupted, then a virus could deliver the entire system to

the attacker, for example by issuing him with an unauthorised clearance. For

this reason, slightly looser rules apply to so-called closed security environments

which are deﬁned to be those where ‘system applications are adequately

protected against the insertion of malicious logic’ [379]. But even if the TCB

remains intact, the virus could still use any available covert channel to signal

information down.

266 Chapter 8 ■Multilevel Security

So in many cases a TCB will provide some protection against viral attacks,

as well as against careless disclosure by users or application software — which

is often more important than malicious disclosure. However, the main effect

of viruses on military doctrine has been to strengthen the perceived case for

multilevel security. The argument goes that even if personnel can be trusted,

one cannot rely on technical measures short of total isolation to prevent viruses

moving up the system, so one must do whatever reasonably possible to stop

them signalling back down.

8.6.5 Polyinstantiation

Another problem that has much exercised the research community is polyin-

stantiation. Suppose that our High user has created a ﬁle named agents,and

our Low user now tries to do the same. If the MLS operating system prohibits

him, it will have leaked information — namely that there is a ﬁle called agents

at High. But if it lets him, it will now have two ﬁles with the same name.

Often we can solve the problem by a naming convention, which could be

as simple as giving Low and High users different directories. But the problem

remains a hard one for databases [1112]. Suppose that a High user allocates

a classiﬁed cargo to a ship. The system will not divulge this information to a

Low user, who might think the ship is empty, and try to allocate it another

cargo or even to change its destination.

The solution favoured in the USA for such systems is that the High user

allocates a Low cover story at the same time as the real High cargo. Thus the

underlying data will look something like Figure 8.6.

In the UK, the theory is simpler — the system will automatically reply

‘classiﬁed’ to a Low user who tries to see or alter a High record. The two

available views would be as in Figure 8.7.

Level Cargo Destination

Secret Missiles Iran

Restricted — —

Unclassiﬁed Engine spares Cyprus

Figure 8.6: How the USA deals with classiﬁed data

Level Cargo Destination

Secret Missiles Iran

Restricted Classiﬁed Classiﬁed

Unclassiﬁed — —

Figure 8.7: How the UK deals with classiﬁed data

8.6 What Goes Wrong 267

This makes the system engineering simpler. It also prevents the mistakes

and covert channels which can still arise with cover stories (e.g., a Low user

tries to add a container of ammunition for Cyprus). The drawback is that

everyone tends to need the highest available clearance in order to get their

work done. (In practice, of course, cover stories still get used in order not to

advertise the existence of a covert mission any more than need be.)

There may be an interesting new application to the world of online gaming.

Different countries have different rules about online content; for example,

the USA limits online gambling, while Germany has strict prohibitions on the

display of swastikas and other insignia of the Third Reich. Now suppose a

second-world-war reenactment society wants to operate in Second Life. If

a German resident sees ﬂags with swastikas, an offence is committed there.

Linden Labs, the operator of Second Life, has suggested authenticating users’

jurisdictions; but it’s not enough just to exclude Germans, as one of them

might look over the fence. An alternative proposal is to tag alternative objects

for visibility, so that a German looking at the Battle of Kursk would see only

inoffensive symbols. Similarly, an American looking at an online casino might

just see a church instead. Here too the lie has its limits; when the American

tries to visit that church he’ll ﬁnd that he can’t get through the door.

8.6.6 Other Practical Problems

Multilevel secure systems are surprisingly expensive and difﬁcult to build and

deploy. There are many sources of cost and confusion.

1. MLS systems are built in small volumes, and often to high standards of

physical robustness, using elaborate documentation, testing and other

quality control measures driven by military purchasing bureaucracies.

2. MLS systems have idiosyncratic administration tools and procedures. A

trained Unix administrator can’t just take on an MLS installation without

signiﬁcant further training. A USAF survey showed that many MLS sys-

tems were installed without their features being used [1044].

3. Many applications need to be rewritten or at least greatly modiﬁed to run

under MLS operating systems [1092]. For example, compartmented mode

workstations that display information at different levels in different win-

dows, and prevent the user from doing cut-and-paste operations from

high to low, often have problems with code which tries to manipulate the

colour map. Access to ﬁles might be quite different, as well as the format

of things like access control lists. Another source of conﬂict with commer-

cial software is the licence server; if a High user invokes an application,

which goes to a licence server for permission to execute, then an MLS

operating system will promptly reclassify the server High and deny access

to Low users. So in practice, you usually end up (a) running two separate

268 Chapter 8 ■Multilevel Security

license servers, thus violating the license terms, or (b) you have an MLS

license server which tracks licenses at all levels (this restricts your choice

of platforms), or (c) you only access the licensed software at one of the

levels.

4. Because processes are automatically upgraded as they see new labels, the

ﬁles they use have to be too. New ﬁles default to the highest label belong-

ing to any possible input. The result of all this is a chronic tendency for

things to be overclassiﬁed.

5. It is often inconvenient to deal with ‘blind write-up’ — when a low level

application sends data to a higher level one, BLP prevents any acknowl-

edgment being sent. The effect is that information vanishes into a ‘black

hole’. The answer to this is varied. Some organizations accept the problem

as a fact of life; in the words of a former NSA chief scientist ‘When you

pray to God, you do not expect an individual acknowledgement of each

prayer before saying the next one’. Others use pumps rather than prayer,

and accept a residual covert bandwidth as a fact of life.

6. The classiﬁcation of data can get complex:

in the run-up to a military operation, the location of ‘innocuous’ stores

such as food could reveal tactical intentions, and so may be suddenly

upgraded. It follows that the tranquility property cannot simply be

assumed;

classiﬁcations are not necessarily monotone. Equipment classiﬁed at

‘conﬁdential’ in the peacetime inventory may easily contain compo-

nents classiﬁed ‘secret’;

information may need to be downgraded. An intelligence analyst might

need to take a satellite photo classiﬁed at TS/SCI, and paste it into an

assessment for ﬁeld commanders at ‘secret’. However, information

could have been covertly hidden in the image by a virus, and retrieved

later once the ﬁle is downgraded. So downgrading procedures may

involve all sorts of special ﬁlters, such as lossy compression of images

and word processors which scrub and reformat text, in the hope that

the only information remaining is that which lies in plain sight. (I will

discuss information hiding in more detail in the context of copyright

marking.)

we may need to worry about the volume of information available to

an attacker. For example, we might be happy to declassify any single

satellite photo, but declassifying the whole collection would reveal

our surveillance capability and the history of our intelligence priori-

ties. Similarly, the government payroll may not be very sensitive per

se, but it is well known that journalists can often identify intelligence

personnel working under civilian cover from studying the evolution of

8.7 Broader Implications of MLS 269

departmental staff lists over a period of a few years. (I will look at this

issue — the ‘aggregation problem’ — in more detail in section 9.3.2.)

a related problem is that the output of an unclassiﬁed program acting

on unclassiﬁed data may be classiﬁed. This is also related to the aggre-

gation problem.

7. There are always system components — such as memory management —

that must be able to read and write at all levels. This objection is dealt

with by abstracting it away, and assuming that memory management is

part of the trusted computing base which enforces our mandatory access

control policy. The practical outcome is that often a quite uncomfortably

large part of the operating system (plus utilities, plus windowing system

software, plus middleware such as database software) ends up part of the

trusted computing base. ‘TCB bloat’ constantly pushes up the cost of eval-

uation and reduces assurance.

8. Finally, although MLS systems can prevent undesired things (such as

information leakage) from happening, they also prevent desired things

from happening too (such as efﬁcient ways of enabling data to be down-

graded from High to Low, which are essential if many systems are to be

useful). So even in military environments, the beneﬁts they provide can

be very questionable. The associated doctrine also sets all sorts of traps for

government systems builders. A recent example comes from the debate

over a UK law to extend wiretaps to Internet Service Providers (ISPs).

(I’ll discuss wiretapping in Part III). Opponents of the bill forced the gov-

ernment to declare that information on the existence of an interception

operation against an identiﬁed target would be classiﬁed ‘Secret’. This

would have made wiretaps on Internet trafﬁc impossible without rede-

veloping all the systems used by Internet Service Providers to support an

MLS security policy — which would have been totally impractical. So the

UK government had to declare that it wouldn’t apply the laid down stan-

dards in this case because of cost.

8.7 Broader Implications of MLS

The reader’s reaction by this point may well be that mandatory access control

is too hard to do properly; there are just too many complications. This may be

true, and we are about to see the technology seriously tested as it’s deployed in

hundreds of millions of Vista PCs and Linux boxes. We will see to what extent

mandatory access control really helps contain the malware threat, whether

to commodity PCs or to servers in hosting centres. We’ll also see whether

variants of the problems described here cause serious or even fatal problems

for the DRM vision.

270 Chapter 8 ■Multilevel Security

However it’s also true that Bell-LaPadula and Biba are the simplest security

policy models we know of, and everything else is even harder. We’ll look at

other models in the next few chapters.

Anyway, although the MLS program has not delivered what was expected,

it has spun off a lot of useful ideas and know-how. Worrying about not just

the direct ways in which a secure system could be defeated but also about the

second- and third-order consequences of the protection mechanisms has been

important in developing the underlying science. Practical work on building

MLS systems also led people to work through many other aspects of computer

security, such as Trusted Path (how does a user know he’s talking to a genuine

copy of the operating system?), Trusted Distribution (how does a user know

he’s installing a genuine copy of the operating system?) and Trusted Facility

Management (how can we be sure it’s all administered correctly?). In effect,

tackling one simpliﬁed example of protection in great detail led to many things

being highlighted which previously were glossed over. The resulting lessons

can be applied to systems with quite different policies.

These lessons were set out in the ‘Rainbow Series’ of books on computer

security, produced by the NSA following the development of SCOMP and the

publication of the Orange Book which it inspired. These books are so called

because of the different coloured covers by which they’re known. The series

did a lot to raise consciousness of operational and evaluation issues that are

otherwise easy to ignore (or to dismiss as boring matters best left to the end

purchasers). In fact, the integration of technical protection mechanisms with

operational and procedural controls is one of the most critical, and neglected,

aspects of security engineering. I will have much more to say on this topic in

Part III, and in the context of a number of case studies throughout this book.

Apart from the ofﬁcial ‘lessons learned’ from MLS, there have been other

effects noticed over the years. In particular, the MLS program has had negative

effects on many of the government institutions that used it. There is a tactical

problem, and a strategic one.

The tactical problem is that the existence of trusted system components

plus a large set of bureaucratic guidelines has a strong tendency to displace

critical thought. Instead of working out a system’s security requirements in

a methodical way, designers just choose what they think is the appropriate

security class of component and then regurgitate the description of this class

as the security speciﬁcation of the overall system [1044].

One should never lose sight of the human motivations which drive a system

design, and the costs which it imposes. Moynihan’s book [907] provides a

critical study of the real purposes and huge costs of obsessive secrecy in US

foreign and military affairs. Following a Senate enquiry, he discovered that

President Truman was never told of the Venona decrypts because the material

was considered ‘Army Property’ — despite its being the main motivation for

the prosecution of Alger Hiss. As his book puts it: ‘Departments and agencies

8.7 Broader Implications of MLS 271

hoard information, and the government becomes a kind of market. Secrets

become organizational assets, never to be shared save in exchange for another

organization’s assets.’ He reports, for example, that in 1996 the number of

original classiﬁcation authorities decreased by 959 to 4,420 (following post-

Cold-War budget cuts) but that the total of all classiﬁcation actions reported

for ﬁscal year 1996 increased by 62 percent to 5,789,625.

I wrote in the ﬁrst edition in 2001: ‘Yet despite the huge increase in secrecy,

the quality of intelligence made available to the political leadership appears to

have declined over time. Effectiveness is undermined by inter-agency feuding

and refusal to share information, and by the lack of effective external critique3.

So a strong case can be made that MLS systems, by making the classiﬁcation

process easier and controlled data sharing harder, actually impair operational

effectiveness’. A few months after the book was published, the attacks of 9/11

drove home the lesson that the US intelligence community, with its resources

fragmented into more than twenty agencies and over a million compartments,

was failing to join up the dots into an overall picture. Since then, massive

efforts have been made to get the agencies to share data. It’s not clear that this

is working; some barriers are torn down, others are erected, and bureaucratic

empire building games continue as always. There have, however, been leaks

of information that the old rules should have prevented. For example, a Bin

Laden video obtained prior to its ofﬁcial release by Al-Qaida in September

2007 spread rapidly through U.S. intelligence agencies and was leaked by

ofﬁcials to TV news, compromising the source [1322].

In the UK, the system of classiﬁcation is pretty much the same as the U.S.

system described in this chapter, but the system itself is secret, with the full

manual being available only to senior ofﬁcials. This was a contributory factor in

a public scandal in which a junior ofﬁcial at the tax ofﬁce wrote a ﬁle containing

the personal information of all the nation’s children and their families to two

CDs, which proceeded to get lost in the post. He simply was not aware that

data this sensitive should have been handled with more care [591]. I’ll describe

this scandal and discuss its implications in more detail in the next chapter.

So multilevel security can be a double-edged sword. It has become en-

trenched in government, and in the security-industrial complex generally, and

is often used in inappropriate ways. Even long-time intelligence insiders have

documented this [671]. There are many problems which we need to be a ‘fox’

rather than a ‘hedgehog’ to solve. Even where a simple, mandatory, access

control system could be appropriate, we often need to control information

ﬂows across, rather than information ﬂows down. Medical systems are a good

example of this, and we will look at them next.

3Although senior people follow the ofﬁcial line when speaking on the record, once in private

they rail at the penalties imposed by the bureaucracy. My favorite quip is from an exasperated

British general: ‘What’s the difference between Jurassic Park and the Ministry of Defence? One’s

a theme park full of dinosaurs, and the other’s a movie!’

272 Chapter 8 ■Multilevel Security

8.8 Summary

Mandatory access control was developed for military applications, most

notably specialized kinds of ﬁrewalls (guards and pumps). They are being

incorporated into commodity platforms such as Vista and Linux. They have

even broader importance in that they have been the main subject of computer

security research since the mid-1970’s, and their assumptions underlie many

of the schemes used for security evaluation. It is important for the practitioner

to understand both their strengths and limitations, so that you can draw on

the considerable research literature when it’s appropriate, and avoid being

dragged into error when it’s not.

Research Problems

Multilevel conﬁdentiality appears to have been comprehensively done to death

by generations of DARPA-funded research students. The opportunity now is

to explore what can be done with the second-generation mandatory access

control systems shipped with Vista and SELinux, and with virtualization

products such as VMware and Xen; what can be done to make it easier

to devise policies for these systems that enable them to do useful work; in

better mechanisms for controlling information ﬂow between compartments;

the interaction which multilevel systems have with other security policies; and

in ways to make mandatory access control systems usable.

An ever broader challenge, sketched out by Earl Boebert after the NSA

launched SELinux, is to adapt mandatory access control mechanisms to safety-

critical systems (see the quote at the head of this chapter, and [197]). As a tool

for building high-assurance, special-purpose devices where the consequences

of errors and failures can be limited, mechanisms such as type enforcement

and role-based access control look like they will be useful outside the world of

security. By locking down intended information ﬂows, designers can reduce

the likelihood of unanticipated interactions.

Further Reading

The report on the Walker spy ring is essential reading for anyone interested

in the system of classiﬁcations and clearances [587]: this describes in great

detail the system’s most spectacular known failure. It brings home the sheer

complexity of running a system in which maybe three million people have a

current SECRET or TOP SECRET clearance at any one time, with a million

applications being processed each year — especially when the system was

Further Reading

The literature on compartmented-mode security is somewhat scattered: most

of the public domain papers are in the proceedings of the NCSC/NISSC and

Further Reading

I don’t know of any comprehensive book on banking computer systems,

although there are many papers on speciﬁc payment systems available from

the Bank for International Settlements [114]. When it comes to developing

robust management controls and business processes that limit the amount of

damage that any one staff member could do, there is a striking lack of hard

material (especially given the demands of Gramm-Leach-Bliley and Sarbanes-

Oxley). There was one academic conference in 1997 [657], and the best book

I know of on the design and evolution of internal controls, by Steven Root,

predates the Enron saga [1082]. As the interpretation put on these new laws

by the big four accountancy ﬁrms makes the weather on internal controls, and

as their gold-plating costs the economy so much, it is certainly in the public

interest for more to be published and discussed. I’ll revisit this topic in Part III.

For the speciﬁcs of ﬁnancial transaction processing systems, the cited arti-

cles [33, 34] provide a basic introduction. More comprehensive, if somewhat

dated, is [354] while [525] describes the CIRRUS network as of the mid-80s.

The transcript of Paul Stubbs’ trial gives a snapshot of the internal controls in

the better electronic banking systems in 2004 [975]; for conspicuous internal

control failure, see for example [1038]. The most informative public domain

source on the technology — though somewhat heavy going — may be the

huge online manuals for standards such as EMV [429] and the equipment that

supports it, such as the IBM 4758 and CCA [641].

CHAPTER

Physical Protection

For if a man watch too long, it is odds he will fall asleepe.

— Francis Bacon

The greatest of faults, I should say,

is to be conscious of none.

— Thomas Carlyle

11.1 Introduction

Most security engineers nowadays are largely concerned with electronic

systems, but there are several reasons why physical protection cannot be

entirely neglected. First, if you’re advising on a company’s overall risk man-

agement strategy, then walls and locks are a factor. Second, as it’s easier to

teach someone with an electrical engineering/computer science background

the basics of physical security than the other way round, interactions between

physical and logical protection will be up to the systems person to manage.

Third, you will often be asked for your opinion on your client’s installations —

which will often have been installed by local contractors who are well known

to your client but have rather narrow horizons as far as system issues are

concerned. You’ll need to be able to give informed, but diplomatic, answers.

Fourth, many security mechanisms can be defeated if a bad man has phys-

ical access to them, whether at the factory, or during shipment, or before

installation. Fifth, many locks have recently been completely compromised by

‘bumping’, an easy covert-entry technique; their manufacturers (even those

selling ‘high-security’ devices) seemed to be unaware of vulnerabilities that

enable their products to be quickly bypassed. Finally, your client’s hosting

365

366 Chapter 11 ■Physical Protection

centres will usually be its most hardened facilities, and will be the responsibility

of the systems managers who will most often seek your advice.

Much of physical security is just common sense, but there are some non-

obvious twists and there have been signiﬁcant recent advances in technology,

notably in lock-picking and other forms of covert entry. There are ideas

from criminology and architecture on how you can reduce the incidence of

crime around your facilities. And perhaps most importantly, there are burglar

alarms — which have a number of interesting system aspects.

For example, in order to defeat a burglar alarm it is sufﬁcient to make

it stop working, or — in many cases — to persuade its operators that it has

become unreliable. This raises the spectre of denial of service attacks, which are

increasingly important yet often difﬁcult to deal with. Just as we have seen

military messaging systems designed to enforce conﬁdentiality and book-

keeping systems whose goal is preserving record authenticity, monitoring

applications give us the classic example of systems designed to be depend-

ably available. If there is a burglar in my bank vault, then I do not care

very much who else gets to know (so I’m not worried about conﬁdential-

ity), or who it was who told me (so authenticity isn’t a major concern);

but I do care very much that an attempt to tell me is not thwarted. Now,

historically, about 90% of computer security research was about conﬁden-

tiality, about 9% about authenticity and 1% about availability. But actual

attacks — and companies’ expenditures — tend to be the other way round:

more is spent on availability than on authenticity and conﬁdentiality com-

bined. And it’s alarm systems, above all else, that can teach us about

availability.

11.2 Threats and Barriers

Physical protection is no different at heart from computer security: you

perform a threat analysis, then design a system that involves equipment and

procedures, then test it. The system itself typically has a number of elements:

Deter–detect–alarm–delay–respond

A facility can deter intruders using hard methods such as guards and razor-

wire fences, or softer methods such as being inconspicuous. It will then have

one or more layers of barriers and sensors whose job is to keep out casual

intruders, detect deliberate intruders, and make it difﬁcult for them to defeat

your security too quickly. This defense-in-depth will be complemented by an

alarm system designed to bring a response to the scene in time. The barriers

will have doors in them for authorized staff to go in and out; this means

some kind of entry control system that could be anything from metal keys to

biometric scanners. Finally, these measures will be supported by operational

11.2 Threats and Barriers 367

controls. How do you cope, for example, with your facility manager having

his family taken hostage by villains?

As I noted earlier, one of the ways in which you get your staff to accept

dual controls and integrate them into their work culture is that these controls

protect them, as well as protecting the assets. Unless the operational aspects

of security are embedded in the ﬁrm’s culture, they won’t work well, and this

applies to physical security as much as to the computer variety. It’s also vital to

get uniﬁed operational security across the physical, business and information

domains: there’s little point in spending $10m to protect a vault containing

$100m of diamonds if a bad man can sneak a false delivery order into your

system, and send a DHL van to pick up the diamonds from reception. That is

another reason why, as the information security guy, you have to pay attention

to the physical side too or you won’t get joined-up protection.

11.2.1 Threat Model

An important design consideration is the level of skill, equipment and moti-

vation that the attacker might have. Movies like ‘Entrapment’ might be good

entertainment, but don’t give a realistic view of the world of theft. As we have

seen in one context after another, ‘security’ isn’t a scalar. It doesn’t make sense

to ask ‘Is device X secure?’ without a context: ‘secure against whom and in

what environment?’

In the absence of an ‘international standard burglar’, the nearest I know to

a working classiﬁcation is one developed by a U.S. Army expert [118].

Derek is a 19-year old addict. He’s looking for a low-risk opportunity to

steal something he can sell for his next ﬁx.

Charlie is a 40-year old inadequate with seven convictions for burglary.

He’s spent seventeen of the last twenty-ﬁve years in prison. Although

not very intelligent he is cunning and experienced; he has picked up

a lot of ‘lore’ during his spells inside. He steals from small shops and

suburban houses, taking whatever he thinks he can sell to local fences.

Bruno is a ‘gentleman criminal’. His business is mostly stealing art. As

a cover, he runs a small art gallery. He has a (forged) university degree

in art history on the wall, and one conviction for robbery eighteen years

ago. After two years in jail, he changed his name and moved to a dif-

ferent part of the country. He has done occasional ‘black bag’ jobs for

intelligence agencies who know his past. He’d like to get into computer

crime, but the most he’s done so far is stripping $100,000 worth of mem-

ory chips from a university’s PCs back in the mid-1990s when there was

a memory famine.

Abdurrahman heads a cell of a dozen militants, most with military train-

ing. They have infantry weapons and explosives, with PhD-grade

368 Chapter 11 ■Physical Protection

technical support provided by a disreputable country. Abdurrahman

himself came third out of a class of 280 at the military academy of that

country but was not promoted because he’s from the wrong ethnic

group. He thinks of himself as a good man rather than a bad man. His

mission is to steal plutonium.

So Derek is unskilled, Charlie is skilled, Bruno is highly skilled and may

have the help of an unskilled insider such as a cleaner, while Abdurrahman

is not only highly skilled but has substantial resources. He may even have

the help of a technician or other skilled insider who has been suborned. (It’s

true that many terrorists these days aren’t even as skilled as Charlie, but it

would not be prudent to design a nuclear power station on the assumption

that Charlie would be the highest grade of attacker.)

While the sociologists focus on Derek, the criminologists on Charlie and

the military on Abdurrahman, our concern is mainly with Bruno. He isn’t the

highest available grade of ‘civilian’ criminal: that distinction probably goes to

the bent bankers and lawyers who launder money for drug gangs. (I’ll talk

about them in a later chapter.) But the physical defenses of banks and computer

rooms tend to be designed with someone like Bruno in mind. (Whether this is

rational, or an overplay, will depend on the business your client is in.)

11.2.2 Deterrence

The ﬁrst consideration is whether you can prevent bad people ever trying to

break in. It’s a good idea to make your asset anonymous and inconspicuous if

you can. It might be a nondescript building in the suburbs; in somewhere like

Hong Kong, with astronomical property prices, it might be half a ﬂoor of an

undistinguished skyscraper.

Location matters; some neighbourhoods have much less crime than oth-

ers. Part of this has to do with whether other property nearby is protected

vigorously, and how easy it is for a crook to tell which properties are pro-

tected. If some owners just install visible alarms, they may redistribute crime

to their neighbours; but invisible alarms that get criminals caught rather than

just sent next door can have strongly positive externalities. For example, Ian

Ayres and Steven Levitt studied the effect on auto thefts of Lojack, a radio

tag that’s embedded invisibly in cars and lets the police ﬁnd them if they’re

stolen. In towns where a lot of cars have Lojack, car thieves are caught quickly

and ‘chop-shops’ that break up stolen cars for parts are closed down. Ayres

and Levitt found that although a motorist who installs Lojack pays about $100

a year, the social beneﬁt from his doing this — the reduced car crime suf-

fered by others — is $1500 [100]. One implication is that good alarm services

may be undersupplied by the free market, as many people will free-ride off

their neighbours: only rich people, or people with newer cars, or who are

11.2 Threats and Barriers 369

particularly loss-averse, will install alarms. The same principle applies to real

estate; an upper-class neighbourhood in which a fair number of houses have

high-grade alarms that quietly call the police is a dangerous place for a burglar

to work.

However, that is by no means all. Since the 1960s, there has arisen a sub-

stantial literature on using environmental design to deﬂect and deter threats.

Much of this evolved in the context of low-income housing, as criminolo-

gists and architects learned which designs made crime more or less likely. In

1961, Elizabeth Wood urged architects to improve the visibility of apartment

units by residents, and create communal spaces where people would gather

and keep apartment entrances in view, thus fostering social surveillance; areas

that are out of sight are more vulnerable [1355]. In 1972, Oscar Newman devel-

oped this into the concept of ‘Defensible Space’: buildings should be designed

‘to release the latent sense of territoriality and community’ of residents [968].

Small courtyards are better than large parks, as intruders are more likely to

be identiﬁed, and residents are more likely to challenge them. At the same

time, Ray Jeffery developed a model that is based on psychology rather than

sociology and thus takes account of the wide differences between individual

offenders; it is reﬂected in our four ‘model’ villains. Intruders are not all the

same, and not all rational [1079].

Jeffery’s ‘Crime Prevention Through Environmental Design’ has been inﬂu-

ential and challenges a number of old-fashioned ideas about deterrence. Old

timers liked bright security lights; but they create glare, and pools of shadow

in which villains can lurk. It’s better to have a civilised front, with windows

overlooking sidewalks and car parks. In the old days, cyclone fences with

barbed wire were thought to be a good thing; but they communicate an

absence of personal control. A communal area with picnic seating, in which

activities happen frequently, has a greater deterrent effect. Trees also help,

as they make shared areas feel safer (perhaps a throwback to an ancestral

environment where grassland with some trees helped us see predators coming

and take refuge from them). Access matters too; defensible spaces should have

single egress points, so that potential intruders are afraid of being trapped.

It’s been found, for example, that CCTV cameras only deter crime in facilities

such as car parks where there’s a single exit [527]. There are also many tricks

developed over the years, from using passing vehicles to enhance site visibility

to planting low thorn bushes under windows. Advice on these can be found

in the more modern standards such as [229].

Another inﬂuential idea is the broken windows theory of George Kelling

and Catherine Coles [700]. They noted that if a building has a broken window

that’s not repaired, then soon vandals will break more, and perhaps squatters

or drug dealers will move in; if litter is left on a sidewalk then eventually

people will start dumping their trash there. The moral is that problems should

be ﬁxed when they’re still small. Kelling was hired as a consultant to help

370 Chapter 11 ■Physical Protection

New York clean up its vandalised subways, and inspired the zero-tolerance

policing movement of police chief William Bratton, who cracked down on

public drinkers, squeegee men and other nuisances. Both petty crime and

serious crime in New York fell sharply. Criminologists still arguing about

whether the fall was due to zero tolerance, or to other simultaneous changes

such as demographics [787] and right-to-carry laws [814].

A related set of ideas can be found in the situational crime prevention

theory of Ronald Clarke. This builds on the work of Jeffery and Newman,

and is broader than just property crime; it proposes a number of principles

for reducing crime generally by increasing the risks and effort, reducing the

rewards and provocations, and removing excuses. Its focus is largely on

designing crime out of products and out of the routines of everyday life;

it’s pragmatic and driven by applications rather than drawing on theories of

psychology and sociology [298]. It involves detailed study of speciﬁc threats;

for example, car theft is considered to be a number of different problems, such

as joyriding by juveniles, theft to get home at night, theft of parts, and theft by

professional gangs of car thieves for dismantling or sale abroad — and these

threats can be countered by quite different measures. Such empirical studies

are often criticised by criminologists who have a sociology background as

lacking ‘theory’, but are gaining inﬂuence and are not far from what security

engineers do. Many of the mechanisms discussed in this book ﬁt easily within

a framework of application-level opportunity reduction.

This framework naturally accommodates the extension of environmental

controls to other topics when needed. Thus, for example, if you’re planning

on anonymity of your premises as a defence against targeted attack, you

have to think about how you limit the number of people who know that the

basement of your Norwich sales ofﬁce actually contains your main hosting

centre. This brings in internal control, culture and even information security

policy. Governments often use multilevel policies for this; there may be a rule

that the location of all public-sector hosting centres is ‘Restricted’. Even in a

commercial ﬁrm that doesn’t burden itself with all the overhead of multilevel

security, some of the ideas I discussed in that context in Chapter 8 may be

useful.

11.2.3 Walls and Barriers

Anyway, once you’ve decided what environmental features you’ll use to deter

Derek or Charlie from trying to break into your site, and how you make it

harder for Bruno to ﬁnd out which of your sites he should break into, you then

have the problem of designing the physical barriers.

The ﬁrst task is to ﬁgure out what you’re really trying to protect. In the old

days, banks used to go to great lengths to make life really tough for robbers,

but this has its limits: a robber can always threaten to shoot a customer. So

11.2 Threats and Barriers 371

by a generation ago, the philosophy had shifted to ‘give him all the cash he

can see’. This philosophy has spread to the rest of retail. In 1997, Starbucks

reviewed physical security following an incident in which three employees

were shot dead in a bungled robbery. They decided to move the safes from the

manager’s ofﬁce to the front of the store, and made these safes highly visible

not just to staff, customers and passers-by, but also to the control room via

CCTV. A side-beneﬁt was improved customer service. The new design was

tested at a number of U.S. locations, where increased sales and loss reductions

gave a good return on investment [341]. Indeed, I notice that young people

increasingly leave their car keys by the front door at home; if someone breaks

into your house in order to steal a car, do you really want to engage them in

hand-to-hand combat?

Second, having settled your protection goals, you have to decide what

security perimeters or boundaries there will be for what purposes, and where

they’ll be located. A growth industry recently has been the provision of

vehicle traps to prevent car bombs being brought close to iconic terrorist

targets. However a common failing is to focus on rare but ‘exciting’ threats

at the expense of mundane ones. It’s common to ﬁnd buildings with stout

walls but whose roofs are easy to penetrate, for example; perhaps a terrorist

would blow himself up at your main gate to no effect, but an environmental

protester could cripple your fab and cost you hundreds of millions in lost

production by climbing on the roof, cutting a hole and dropping some burning

newspaper.

For this reason, organisations such as NIST, the Builders’ Hardware Man-

ufacturers’ Association, Underwriters’ Laboratories, and their equivalents in

other countries have a plethora of test results and standards for walls, roofs,

safes and so on. The basic idea is to assess how long a barrier will resist

an attacker who has certain resources — typically hand tools or power tools.

Normal building materials don’t offer much delay at all; a man can get through

a cavity brick wall in less than a minute using a sledgehammer, and regardless

of how good a lock you put on your front door, a police unit raiding your

house will typically break the door off its hinges with a battering-ram. So could

a robber. Thus for many years the designers of data centres, bank vaults and

the like have favoured reinforced concrete walls, ﬂoors and roofs, with steel

doorframes. Of course, if the bad guys can work undisturbed all weekend,

then even eight inches of concrete won’t keep them out.

There’s a further problem in that the organisations that certify locks, safes

and vaults often place unrealistic constraints on the tools available to an

attacker. The lock on your car steering wheel is certiﬁed to resist a man putting

his weight on it; car thieves just use a scaffolding pole, which gives them

enough leverage to break it. The typical bank vault is certiﬁed to resist attack

for ten minutes, yet your local Fire Department can get in there in two minutes

using an abrasive wheel. And if the bad guys have access to proper explosives

372 Chapter 11 ■Physical Protection

such as shaped charges, they can get through almost anything in seconds.

Another issue is the thermic lance, or burning bar, which will cut through

most barrier materials quickly: safe engineers use them to get into a vault

whose combination has been lost. Robbers can get them too. So barriers can’t

be seen in isolation. You have to evaluate them in the context of assumptions

about the threats, and about the intrusion detection and response on which

you can rely.

11.2.4 Mechanical Locks

The locksmithing industry has been seriously upset in the last couple of years

by a couple of developments that have exposed the vulnerability of many

low-cost mechanical locks.

The ﬁrst of these is bumping. This technique enables many locks to be opened

quickly and without damage by unskilled people using tools that are now

readily available. Its main target is the pin-tumbler lock originally patented by

Linus Yale in 1860 (see Figure 11.1). This was actually used in ancient Egypt,

but Yale rediscovered it and it’s often known as a ’Yale lock’, although many

ﬁrms make versions nowadays.

These locks have a cylindrical plug set inside a shell, and prevented from

rotating by a number of pin stacks. Each stack usually consists of two or three

pins, one on top of the other. The bottom pin or key pin makes direct contact

with the key; behind it is a spring-loaded top pin or driver pin that forces the

bottom pin as far down as possible in the keyway. When the correct key is

inserted, the gaps between the top pin and the bottom pin align with the edge

of the plug, creating a shear line; the plug can now be turned. A typical house or

ofﬁce lock might have ﬁve or six pins each of which could have the gap in ten

different positions, giving a theoretical key diversity of 105or 106possible key

Figure 11.1: A cutaway pin-tumbler lock (Courtesy of Marc Weber Tobias)

11.2 Threats and Barriers 373

differs. The actual number will be less because of mechanical tolerances and

key-cutting restrictions.

It had been known for years that such locks can be picked, given special

tools. You can ﬁnd details in the MIT Lock Picking Manual [1258] or in treatises

such as that by Marc Weber Tobias [1253]: the basic idea is that you twist the

plug slightly using a tension wrench, and then manipulate the pins with a

lockpick until they all line up along the shear line. Such techniques are used

by intelligence agencies, locksmiths and high-grade crooks; but they take a lot

of practice, and it’s unlawful to possess the tools in many jurisdictions (for the

laws in the USA, see [1255]. Until recently, lockpicking was generally thought

to be a threat only to high-value targets where covert entry was of particular

value to an attacker, such as investment banks and embassies.

The new discovery is that an attacker can insert a specially made bump key

each of whose teeth is set at the lowest pin position and whose shoulder is

slightly rounded. (Such keys are also known as ‘999’ keys as all the teeth are

at the lowest position, or bitting, namely number 9.) He can then place the key

under slight torsion with his ﬁngertips and tap the key head with a rubber

mallet. The shock causes the pins to bounce upwards; the applied torsion

causes them to stick as the spring pushes them back down, but with the gap at

the cylinder edge. The net effect is that with a few taps of the mallet, the lock

can be opened.

This trick had been known for years, but recently became much more effec-

tive because of better tools and techniques. It was publicised by a 2005 white

paper written by Barry Wels and Rop Gonggrijp of The Open Organization Of

Lockpickers (TOOOL), a Dutch ‘lock sports’ group (as the pastime of amateur

locksmithing is starting to be known [1337]). TV coverage spread the message

to a wide audience. There followed a technical analysis by lock expert Marc

Weber Tobias [1254]; in his view, the main threat from bumping is that it

deskills lockpicking. The consequences are potentially serious. It’s been found,

for example, that the locks in U.S. mailboxes can be opened easily, as can the

pin-tumbler locks with 70% of the U.S. domestic market. The Dutch paper,

and the subsequent publicity, have kicked off an arms race, with vendors

producing more complex designs and amateur locksmiths reporting bumping

attacks on many of them.

Until recently, locks from Medeco were thought to be unpickable (as well

as being certiﬁed as such), and the company had a dominant position in the

high-security lock market. Medeco uses secondary keying not in the form

of a sidebar but in the angle at which cuts are made in the key. In this

‘biaxial’ system, angled cuts rotate the pins to engage sliders. In 2005, Medeco

introduced the m3 which also has a simple sidebar in the form of a slider cut

into the side of the key. In 2007, Tobias reported an attack on the m3 and

biaxial locks, using a bent paperclip to set the slider and then a combination of

bumping and picking to rotate the plug [1256].

374 Chapter 11 ■Physical Protection

What can a householder do? As an experiment, I replaced my own front

door lock. The only high-security product I could ﬁnd in a store within an

hour’s drive turned out to be a rebranded Mul-T-Lock device from Israel. It

took two attempts to install, jamming the ﬁrst time; it then took about a week

for family members to learn to use the more complex deadbolt, which can

easily fail open if operated carelessly. And the next time we were visited by

someone with an intelligence background, he remarked that in the UK only

drug dealers ﬁtted such locks; so if the police ever pass by, I might end up on

their database as a suspected pusher. This dubious improvement to my home

security cost me $200 as opposed to under $20 for a standard product; and as

in practice a burglar could always break a window, our actual protection still

depends more on our location and our dogs than on any hardware. Indeed,

Yochanan Shachmurove and colleagues surveyed the residents of Greenwich,

Connecticut, and built a model of how domestic burglaries varied as a function

of the precautions taken; locks and deadbolts had essentially no effect, as there

were always alternative means of entry such as windows. The most effective

deterrents were alarms and visible signs of occupancy such as cars in the

drive [1154].

The situation for commercial ﬁrms is slightly better (but not much). The

usual standards for high-security locks in the USA, UL 437 and ANSI 156.30,

specify resistance to picking and drilling, but not to bumping; and although

pick-resistant locks are generally more difﬁcult to bump, this is no guarantee.

Knowledge does exist about which lock designs resist bumping, but you have

to look for it. (Tobias’ paper, and www.toool.org, are good starting points.) UL

has just recently taken up the issue of bumping and has formed a task force to

determine whether this method of attack should be included in their testing

of high security locks. BHMA/ANSI are also looking at the issue.

Purchasers therefore face a lemons market — as one might suspect anyway

from the glossiness and lack of technical content of many lock vendors’

marketing literature. And even expensive pick-resistant locks are often poorly

installed by builders or OEMs; when I once had to break into a cryptographic

processor with a really expensive lock, I found it could be levered open easily as

the lock turned a cam that was made of soft metal. Indeed a recent security alert

by Tobias disclosed that one of the most popular high security deadbolts could

be mechanically bypassed by sliding a narrow screwdriver down the keyway,

catching the bolt at the end and turning it, even without defeating the

extensive security protections within the lock. This design had existed for more

than twenty years and the vulnerability was unknown to the manufacturer

before the disclosure. Many high security installations employ this or similar

hardware.

The second recent class of problems are master key attacks. These have also

been known to locksmiths for some time but have recently been improved and

published, in this case by Matt Blaze. Master key systems are designed so that

11.2 Threats and Barriers 375

in addition to the individual key for each door in a building, there can be a

top-level master key that opens them all — say, for use by the cleaners. More

complex schemes are common; in our building, for example, I can open my

students’ doors while the system administrators and cleaners can open mine.

In pin-tumbler locks, such schemes are implemented by having extra cuts in

some of the pin stacks. Thus instead of having a top pin and a bottom pin

with a single cut between them, some of the pin stacks will have a middle

pin as well.

The master-key attack is to search for the extra cuts one at a time. Suppose

my key bitting is 557346, and the master key for my corridor is 232346. I make

a key with the bitting 157346, and try it in the lock. It doesn’t work. I then

ﬁle the ﬁrst position down to 257346. As 2 is a valid bitting for the ﬁrst pin, this

opens the lock, and as it’s different from my user bitting of 5, I know it is the

master key bitting for that pin. I will have to try on average four bittings for

each pin, and if three pins are master-keyed then I will have a master key after

about twelve tests. So master keying allows much greater convenience not just

to the building occupants but also to the burglar. This is really important, as

most large commercial premises use master keying. There are master-keying

systems that resist this attack — for example, the Austrian lockmaker Evva has

a system involving magnets embedded in metal keys which are much harder

to duplicate. But most ﬁelded systems appear vulnerable.

Another thing to worry about is, as always, revocation. Keyholders leave,

and may become hostile. They may have made a copy of their key, and

sell it to an attacker. Mechanical locks are easy to change singly but locking

systems generally cope very poorly with revocation. Master-key attacks are

important here, and so is bumping. Indeed, many expensive, pick-resistant

locks actually make the problem worse. They often depend on a secondary

keying mechanism such as a sidebar: the keys look like two normal pin-tumbler

keys welded together, as in Figure 11.2. The sidebar is often the same for all

the locks in the building (master-keyed systems generally require common

sidebars in locks that share master keys). So if a bad man can get hold of a

genuine key belonging to one of my students, he may be able to turn it into

a bump key that will open my door, and indeed every door in the building,

as in Figure 11.3. This may not be a problem in normal commercial premises,

but it deﬁnitely is for banks, bullion dealers and wholesale jewelers where

Figure 11.2: Key for a sidebar lock Figure 11.3: Bump key for a sidebar lock

376 Chapter 11 ■Physical Protection

attackers might spend two years planning a raid. Indeed, if such a facility had

a master-keying system using sidebar locks, and a staff member were even

suspected of having leaked a key, the prudent course of action would be to

replace every single lock.

The combined effect of bumping, bad deadbolts, master-key attacks and

other recent discoveries might be summarised as follows. Within a few

years — as the tools and knowledge spread — a career criminal like Charlie

will be able to open almost any house lock quickly and without leaving any

forensic trace, while more professional attackers like Bruno and Abdurrahman

will be able to open the locks in most commercial premises too. House

locks may not matter all that much, as Charlie will just go through the

window anyway; but the vulnerability of most mechanical locks in commercial

premises could have much more complex and serious implications. If your

responsibilities include the physical protection of computer or other assets, it’s

time to start thinking about them.

11.2.5 Electronic Locks

The difﬁculty of revocation is just one reason why electronic locks are starting

to gain market share. They have been around for a long time — hotels have

been using card locks since the 1970s. There’s an enormous diversity of

product offerings, using all sorts of mechanisms from contactless smartcards

through PIN pads to biometrics. Many of these can be bypassed in various

ways, and most of the chapters of this book can be applied in one way

or another to their design, evaluation and assurance. There are also some

electromechanical locks that combine mechanical and electronic (or magnetic)

components; some of these we just don’t know how to attack short of physical

destruction. But, from the viewpoint of a company using locks to protect

sensitive premises, the big problem is not so much the locks themselves but

how you hook up dozens or hundreds of locks in a building. Think of a

research laboratory some of whose rooms contain valuable inventions that

haven’t been patented yet, or a law ﬁrm where the ofﬁces might contain highly

sensitive documents on forthcoming takeovers. Here you worry about insiders

as well as outsiders.

In the long run, buildings may become aware of who is where, using

multiple sensors, and integrate physical with logical access control. Knowing

who went through which door in real time enables interesting security policies

to be enforced; for example, if classiﬁed material is being handled, you can

sound an alarm if there’s anyone in the room without the right clearance.

Buildings can monitor objects as well as people; in an experiment at our lab,

both people and devices carried active badges for location tracking [1318].

Electronic systems can be fully, or almost always, online, making revocation

11.2 Threats and Barriers 377

easy. As well as enforcing security policy, smart buildings could provide

other beneﬁts, such as saving energy by turning lights off and by tailoring

airconditioning to the presence of occupants. But we’re not there yet.

One practical problem, as we found when we built our new lab building, is

that only a few ﬁrms sell turnkey entry control systems. We initially wanted

to have biometric entry control based on iris scanners, as they were invented

by one of my faculty colleagues, John Daugman. But back in 2000, we couldn’t

do that. The vendors’ protocols didn’t support the kit and we didn’t have the

time and the people to build our own entry control system from scratch. (And

if a computer science department can’t do that, the average customer has no

chance.) We learned that the existing vendors operate just as other systems

houses do: they make their money from lockin (in the economic, rather than

locksmithing sense). However, the systems you buy can be extraordinarily

troublesome, dysfunctional and expensive. You end up paying $2000 for a

door lock that cost maybe $10 to manufacture, because of proprietary cabling

systems and card designs. The main limit to the lockin is the cost of ripping and

replacing the whole system — hence the vendors’ love of proprietary cabling.

Our lab is now moving to a more open system based on standard contactless

smartcards that are available from multiple vendors. The experience has taught

us that an entry control system should be managed like any other computer

system purchase, with very careful attention to maintenance costs, standards,

extensibility, and total cost of ownership. We are keen to get a system we

can install and maintain ourselves, and that allows us to specify security

policies at a decent level of abstraction. (Our old system just has a matrix

specifying which key opens which lock.) We are just starting to see the sort

of components that will make decent systems integration possible — such as

reasonably-priced door locks that run off the building’s standard Ethernet. In

short, the locksmithing industry is ripe for competition and modernisation.

It’s going to go digital, like most other industries.

It reminds me of the long conﬂict between phone companies and computer

companies. The phone companies had their solid, established ways of doing

things and assumed they could dictate to the computer industry how data

would be sent along their lines. They lost; computer ﬁrms were nimbler and

more entrepreneurial, and understood the technology better. I expect the

same will happen with locks. Within ten years, commercial entry control sys-

tems will just be computer systems, albeit with some specialised peripherals.

They will be run by your systems administrator rather than by the retired

policeman who now controls your site guards. They will ﬁnally integrate

with environmental controls, personnel systems and alarms, making the smart

building practical. The entry control industry will resist for a while, and use all

the complex government and insurance certiﬁcation requirements that have

accreted over the years, just as the phone companies used their own regulators

378 Chapter 11 ■Physical Protection

to try to strangle almost every innovation from the early data networks to

VOIP. And just as computer ﬁrms have had to learn about dependability as

they got into online systems, so there are other dependability lessons to be

learned when doing physical security. This brings us to the most automated

and sophisticated aspect of physical security, namely alarms.

11.3 Alarms

Alarms are used to deal with much more than burglary. Their applications

range from monitoring freezer temperatures in supermarkets (so staff don’t

‘accidentally’ switch off freezer cabinets in the hope of being given food to take

home), right through to improvised explosive devices in Iraq and elsewhere

that are sometimes booby-trapped. However, it’s convenient to discuss them

in the context of burglary and of protecting rooms where computer equipment

and other assets are kept. Alarms also give us a good grounding in the wider

problem of service denial attacks, which dominate the business of electronic

warfare and are a problem elsewhere too.

Standards and requirements for alarms vary between countries and between

different types of risk. You will normally use a local specialist ﬁrm for this

kind of work; but as a security engineer you must be aware of the issues.

Alarms often affect larger system designs: in my own professional practice

this has ranged from the alarms built into automatic teller machines, through

the evaluation of the security of the communications used by an alarm system

for large risks such as wholesale jewelers, to continually staffed systems used

to protect bank computer rooms.

An alarm in a bank vault is very well protected from tampering (at least

by outsiders), so is a rather simple case. In order to look at the problem more

generally, I’ll consider the task of designing an alarm system for an art gallery.

This is more interesting, because attackers can come in during the day as

members of the public and get up to mischief. We’ll imagine that the attacker

is Bruno — the educated professional art thief. The common view of Bruno is

that he organizes cunning attacks on alarm systems, having spent days poring

over the building plans in the local town hall. You probably read about this

kind of crime several times a year in the papers.

How to steal a painting (1)

A Picasso is stolen from a gallery with supposedly ‘state-of-the-art’ alarm systems

by a thief who removes a dozen rooﬁng tiles and lowers himself down a rope so as

not to activate the pressure mats under the carpet. He grabs the painting, climbs

back out without touching the ﬂoor, and probably sells the thing for a quarter of

a million dollars to a wealthy cocaine dealer.

11.3 Alarms 379

The press loves this kind of stuff, and it does happen from time to time.

Reality is both simpler and stranger. Let’s work through the threat scenarios

systematically.

11.3.1 How not to Protect a Painting

A common mistake when designing alarm systems is to be captivated by

the latest sensor technology. There’s a lot of impressive stuff on the market,

such as a ﬁber optic cable which you can loop round protected objects and

which will alarm if the cable is stretched or relaxed by less than 100nm — a

ten-thousandth of a millimeter. Isn’t modern science marvellous? So the naive

art gallery owner will buy a few feet of this magic cable, glue it to the back of

his prize Picasso and connect it to an alarm company.

How to steal a painting (2)

Bruno’s attack is to visit as a tourist and hide in a broom cupboard. At one in the

morning, he emerges, snatches the painting and heads for the ﬁre exit. Off goes

the alarm, but so what! In less than a minute, Bruno will be on his motorbike. By

the time the cops arrive twelve minutes later he has vanished.

This sort of theft is much more likely than a bosun’s chair through the roof.

It’s often easy because alarms are rarely integrated well with building entry

controls. Many designers don’t realise that unless you can positively account

for all the people who’ve entered the premises during the day, it may be

prudent to take some precautions against the ‘stay-behind’ villain — even if

this is only an inspection tour after the gallery has closed. So serious physical

security means serious controls on people. In fact, the ﬁrst recorded use of

the RSA cryptosystem — in 1978 — was not to encrypt communications but

to provide digital signatures on credentials used by staff to get past the entry

barrier to a plutonium reactor at Idaho Falls. The credentials contained data

such as body weight and hand geometry [1170, 1174]. But I’m still amazed by

the ease with which building entry controls are defeated at most secure sites I

visit — whether by mildly technical means, such as sitting on somebody else’s

shoulders to go through an entry booth, or even just by helpful people holding

the door open.

In addition, the alarm response process often hasn’t been thought through

carefully. (The Titanic Effect of over-reliance on the latest gee-whiz technology

often blinds people to common sense.) As we’ll see below, this leads to still

simpler attacks on most systems.

So we mustn’t think of the alarm mechanism in isolation. As I mentioned

above, a physical protection system has several steps: deter — detect — alarm —

delay — respond, and the emphasis will vary from one application to another.

If our opponent is Derek or Charlie, we will mostly be concerned with

380 Chapter 11 ■Physical Protection

deterrence. At the sort of targets Abdurrahman’s interested in, an attack will

almost certainly be detected; the main problem is to delay him long enough

for the Marines to arrive. Bruno is the most interesting case as we won’t have

the military budget to spend on keeping him out, and there are many more

premises whose defenders worry about Bruno than about Abdurrahman. So

you have to look carefully at the circumstances, and decide whether the bigger

problem is with detection, with delay or with response.

11.3.2 Sensor Defeats

Burglar alarms use a wide range of sensors, including:

vibration detectors, to sense fence disturbance, footsteps, breaking glass

or other attacks on buildings or perimeters;

switches on doors and windows;

passive infrared devices to detect body heat;

motion detectors using ultrasonics or microwave;

invisible barriers of microwave or infrared beams;

pressure pads under the carpet, which in extreme cases may extend to

instrumenting the entire ﬂoor with pressure transducers under each tile;

video cameras, maybe with movement detectors, to alarm automatically

or provide a live video feed to a monitoring center;

movement sensors on equipment, ranging from simple tie-down cables

through seismometers to loops of optical ﬁber.

Most sensors can be circumvented one way or another. Fence disturbance

sensors can be defeated by vaulting the fence; motion sensors by moving very

slowly; door and window switches by breaking through a wall. Designing

a good combination of sensors comes down to skill and experience (with

the latter not always guaranteeing the former). A standard, if slightly dated,

reference on sensor installation is [283].

The main problem is limiting the number of false alarms. Ultrasonics don’t

perform well near moving air such as central heating inlets, while vibration

detectors can be rendered useless by trafﬁc. Severe weather, such as lightning,

will trigger most systems, and a hurricane can increase the number of calls per

day on a town’s police force from dozens to thousands. In some places, even

normal weather can make protection difﬁcult: a site where the intruder might

be able to ski over your sensors (and even over your fence) is an interesting

challenge for the security engineer. (For an instructive worked example of

intruder detection for a nuclear power station in a snow zone see [118]).

But regardless of whether you’re in Alaska or Arizona, the principal dilemma

is that the closer you get to the object being protected, the more tightly you

11.3 Alarms 381

can control the environment and so the lower the achievable false alarm rate.

Conversely, at the perimeter it’s hard to keep the false alarm rate down. But to

delay an intruder long enough for the guards to get there, the outer perimeter

is exactly where you need reliable sensors.

How to steal a painting (3)

So Bruno’s next attack is to wait for a dark and stormy night. He sets off the

alarm somehow, taking care not to get caught on CCTV or otherwise leave any

hard evidence that the alarm was a real one. He retires a few hundred yards and

hides in the bushes. The guards come out and ﬁnd nothing. He waits half an hour

and sets off the alarm again. This time the guards don’t bother, so in he goes.

False alarms — whether induced deliberately or not — are the bane of the

industry. They provide a direct denial-of-service attack on the alarm response

force. Experience from the world of electronic warfare is that a false alarm

rate of greater than about 15% degrades the performance of radar operators;

and most intruder alarm responders are operating well above this threshold.

Deliberately induced false alarms are especially effective against sites that

don’t have round-the-clock guards. Many police forces have a policy that after

a certain number of false alarms from a given site (typically three to ﬁve in a

year), they will no longer send a squad car there until the alarm company, or

another keyholder, has been there to check.

False alarms degrade systems in other ways. The rate at which they are

caused by environmental stimuli such as weather conditions and trafﬁc noise

limits the sensitivity of the sensors that can usefully be deployed. Also, the

very success of the alarm industry has greatly increased the total number

of alarms and thus decreased police tolerance of false alarms. A common

strategy is to have remote video surveillance as a second line of defense, so the

customer’s premises can be inspected by the alarm company’s dispatcher; and

many police forces prioritize alarms conﬁrmed by such means [661]. But even

online video links are not a panacea. The attacker can disable the lighting, or

start a ﬁre. He can set off alarms in other buildings in the same street. The

failure of a telephone exchange, as a result of a ﬂood or hurricane, may well

lead to opportunistic looting.

After environmental constraints such as trafﬁc and weather, Bruno’s next

ally is time. Vegetation grows into the path of sensor beams, fences become

slack so the vibration sensors don’t work so well, the criminal community

learns new tricks, and meanwhile the sentries become complacent.

For this reason, sites with a serious physical protection requirement typically

have several concentric perimeters. The traditional approach was an outer

fence to keep out drunks, wildlife and other low-grade intruders; then level

grass with buried sensors, then an inner fence with an infrared barrier, and

ﬁnally a building of sufﬁciently massive construction to delay the bad guys

382 Chapter 11 ■Physical Protection

until the cavalry gets there. The regulations laid down by the International

Atomic Energy Agency for sites that hold more than 15g of plutonium are

an instructive read [640]. A modern hosting centre might follow the same

strategy; it may be in a nondescript building whose walls keep out the drunks

and the rats, but with more serious internal walls and sensors protecting the

machine room.

At most sites this kind of protection won’t be possible. It will be too

expensive. And even if you have loads of money, you may be in a city like

Hong Kong where real estate’s in really short supply: like it or not, your bank

computer room will just be a ﬂoor of an ofﬁce building and you’ll have to

protect it as best you can.

Anyway, the combination of sensors and physical barriers which you select

and install are still less than half the story.

11.3.3 Feature Interactions

Intruder alarms and barriers interact in a number of ways with other services.

The most obvious of these is electricity. A power cut will leave many sites

dark and unprotected, so a serious alarm installation needs backup power. A

less obvious interaction is with ﬁre alarms and ﬁreﬁghting.

How to steal a painting (4)

Bruno visits the gallery as a tourist and leaves a smoke grenade on a timer. It

goes off at one in the morning and sets off the ﬁre alarm, which in turn causes the

burglar alarm to ignore signals from its passive infrared sensors. (If it doesn’t, the

alarm dispatcher will ignore them anyway as he concentrates on getting the ﬁre

trucks to the scene.) Bruno smashes his way in through a ﬁre exit and grabs the

Picasso. He’ll probably manage to escape in the general chaos, but if he doesn’t he

has a cunning plan: to claim he was a public-spirited bystander who saw the ﬁre

and risked his life to save the town’s priceless cultural heritage. The police might

not believe him, but they’ll have a hard time prosecuting him.

The interaction between ﬁre and intrusion works in a number of ways. At

nuclear reactors, there’s typically a security rule that if a bomb is discovered,

the site’s locked down, with no-one allowed in or out; and a ﬁre safety rule that

in the event of a blaze, much of the staff have to be evacuated (plus perhaps

some of the local population too). This raises the interesting question of which

rule prevails should a bomb ever go off. And there are ﬁre precautions that

can only be used if there are effective means of keeping out innocent intruders.

Many computer rooms have automatic ﬁre extinguishers, and since fears over

the ozone layer made Halon unavailable, this means carbon dioxide ﬂooding.

ACO

2dump is lethal to untrained personnel. Getting out of a room on the

air you have in your lungs is much harder than it looks when visibility drops

11.3 Alarms 383

to a few inches and you are disoriented by the terrible shrieking noise of the

dump. A malfunctioning intruder alarm that let a drunk into your computer

room, where he lit up a cigarette and was promptly executed by your ﬁre

extinguisher, might raise a few chuckles among the anti-smoking militants but

is unlikely to make your lawyers very happy.

In any case, the most severe feature interactions are between alarm and

communication systems.

11.3.4 Attacks on Communications

A sophisticated attacker is at least as likely to attack the communications as

the sensors. Sometimes this will mean the cabling between the sensors and the

alarm controller.

How to steal a painting (5)

Bruno goes into an art gallery and, while the staff are distracted, he cuts the wire

from a window switch. He goes back that evening and helps himself.

It’s also quite possible that one of your staff, or a cleaner, will be bribed,

seduced or coerced into creating a vulnerability (attacks on really high-

value targets such as bank cash processing centres and diamond exchanges

commonly involve insiders). So frequent operational testing is a good idea,

along with sensor overlap, means to detect equipment substitution (such as

seals), strict conﬁguration management and tamper-resistant cabling. High-

value sites that take seriously the possibility of suborned insiders insist that

alarm maintenance and testing be done by two people rather than one; another

edge case is the prison system, where attacks on sensors, cabling and indeed

the very fabric of the building are so frequent that a continuing program of

test and inspection is essential. It can be useful to ask yourself, ‘How would I

do this differently if half my staff were convicts on day release?’

The old-fashioned way of protecting the communications between the alarm

sensors and the controller was physical: lay multiple wires to each sensor and

bury them in concrete, or use armored gas-pressurized cables. The more

modern way is to encrypt the communications. An example is Argus, a system

originally developed for nuclear labs [483].

But the more usual attack on communications is to go for the link between

the alarm controller and the security company which provides or organizes

the response force.

How to steal a painting (6)

Bruno phones up his rival gallery claiming to be from the security company that

handles their alarms. He says that they’re updating their computers so could

384 Chapter 11 ■Physical Protection

they please tell him the serial number on their alarm controller unit? An ofﬁce

junior helpfully does so — not realising that the serial number on the box is also

the cryptographic key that secures the communications. Bruno buys an identical

controller for $200 and, after half an hour learning how to use an EEPROM

programmer, he has a functionally identical unit which he splices into his rival’s

phone line. This continues to report ‘all’s well’ even when it isn’t.

Substituting bogus alarm equipment, or a computer that mimics it, is known

as ‘spooﬁng’. There have been reports for many years of ‘black boxes’ that

spoof various alarm controllers. As early as 1981, thieves made off with

$1.5 million in jade statuary and gold jewelry imported from China, driving

the importer into bankruptcy. The alarm system protecting its warehouse in

Hackensack, New Jersey, was cut off. Normally that would trigger an alarm at

a security company, but the burglars attached a homemade electronic device

to an external cable to ensure continuous voltage [581].

With the better modern systems, either the alarm controller in the vault

sends a cryptographic pseudorandom sequence to the alarm company, which

will assume the worst if it’s interrupted, or the alarm company sends peri-

odic random challenges to the controller which are encrypted and returned,

just as with IFF. However, the design is often faulty, having been done by

engineers with no training in security protocols. The crypto algorithm may

be weak, or its key may be too short (whether because of incompetence or

export regulations). Even if not, Bruno might be able to record the pseu-

dorandom sequence and replay it slightly more slowly, so that by early

Monday morning he might have accumulated ﬁve minutes of ‘slack’ to cover

a lightning raid.

An even more frequent cause of failure is the gross design blunder. One

typical example is having a dial-up modem port for remote maintenance,

with a default password that most users never change. Another is making

the crypto key equal to the device serial number. As well as being vulnerable

to social engineering, the serial number often appears in the purchase order,

invoice, and other paperwork which lots of people get to see. (In general, it’s a

good idea to buy your alarm controller for cash. This also makes it less likely

that you’ll get one that’s been ‘spiked’. But big ﬁrms often have difﬁculty

doing this.)

By now you’ve probably decided not to go into the art gallery business. But

I’ve saved the best for last. Here is the most powerful attack on burglar alarm

systems. It’s a variant on (3) but rather than targeting the sensors, it goes for

the communications.

How to steal a painting (7)

Bruno cuts the telephone line to his rival’s gallery and hides a few hundred yards

away in the bushes. He counts the number of men in blue uniforms who arrive,

11.3 Alarms 385

and the number who depart. If the two numbers are equal, then it’s a fair guess

the custodian has said, ‘Oh bother, we‘ll ﬁx it in the morning’, or words to that

effect. He now knows he has several hours to work.

This is more or less the standard way to attack a bank vault, and it’s also

been used on computer installations. The modus operandi can vary from

simply reversing a truck into the phone company’s kerbside junction box, to

more sophisticated attempts to cause multiple simultaneous alarms in different

premises and thus swamp the local police force. (This is why it’s so much more

powerful than just rattling the fence.)

In one case, thieves in New Jersey cut three main telephone cables, knock-

ing out phones and alarm apparatus in three police stations and thousands

of homes and businesses in the Hackensack Meadowlands. They used this

opportunity to steal Lucien Piccard wristwatches from the American distribu-

tor, with a value of $2.1 million wholesale and perhaps $8 million retail [581].

In another, an Oklahoma deputy sheriff cut the phone lines to 50,000 homes in

Tulsa before burgling a narcotics warehouse [1275]. In a third, a villain blew

up a telephone exchange, interrupting service to dozens of shops in London’s

jewelry quarter. Blanket service denial attacks of this kind, which saturate

the response force’s capacity, are the burglarious equivalent of a nuclear

strike.

In future they might not involve explosives but a software-based distributed

denial-of-service attack on network facilities, as computers and communica-

tions converge. Rather than causing all the alarms to go off in a neighborhood

(which could be protected to some extent by swamping it with police) it might

be possible to set off several thousand alarms all over New York, creating an

effect similar to that of a hurricane or a power cut but at a time convenient for

the crooks. Another possibility might be to run a service-denial attack against

the alarm company’s control centre.

An angle which seriously concerns insurers is that phone company staff

might be bribed to create false alarms. So insurance companies would prefer it

if alarm communications consisted of anonymous packets, which most of the

phone company’s staff could not relate to any particular alarm. This would

make targeted service denial attacks harder. But phone companies — who

carry most of the alarm signal trafﬁc — prefer to concentrate it in exchanges,

which makes targeted service denial attacks easier. The police are also gener-

ally suspicious of anonymous communications. These tensions are discussed

in [957].

For these reasons, the rule in the London insurance market (which does

most of the world’s major reinsurance business) is that alarm controllers

in places insured for over £20 million must have two independent means of

communication. One option is a leased line and a packet radio service. Another

is a radio system with two antennas, each of which will send an alarm if the

386 Chapter 11 ■Physical Protection

other is tampered with.1In the nuclear world, IAEA regulations stipulate that

sites containing more than 500g of plutonium or 2Kg of U-235 must have their

alarm control center and response force on the premises [640].

Where the asset you’re protecting isn’t a vault but a hosting center, the

network is also critical to your operations. There’s little point in having eight-

inch concrete walls and roofs if the single ﬁbre connecting you to the net runs

through a kerbside junction box. You’ll want two buried ﬁbres going to two

different telcos — and do you want them to be using switches and routers from

different vendors? Even so, the simplest way for a knowledgeable opponent

to take out a hosting centre is usually to cut its communications. That’s why

ﬁrms have two, three or even four centres. But it would still only take four, six

or eight holes in the ground to close down your operations. Who wants to dig,

who knows where to, and would you detect them in time?

Finally, it’s worth bearing in mind that many physical security incidents

arise from angry people coming into the workplace — whether spouses, former

employees or customers. Alarm systems should be able to cope with incidents

that arise during the day as well as at night.

11.3.5 Lessons Learned

The reader might still ask why a book that’s essentially about security in

computer systems should spend several pages describing walls, locks and

alarm systems. There are many reasons.

Dealing with service denial attacks is the hardest part of many secure

system designs. As the bad guys come to understand system level vul-

nerabilities, it’s also often the most important. Intruder alarms give us

one of the largest available bodies of applicable knowledge and

experience.

The lesson that one must look at the overall system — from intrusion

through detection, alarm, delay and response — is widely applicable,

yet increasingly hard to follow in general purpose distributed systems.

The observation that the outermost perimeter defenses are the ones that

you’d most like to rely on, but also the ones on which the least reliance

can be placed, is also quite general.

The trade-off between the missed alarm rate and the false alarm rate —

the receiver operating characteristic — is a pervasive problem in security

engineering.

1I used to wonder, back in the days when I was a banker, whether two bad men who practised

a bit could cut both cables simultaneously. I concluded that the threat wasn’t worth bothering

about for bank branches with a mere $100,000 or so in the vault. Our large cash processing centers

were staffed 24 by 7, so the threat model there focused on dishonest insiders, hostage taking and

so on.

11.4 Summary 387

There are some lessons we can learn from the alarm business. For

example, the U.S. Transportation Security Administration inserts false

alarms into airport luggage to ensure that screeners stay alert; there

are X-ray machines whose software inserts an image of a gun or bomb

about once per shift, and there are also penetration teams who insert

real objects into real suitcases. This still doesn’t work very well — a 2006

report showed that 75% of the threats got through at Los Angeles and

60% at O’Hare where the screening is done once per checkpoint per shift.

But it may be ﬁxable: at San Francisco, where screeners work for a pri-

vate company and are tested several times per shift, only 20% of threats

get through [492].

Failure to understand the threat model — designing for Charlie and hop-

ing to keep out Bruno — causes many real life failures. It’s necessary to

know what actually goes wrong, not just what crime writers think goes

wrong.

And ﬁnally, you can’t just leave the technical aspects of a security engi-

neering project to specialist subcontractors, as critical stuff will always

fall down between the cracks.

As well as these system-level lessons, there are a number of other applica-

tions where the experience of the burglar alarm industry is relevant. I already

mentioned improvised explosive devices; in a later chapter, I’ll discuss tamper-

resistant processors that are designed to detect attempts to dismantle them

and respond by destroying all their cryptographic key material.

11.4 Summary

Like it or not, security engineers have to deal with physical protection as

well as with computers and cipher systems. Indeed, just as the conﬂuence

of computers and telecomms saw computer-industry standards and methods

of working displace the old phone company ways of doing things, so the

increasing automation of physical protection systems will bring the world

of barriers, locks and alarms within our orbit. Future buildings are likely to

have much more integrated entry controls, alarms and system security. Their

management will be the job of systems administrators rather than retired

policemen.

In this chapter, I highlighted a few things worth noting. First, environmental

protection matters; things like architecture, landscaping and lighting can make

a difference, and quite a lot is known about them.

Second, physical locks are not as secure as you might think. Recent devel-

opments in covert entry technology have led to wide publication of attacks

that compromise the most widely-used mechanical locks and even the most

388 Chapter 11 ■Physical Protection

widely-used high-security locks. The bump keys and other tools needed for

such attacks on many locks are easily available online.

Third, there’s quite a lot to learn from the one aspect of physical security

that is already fairly well automated, namely alarms. Alarms provide us with

a good example of a system whose security policy hinges on availability rather

than on conﬁdentiality or integrity. They can give us some useful insights

when dealing with service-denial attacks in other contexts.

Research Problems

At the strategic level, the conﬂuence of physical security and systems security

is bound to throw up all sorts of new problems. I expect that novel research

challenges will be found by those who ﬁrst explore the information / physical

security boundary in new applications. From the viewpoint of security eco-

nomics, I’m eager to see whether the locksmithing industry will be disrupted

by its collision with digital systems, or whether the incumbents will manage

to adapt. I suspect it will be disrupted — but what does this teach us about the

strategies existing industries should adopt as the world goes digital?

At the technical level, we will probably need better middleware, in the

sense of mechanisms for specifying and implementing policy engines that can

manage both physical and other forms of protection. And as for low-level

mechanisms, we could do with better tools to manage keys in embedded

systems. As one engineer from Philips put it to me, will the smart building

mean that I have to perform a security protocol every time I change a lightbulb?

Further Reading

The best all round reference I know of on alarm systems is [118] while the

system issues are discussed succinctly in [957]. Resources for speciﬁc countries

are often available through trade societies such as the American Society

for Industrial Security [25], and though the local insurance industry; many

countries have a not-for-proﬁt body such as Underwriters’ Laboratories [1268]

in the USA, and schemes to certify products, installations or both. For progress

on lock bumping and related topics, I’d monitor troublemakers like the

Toool group, Marc Weber Tobias, and Matt Blaze; Matt has also written on

safecracking [186]. Research papers on the latest sensor technologies appear

at the IEEE Carnahan conferences [643]. Finally, the systems used to monitor

compliance with nuclear arms control treaties are written up in [1171].

CHAPTER

Monitoring and Metering

The market is not an invention of capitalism. It has existed for centuries. It is an

invention of civilization.

— Mikhail Gorbachev

12.1 Introduction

Many secure systems are concerned with monitoring and metering the envi-

ronment. They go back a long way. James Watt, the inventor of the steam

engine, licensed his patents using a sealed counter that measured the number

of revolutions an engine had made; his inspectors read these from time to time

and billed the licensee for royalties.

Electronic systems that use cryptography and tamper-resistance are rapidly

displacing older mechanical systems, and also opening up all sorts of new

applications. Ticketing is a huge application, from transport tickets through

sports tickets to theatre tickets; my case study for ticketing is the meters

used for utilities such as gas and electricity. Then I’ll turn to vehicle sys-

tems; the most familiar of these may be taxi meters but I’ll mainly discuss

tachographs — devices used in Europe to record the speed and working

hours of truck and coach drivers, and in the USA to record the comings and

goings of bank trucks. My third case study is the electronic postage meter used

to frank letters.

You will recall that in order to defeat a burglar alarm it is sufﬁcient to make

it appear unreliable. Such service-denial attacks can be tricky enough to deal

with; meters add further subtleties.

When we discussed an alarm in a bank vault, we were largely con-

cerned with attacks on communications (though sensor defeats also matter).

389

390 Chapter 12 ■Monitoring and Metering

But many metering systems are much more exposed physically. A taxi driver

(or owner) may want the meter to read more miles or more minutes than were

actually worked, so he may manipulate its inputs or try to disrupt it so that

it over-measures. With tachographs, it’s the reverse: the truck driver usually

wants to drive above the speed limit, or work dangerously long hours, so

he wants to tachograph to ignore some of the driving. Utility consumers

similarly have a motive to cause their meters to ignore some of the passing

electricity. In these the attacker can either cause the device to make false

readings, or simply to fail. There are also markets for bad people who can

sell exploits, whether by forging tickets for electricity meters or selling devices

that can be installed in vehicles to deceive a taxi meter or tachograph.

In many metering and vehicle monitoring systems (as indeed with nuclear

veriﬁcation) we are also concerned with evidence. An opponent could get an

advantage either by manipulating communications (such as by replaying old

messages) or by falsely claiming that someone else had done so. As for postal

franking systems, it’s not sufﬁcient for the attacker to cause a failure (as then

he can’t post his letters) but the threat model has some interesting twists; the

post ofﬁce is mostly concerned with stopping wholesale fraud, such as crooked

direct marketers who bribe postal employees to slip a truckload of mail into

the system. It’s thus directed internally more than externally.

Metering systems also have quite a lot in common with systems designed to

enforce the copyright of software and other digital media, which I will discuss

in a later chapter.

12.2 Prepayment Meters

Our ﬁrst case study comes from prepayment metering. There are many systems

where the user pays in one place for a token — whether a magic number, or

a cardboard ticket with a magnetic strip, or even a rechargeable token such as

a smartcard — and uses this stored value in some other place.

Examples include the stored-value cards that operate photocopiers in

libraries, lift passes at ski resorts, and washing machine tokens in univer-

sity halls of residence. Many transport tickets are similar — especially if the

terminals which validate the tickets are mounted on buses or trains and so are

not usually online.

The main protection goal in these systems is to prevent the stored-value

tokens being duplicated or forged en masse. Duplicating a single subway

ticket is not too hard, and repeating a magic number a second time is trivial.

This can be made irrelevant if we make all the tokens unique and log their use

at both ends. But things get more complicated when the device that accepts the

token does not have a channel of communication back to the ticket issuer, so

all the replay and forgery detection must be done ofﬂine — in a terminal that

12.2 Prepayment Meters 391

is often vulnerable to physical attack. So if we simply encipher all our tokens

using a universal master key, a villain could extract it from a stolen terminal

and set up in businesses selling tokens.

There are also attacks on the server end of things. One neat attack on a

vending card system used in the staff canteen of one of our local supermarkets

exploited the fact that when a card was recharged, the vending machine ﬁrst

read the old amount, then asked for money, and then wrote the amended

amount. The attack was to insert a card with some money in it, say £49, on

top of a blank card. The top card would then be removed and a £1 coin

inserted in the machine, which would duly write £50 to the blank card. This

left the perpetrator with two cards, with a total value of £99. This kind of

attack was supposed to be prevented by two levers that extended to grip

the card in the machine. However, by cutting the corners off the top card,

this precaution could easily be defeated (see Figure 12.1) [749]. This attack is

interesting because no amount of encryption of the card contents will make

any difference. Although it could in theory be stopped by keeping logs at both

ends, they would have to be designed a bit more carefully than is usual.

But we mustn’t get carried away with neat tricks like this, or we risk getting

so involved with even more clever countermeasures that we fall prey to the

Titanic Effect again by ignoring the system level issues. In most ticketing

systems, petty fraud is easy. A free rider can jump the barrier at a subway

station; an electricity meter can have a bypass switch wired across it; things

like barcoded ski lift passes and parking lot tickets can be forged with a

Figure 12.1: Superposing two payment cards

392 Chapter 12 ■Monitoring and Metering

scanner and printer. The goal is to prevent fraud becoming systematic. So

petty fraud should be at least slightly inconvenient and — more impor-

tantly — there should be more serious mechanisms to prevent anyone forging

tickets on a large enough scale to develop a black market that could affect your

client’s business.

The ﬁrst example I’ll discuss in detail is the prepayment electricity meter. I

chose this because I was lucky enough to consult on a project to electrify three

million households in South Africa (a central election pledge made by Nelson

Mandela when he took power). This work is described in some detail in [59].

Most of the lessons learned apply directly to other ticketing systems.

12.2.1 Utility Metering

In a number of European countries, householders who can’t get credit (because

they are on welfare, have court judgements against them, or whatever) buy gas

and electricity services using prepayment meters (Figure 2.2). In the old days

they were coin-operated, but the costs of coin collection led vendors to develop

token-based meters instead. The UK now has 3.6 million electricity meters and

2 million gas meters. In South Africa, development was particularly rapid

because of a national priority project to electrify the townships; as many of

the houses were informally constructed, and the owners did not even have

addresses (let alone credit ratings), prepayment was the only way to go. There

are now 5.5 million of these meters in use in South Africa, which has exported

1.5 million to other countries in Africa, Latin America and elsewhere.

The customer goes to a shop and buys a token, which may be a smartcard,

or a disposable cardboard ticket with a magnetic strip, or even just a magic

number. Of the UK’s electricity meters, 2.4 million use smartcards1and 1.2

million use magnetic tickets. Most of South Africa’s meters use a magic

number. This is perhaps the most convenient for the customer, as no special

vending apparatus is required: a ticket can be dispensed at a supermarket

checkout, at an ATM, or even over the phone.

The token is really just a string of bits containing one or more instructions,

encrypted using a key unique to the meter, which decodes them and acts

on them. Most tokens say something like ‘meter 12345 — dispense 50KWh of

electricity!’ The idea is that the meter will dispense the purchased amount and

then interrupt the supply. Some tokens have engineering functions too. For

example, if the power company charges different rates for the daytime and

evening, the meter may have to know the relative prices and the times at which

the tariffs change. Special tokens may be used to change these, and to change

11.6 million of these smartcards are repackaged in plastic keys; the other 0.8 are normal

smartcards. The packaging may improve usability, especially in a darkened house, but does not

affect the cryptographic security.

12.2 Prepayment Meters 393

Figure 12.2: A prepayment electricity meter (Courtesy of Schlumberger)

keys. The meters that use smartcards are able to report consumption patterns,

tampering attempts and so on back to the power company; however the

magnetic-ticket and magic-number meters do not have such a back channel.

The manufacture of these meters has become big business. Growth in the

third world is strong: prepayment metering was the only way the government

in South Africa could meet its election pledge to electrify millions of homes

quickly. In the developed world, the main impetus for prepayment metering

is reducing administrative costs. Electric utilities ﬁnd that billing systems can

devour 20 percent of retail customer revenue, when you add up the costs

of meter reading, billing, credit control, bad debts and so on. Prepayment

systems typically cost under 10 percent: the shop that sells the tokens gets ﬁve

percent, while the meters and the infrastructure cost about the same again.

12.2.2 How the System Works

The security requirements for prepayment meters seem straightforward.

Tokens should not be easy to forge, while genuine tokens should not work

in the wrong meter, or in the right meter twice. One strategy is to make

tokens tamper-resistant, by using smartcard chips of some kind or another;

394 Chapter 12 ■Monitoring and Metering

the alternative is to tie each token to a unique meter, so that someone can’t

use the same magic number in two different meters, and also make each token

unique using serial numbers or random numbers, so that the same token can’t

be used twice in the same meter. But it has taken a surprising amount of ﬁeld

experience to develop the idea into a robust system.

The meter needs a cryptographic key to authenticate its instructions from

the vending station. The original system had a single vending machine for

each neighbourhood, usually located in a local store. The machine has a

vend key KVwhich acts as the master key for a neighborhood and derives the

device key when needed by encrypting the meter ID under the vend key:

KID ={ID}KV

This is the same key diversiﬁcation technique described for parking lot

access devices in Chapter 3. Diversifying the vend key KVto a group of meter

keys KID provides a very simple solution where all the tokens are bought

locally. However, once the system rolled out, we found that real life was less

straightforward. In Britain, deregulation of the electricity industry led to a

multitude of electricity companies who buy power from generators and sell it

onward to households through a common infrastructure, so metering systems

must support multiple power companies with different tariff structures. In

South Africa, many people commute long distances from townships or home-

lands to their places of work, so they are never at home during business hours

and want to buy tickets where they work. So we had to support multiple

retailers, by letting customers register at an out-of-area vending station. This

meant protocols to send a customer meter key from the vending station that

‘owns’ the meter to another station, and to pass sales data in the opposite

direction for balancing and settlement, somewhat like in ATM networks. The

most recent development (2007) is online vending; a customer can buy a magic

number over the Internet or via their mobile phone from a central token server.

This server can deal directly with four million customers and also about 10,000

online vend points such as ATMs.

Statistical balancing is used to detect what are euphemistically known as non-

technical losses, that is, theft of power through meter tampering or unauthorized

direct connections to mains cables. The mechanism is to compare the readings

on a feeder meter, which might supply 30 houses, with token sales to those

houses. This turns out to be harder than it looks. Customers hoard tickets,

meter readers lie about the date when they read the meter, and many other

things go wrong. Vending statistics are also used in conventional balancing

systems, like those discussed in Chapter 10.

There have been a few cases where vending machines were stolen and

used to sell tokens in competition with the utility. These ‘ghost vendors’

are extremely difﬁcult to track down, and the early ones generally stayed in

business until the keys in all the meters were changed. The countermeasure has

12.2 Prepayment Meters 395

been to get the vending machines to maintain a credit balance in the tamper-

resistant security processor that also protects vend keys and foreign meter

keys. The balance is decremented with each sale and only credited again when

cash is banked with the local operating company; the company then sends a

magic number that reloads the vending machine with credit. The operating

company in turn has to account to the next level up in the distribution network,

and so on. So here we have an accounting system enforced by a value counter

at the point of sale, rather than just by ledger data kept on servers in a vault.

Subversion of value counters can in theory be picked up by statistical and

balancing checks at higher layers.

This distribution of security state is seen in other applications too, such

as in some smartcard-based electronic purse schemes. However, the strategic

direction for power vending is now believed to be centralisation. Now that the

communications infrastructure is more dependable, many of the original 1200

vending machines will be replaced by online vending points that get their

tokens in real time from the central service. (In banking, too, there is a move

away from ofﬂine operation as communications get more dependable.)

So what can go wrong?

12.2.3 What Goes Wrong

Service denial remains an important issue. Where there is no return channel

from the meter to the vending station, the only evidence of how much

electricity has been sold resides in the vending equipment itself. The agents

who operate the vending machines are typically small shopkeepers or other

township entrepreneurs who have little capital so are allowed to sell electricity

on credit. In some cases, agents who couldn’t pay the electricity bill to the

operating company at the end of the month just dumped their equipment

and claimed that it had been stolen. This is manageable with small agents,

but when an organization such as a local government is allowed to sell large

amounts of electricity through multiple outlets, there is deﬁnitely an exposure.

A lot of the complexity was needed to deal with untrustworthy (and mutually

mistrustful) principals.

As with burglar alarms, environmental robustness is critical. Apart from the

huge range of temperatures (as variable in South Africa as in the continental

United States) many areas have severe thunderstorms: the meter is in effect a

microprocessor with a 3-kilometer lightning conductor attached.

When meters were destroyed by lightning, the customers complained and

got credit for the value they said was still unused. So their next step was

to poke live mains wires into the meter to try to emulate the effects of the

lightning. It turned out that one make of meter would give unlimited credit if

a particular part of the circuitry (that lay under the token slot) was destroyed.

So service denial attacks worked well enough to become popular.

396 Chapter 12 ■Monitoring and Metering

It was to get worse. The most expensive security failure in the program

came when kids in Soweto observed that when there was a brown-out — a

fall in voltage from 220 to 180 volts — then a particular make of meter went to

maximum credit. Soon kids were throwing steel chains over the 11KV feeders

and crediting all the meters in the neighborhood. This was the fault of a simple

bug in the meter ROM, which wasn’t picked up because brown-out testing

hadn’t been speciﬁed. The underlying problem was that developed-country

environmental standards were inadequate for use in Africa and had to be

rewritten. The effect on the business was that 100,000 meters had to be pulled

out and re-ROMmed; the responsible company almost went bust.

There were numerous other bugs. One make of meter didn’t vend a speciﬁed

quantity of electricity, but so much worth of electricity at such-and-such a rate.

It turned out that the tariff could be set to a minute amount by vending staff,

so that it would operate almost for ever. Another allowed refunds, but a copy

of the refunded token could still be used (blacklisting the serial numbers of

refunded tokens in subsequent token commands is hard, as tokens are hoarded

and used out of order). Another remembered only the last token serial number

entered, so by alternately entering duplicates of two tokens it could be charged

up indeﬁnitely.

As with cash machines, the real security breaches resulted from bugs

and blunders, which could be quite obscure, but were discovered by accident

and exploited in quite opportunistic ways. These exploits were sometimes on

a large scale, costing millions to ﬁx.

Other lessons learned, which we wrote up in [59], were:

prepayment may be cheap so long as you control the marketing

channel, but when you try to make it even cheaper by selling prepay-

ment tokens through third parties (such as banks and supermarkets) it

can rapidly become expensive, complicated and risky. This is largely

because of the security engineering problems created by mutual mistrust

between the various organizations involved;

changes to a business process can be very expensive if they affect the

security infrastructure. For example, the requirement to sell meter tokens

at distant shops, to support commuters, was not anticipated and was

costly to implement;

recycle technology if you can, as it’s likely to have fewer bugs than some-

thing designed on a blank sheet of paper. Much of what we needed for

prepayment metering was borrowed from the world of cash

machines;

use multiple experts. One expert alone can not usually span all the issues,

and even the best will miss things;

12.3 Taxi Meters, Tachographs and Truck Speed Limiters 397

no matter what is done, small mistakes with large consequences will still

creep in. So you absolutely need prolonged ﬁeld testing. This is where

many errors and impracticalities will ﬁrst make themselves known.

Meters are a good case study for ticketing. Transport ticketing, theater

ticketing and sports ticketing may be larger applications, but I don’t know of

any serious and publicly available studies of their failure modes. In general

the end systems — such as the meters or turnstiles — are fairly soft, so the

main concern is to prevent large scale fraud. This means paying a lot of

attention to the intermediate servers such as vending machines, and hardening

them to ensure they will resist manipulation and tampering. In the case

of London transport tickets, deregulation of the railways led to reports of

problems with train companies manipulating ticket sales by booking them at

stations where they got a larger percentage of the takings, and clearly if you’re

designing a system that shares revenue between multiple vendors, you should

try to think of how arbitrage opportunities can be minimised. One still does

what one economically can to prevent the end users developing systematic

attacks on the end systems that are too hard to detect.

I’ll now look at a class of applications where there are severe and prolonged

attacks on end systems which must therefore be made much more tamper

resistant than electricity meters. The threat model includes sensor manipula-

tion, service denial, accounting ﬁddles, procedural defeats and the corruption

of operating staff. This exemplary ﬁeld of study is vehicle monitoring systems.

12.3 Taxi Meters, Tachographs and Truck Speed

Limiters

A number of systems are used to monitor and control vehicles. The most

familiar is probably the odometer in your car. When buying a used car

you’ll be concerned that the car has been clocked, that is, had its indicated

mileage reduced. As odometers become digital, clocking is becoming a type

of computer fraud; a conviction has already been reported [274]. A related

problem is chipping, that is, replacing or reprogramming the engine con-

troller. This can be done for two basic reasons. First, the engine controller

acts as the server for the remote key-entry systems that protect most mod-

ern cars from theft, as described in Chapter 3; so if you want to steal a

car without stealing the key, the engine controller is the natural target (you

might replace the controller in the street, or else tow the car and replace

or reprogram the controller later). Second, people reprogram their cars’

engine controllers to make them go faster, and the manufacturers dislike this

because of the increased warranty claims from burned-out engines. So they

try to make the controllers more tamper resistant, or at least tamper-evident.

398 Chapter 12 ■Monitoring and Metering

This fascinating arms race is described in [426]. Some vehicles now keep

logs that are uploaded to the manufacturer during servicing. General Motors

started equipping some vehicles with black boxes to record crash data in

1990. By the time the logging became public in 1999, some six million vehi-

cles had been instrumented, and the disclosure caused protests from privacy

activists [1282]. Indeed, there’s now a whole conference, ESCAR, devoted to

electronic security in cars.

There are a number of monitoring systems separate from those provided

by the engine manufacturer, and the most familiar may be the taxi meter.

A taxi driver has an incentive to manipulate the meter to show more miles

travelled (or minutes waited) if he can get away with it. There are various

other kinds of ‘black box’ used to record the movement of vehicles from

aircraft through ﬁshing vessels to armored bank trucks, and their operators

have differing levels of motive for tampering with them. A recent development

is the black boxes supplied by insurers who sell ‘pay-as-you-drive’ insurance

to young and high-risk drivers; these boxes contain satellite navigation devices

that let the insurer charge a couple of pennies a mile for driving along a country

road in the afternoon but a couple of dollars a mile for evening driving in

an inner city [1264]. It’s conceivable that within a few years this will be the

only type of insurance available to many youngsters; if the dangerous drivers

ﬂock to any ﬂat-rate contracts still on offer, they may become unaffordable.

In that case, any young man who wants to impress girls by driving around

town on a Saturday night will have a strong incentive to beat the black

box.

12.3.1 The Tachograph

The case study I’m going to use here is the tachograph. These are devices used

to monitor truck drivers’ speed and working hours; they have recently been

the subject of a huge experiment in Europe, in which old analogue devices are

being replaced by digital ones. This gives us some interesting data on how such

equipment works, and can fail; and it’s an example of how a move to digital

technology didn’t necessarily make things better. It contains useful warnings

for engineers trying to modernise systems that do analogue monitoring in

hostile environments.

Vehicle accidents resulting from a driver falling asleep at the wheel cause

several times more accidents than drunkenness (20 percent versus 3 percent

of accidents in the UK, for example). Accidents involving trucks are more

likely to lead to fatal injuries because of the truck’s mass. So most countries

regulate truck drivers’ working hours. While these laws are enforced in the

USA using weigh stations and drivers’ log books, countries in Europe use

tachographs that record a 24-hour history of the vehicle’s speed. Until 2005–6,

this was recorded on a circular waxed paper chart (Figure 12.3); since then,

12.3 Taxi Meters, Tachographs and Truck Speed Limiters 399

Figure 12.3: A tachograph chart

digital tachographs have been introduced and the two systems are currently

running side-by-side2. Eventually the analogue systems will be phased out; in

the meantime they provide an interesting study of the relative strengths and

weaknesses of analogue and digital systems. First let’s look at the old analogue

system, which is still used in most trucks on Europe’s roads.

The chart is loaded into the tachograph, which is part of the vehicle’s

speedometer/odometer unit. It turns slowly on a turntable inside the instru-

ment and a speed history is inscribed by a ﬁne stylus connected to the

speedometer. With some exceptions that needn’t concern us, it is an offence

to drive a truck in Europe unless you have a tachograph; if it’s analogue you

must have a chart installed, and have written on it your starting time and

location. You must also keep several days’ charts with you to establish that

you’ve complied with the relevant driving hours regulations (typically 8.5

hours per day with rules for rest breaks per day and rest days per week). If it’s

digital you have to have a driver card plugged into it; the card and the vehicle

unit both keep records.

2Vehicles registered since August 2004 in the UK have had to have digital systems ﬁtted, cards

have been issued since June 2005 and since August 2006 the use of digital systems in new vehicles

has been mandatory; the dates vary slightly for other EU countries.

400 Chapter 12 ■Monitoring and Metering

European law also restricts trucks to 100 Km/h (62 mph) on freeways

and less on other roads. This is enforced not just by police speed traps and

the tachograph record, but directly by a speed limiter that is also driven

by the tachograph. Tachograph charts are also used to investigate other

offences, such as unlicensed toxic waste dumping, and by ﬂeet operators to

detect fuel theft. So there are plenty reasons why a truck driver might want

to ﬁddle his tachograph. Indeed, it’s a general principle in security engineering

that one shouldn’t aggregate targets. So NATO rules prohibit money or other

valuables being carried in a container for classiﬁed information — you don’t

want someone who set out to steal your regiment’s payroll getting away

with your spy satellite photographs too. Forcing a truck driver to defeat

his tachograph in order to circumvent his speed limiter, and vice versa,

was a serious design error — but one that’s now too entrenched to change

easily.

Most of what we have to say applies just as well to taxi meters and other

monitoring devices. While the truck driver wants his vehicle to appear to have

gone less distance, the taxi driver wants the opposite. This has little effect on

the actual tampering techniques.

12.3.2 What Goes Wrong

According to a 1998 survey of 1060 convictions of drivers and operators [45],

the offences were distributed as follows.

12.3.2.1 How Most Tachograph Manipulation Is Done

About 70% of offences that result in conviction do not involve tampering but

exploit procedural weaknesses. For example, a company with premises in

Dundee and Southampton should have four drivers in order to operate one

vehicle per day in each direction, as the distance is about 500 miles and the

journey takes about 10 hours — which is illegal for a single driver to do every

day. The standard ﬁddle is to have two drivers who meet at an intermediate

point such as Penrith, change trucks, and insert new paper charts into the

tachographs. So the driver who had come from Southampton now returns

home with the vehicle from Dundee. When stopped and asked for his charts,

he shows the current chart from Penrith to Southampton, the previous day’s

for Southampton to Penrith, the day before’s for Penrith to Southampton, and

so on. In this way he can give the false impression that he spent every other

night in Penrith and was thus legal. This (widespread) practice, of swapping

vehicles halfway through the working day, is called ghosting. It’s even harder

to detect in mainland Europe, where a driver might be operating out of a depot

in France on Monday, in Belgium on Tuesday and in Holland on Wednesday.

12.3 Taxi Meters, Tachographs and Truck Speed Limiters 401

Simpler frauds include setting the clock wrongly, pretending that a hitch-

hiker is a relief driver, and recording the start point as a village with a very

common name — such as ‘Milton’ in England or ‘La Hoya’ in Spain. If stopped,

the driver can claim he started from a nearby Milton or La Hoya.

Such tricks often involve collusion between the driver and the operator.

When the operator is ordered to produce charts and supporting documents

such as pay records, weigh station slips and ferry tickets, his ofﬁce may well

conveniently burn down. (It’s remarkable how many truck companies operate

out of small cheap wooden sheds that are located a safe distance from the

trucks in their yard.)

12.3.2.2 Tampering with the Supply

The next largest category of fraud, amounting to about 20% of the total,

involves tampering with the supply to the tachograph instrument, including

interference with the power and impulse supply, cables and seals.

Old-fashioned tachographs used a rotating wire cable — as did the

speedometers in cars up until the early 1980s — that was hard to ﬁddle

with; if you jammed the truck’s odometer it was quite likely that you’d shear

off the cable. More recent analogue tachographs are ‘electronic’, in that they

use electric cables rather than rotating wire. The input comes from a sensor in

the gearbox, which sends electrical impulses as the prop shaft rotates. This has

made ﬁddling much easier! A common attack is to unscrew the sensor about

a tenth of an inch, which causes the impulses to cease, as if the vehicle were

stationary. To prevent this, sensors are ﬁxed in place with a wire and lead seal.

Fitters are bribed to wrap the wire anticlockwise rather than clockwise, which

causes it to loosen rather than break when the sensor is unscrewed. The fact

that seals are issued to workshops rather than to individual ﬁtters complicates

prosecution.

But most of the ﬁddles are much simpler still. Drivers short out the cable

or replace the tachograph fuse with a blown one. (One manufacturer tried to

stop this trick by putting the truck’s anti-lock braking system on the same fuse.

Many drivers preferred to get home sooner than to drive a safe vehicle.) Again,

there is evidence of a power supply interruption on the chart in Figure 12.3:

around 11 A.M., there are several places where the speed indicated in the

outside trace goes suddenly from zero to over 100 km/h. These indicate power

interruptions, except where there’s also a discontinuity in the distance trace.

There, the unit was open.

12.3.2.3 Tampering with the Instrument

The third category of fraud is tampering with the tachograph unit itself. This

amounts for some 6% of offences, but declined through the 1990s as tampering

402 Chapter 12 ■Monitoring and Metering

with digital communications is much easier than tampering with a rotating

wire cable used to be. The typical offence in this category is miscalibration,

usually done in cahoots with the ﬁtter but sometimes by the driver defeating

the seal on the device.

12.3.2.4 High-Tech Attacks

The state of the tampering art at the time of the 1998 survey was the equipment

in Figure 12.4. The plastic cylinder on the left of the photo is marked ‘Voltage

Regulator — Made in Japan’ and is certainly not a voltage regulator. (It

actually appears to be made in Italy.) It is spliced into the tachograph cable

and controlled by the driver using the remote control key fob. A ﬁrst press

causes the indicated speed to drop by 10%, a second press causes a drop of

20%, a third press causes it to fall to zero, and a fourth causes the device to

return to proper operation.

This kind of device accounted for under 1% of convictions but its use is

believed to be much more widespread. It’s extremely hard to ﬁnd as it can

be hidden at many different places in the truck’s cable harness. Police ofﬁcers

who stop a speeding truck equipped with such a device, and can’t ﬁnd it, have

difﬁculty getting a conviction: the sealed and apparently correctly calibrated

tachograph contradicts the evidence from their radar or camera.

Figure 12.4: A tachograph with an interruptor controlled by the driver using a radio key

fob. (Courtesy of Hampshire Constabulary, England)

12.3 Taxi Meters, Tachographs and Truck Speed Limiters 403

12.3.3 The Digital Tachograph Project

The countermeasures taken against tachograph manipulation vary by country.

In Britain, trucks are stopped at the roadside for random checks by vehicle

inspectors, and particularly suspect trucks may be shadowed across the

country. In the Netherlands, inspectors prefer to descend on a trucking

company and go through their delivery documents, drivers’ timesheets, fuel

records etc. In Italy, data from the toll booths on the freeways are used to

prosecute drivers who’ve averaged more than the speed limit (you can often

see trucks parked just in front of Italian toll booths). But such measures

are only partially effective, and drivers can arbitrage between the differing

control regimes. For example, a truck driver operating between France and

Holland can keep his documents at a depot in France where the Dutch vehicle

inspectors can’t get at them.

So the European Union took the initiative to design a uniﬁed electronic

tachograph system to replace the existing paper-based charts with smartcards.

Each driver can now get a ‘driver card’ that contains a record of his driving

hours over the last 28 days. Every new vehicle has a vehicle unit that can hold a

year’s history. There are two further types of credential: workshop cards used

by mechanics to calibrate devices, and control cards used by law enforcement

ofﬁcers to read them out at the roadside. In 1998, I was hired by the UK

Department of Transport to look at the new scheme and try to ﬁgure out what

would go wrong. After talking to a wide range of people from policemen and

vehicle inspectors to tachograph vendors and accident investigators, I wrote a

report [45]. I revisited the ﬁeld in 2007 when writing the second edition of this

book; it was simultaneously pleasing and depressing to ﬁnd that I’d mostly

predicted the problems correctly. However a few interesting new twists also

emerged.

The most substantial objection raised to the project was that it was not clear

how going digital will help combat the procedural frauds that make up 70%

of the current total. Indeed, our pair of drivers ‘ghosting’ between Dundee

and Southampton will have their lives made even easier. It will take maybe

ten years — the lifetime of a truck — to change over to the new system and

meantime a crooked company can run one new digital truck and one old

analogue one. Each driver will now have one chart and one card, with ﬁve

hours a day on each, rather than two charts which they might accidentally mix

up when stopped. This has turned out to be well-founded. In the UK, it’s now

estimated that 20% of the vehicle ﬂeet has digital tachographs — somewhat

more than would be expected — which suggests that operators have been

installing digital devices before they need to as they’re easier to ﬁddle. (So,

in the short term at least, the equipment vendors appear to be proﬁting from

the poor design of the system: in the medium term they may well proﬁt even

more, if European governments decide on yet another technology change.)

404 Chapter 12 ■Monitoring and Metering

Another objection was that enforcement would be made harder by the loss

of detailed speed and driving hours information. Back in 1998, the Germans

had wanted the driver card to be a memory device so it could contain detailed

records; the French insisted on a smartcard, as they’re proud of their smartcard

industry. So the driver card has only 32K of memory, and can only contain a

limited number of alarm events. (Indeed until a late change in the regulations

it didn’t contain data on rest periods at all.) The choice of a smartcard rather

than a memory card was probably the most critical wrong decision in the

whole programme.

12.3.3.1 System Level Problems

The response to this problem varies by country. Germany has gone for an

infrastructure of ﬂeet management systems that accept digital tachograph

data, digitized versions of the analog data from the existing paper charts,

fuel data, delivery data and even payroll, and reconcile them all to provide

not just management information for the trucking company but surveillance

data for the police. Britain has something similar, although it’s up to the police

to decide which companies to inspect; unless they do so, data on driving

infringements is only available to the employer. Germany has also introduced

a system of road pricing for heavy goods vehicles that gives further inputs

into ﬂeet management systems.

Britain thought initially of road pricing, with tachograph data correlated

with GPS location sensors in the trucks, and of using the country’s net-

work of automatic number plate reader (ANPR) cameras. The nationwide

road-charging plan has become bogged down, as initial plans involved road

charging for cars too and drew heavy resistance from motoring organisations.

So in the UK ANPR has become the main means of complementary surveil-

lance. It was initially installed around London to make IRA bombing attacks

harder, and has now been extended nationwide. It was initially justiﬁed on

the basis of detecting car tax evaders, but has turned out to be useful in many

other policing tasks. We see ANPR data adduced in more and more prose-

cutions, for everything from terrorism down to burglary. ‘Denying criminals

the use of the roads’ has now become an element in UK police doctrine.

In the case of drivers’ hours enforcement, the strategy is to verify a sample of

logged journeys against the ANPR database; where discrepancies are found,

the company’s operations are then scrutinised more closely.

However, disagreements about privacy issues and about national economic

interests have prevented any EU-wide standardization. It’s up to individual

countries whether they require truck companies to download and analyze

the data from their trucks. And even among countries that require this, ﬂeet

management systems aren’t a panacea, because of arbitrage. For example, the

German police are much more vigorous at enforcing drivers’ hours regulations

12.3 Taxi Meters, Tachographs and Truck Speed Limiters 405

than their Italian counterparts. So, under the analogue system, an Italian driver

who normally doesn’t bother to put a chart in his machine will do so while

driving over the Alps. Meanwhile, the driver of the German truck going the

other way takes his chart out. The net effect is that all drivers in a given

country are subject to the same level of law enforcement. But if the driving

data get regularly uploaded from the Italian driver’s card and kept on a PC at a

truck company in Rome then they’ll be subject to Italian levels of enforcement

(or even less if the Italian police decide they don’t care about accidents

in Germany). The ﬁx to this was extraterritoriality; an Italian truck driver

stopped in Germany can be prosecuted there if he can’t show satisfactory

records of his driving in Italy for the week before he crossed the border.

12.3.3.2 Other Problems

So the move from analogue to digital isn’t always an improvement. As well as

the lower tamper-resistance of electronic versus mechanical signalling, and the

system level problem that the location of the security state can’t be tackled in a

uniform way, there are several further interesting problems with tachographs

being digital.

A curious problem for the policy folks is that digital tachographs have for

the ﬁrst time caused digital signatures to turn up in court in large numbers.

For many years, security researchers have been writing academic papers with

punchlines like ‘the judge then raises X to the power Y, ﬁnds it’s equal to Z,

and sends Bob to jail’. The reality is very different. Apparently judges ﬁnd

digital signatures too ‘difﬁcult’ as they’re all in hex. The police, always eager

to please, have resolved the problem by applying standard procedures for

‘securing’ digital evidence. When they raid a dodgy trucking company, they

image the PC’s disk drive and take copies on DVDs that are sealed in evidence

bags. One gets given to the defence and one kept for appeal. The paper

logs documenting the copying are available for Their Worships to inspect.

Everyone’s happy, and truckers duly get ﬁned as before.

From the operational viewpoint, the main emerging problem is that many

drivers have more than one driver card. This is an offence everywhere but

that doesn’t stop it! One source of cards is to borrow them from drivers

who use them only occasionally — for example because they usually drive

analogue trucks, or trucks under 3.5 tonnes. Another is that many drivers

have more than one address; the Jean Moulin of Toulouse may also be Jean

Moulin of Antwerp. A database, ‘Tachonet’, was set up to try to catch duplicate

applications across European countries but it doesn’t seem to work very well.

For example, drivers may forget their middle name in one of their countries of

residence.

Second, there will be new kinds of service denial attacks (as well as the

traditional ones involving gearbox sensors, fuses and so on). A truck driver

406 Chapter 12 ■Monitoring and Metering

can easily destroy his smartcard by feeding it with mains electricity (in fact,

even a truck’s 24 volts will do ﬁne). Under the regulations he is allowed to

drive for 15 days while waiting for a replacement. As static electricity destroys

maybe 1% of cards a year anyway, it is hard to prosecute drivers for doing this

occasionally. Similar card-destruction attacks have been perpetrated on bank

smartcard systems in order to force a merchant back into less robust fallback

modes of operation.

Third, I mentioned that the loss of detailed, redundant data on the tacho-

graph chart makes enforcement harder. At present, experienced vehicle

inspectors have a ‘feel’ for when a chart isn’t right, but the analogue trace is

replaced by a binary signal saying either that the driver infringed the regula-

tions or that he didn’t. This spills over into other enforcement tasks; analogue

charts were often used to collect evidence of illegal toxic waste dumping, for

example, as the recorded speed patterns often give a knowledgeable inspector

a good idea of the truck’s route.

Next, some of the cards in the system (notably the workshop cards used

to set up the instruments, and the control cards used by police and vehicle

inspectors) are very powerful. They can be used to erase evidence of wrong-

doing. For example, if you use a workshop card to wind back the clock in

a vehicle unit from 10th July to 8th July, then the entries for July 9th and

10th become unreadable. (Of course the vendors should have implemented a

proper append-only ﬁle system, but they had only 32Kb smartcards to work

with not 32Mb memory cards.) Some countries have therefore gone to great

lengths to minimise the number of workshop cards that fall into bad hands.

In the UK, for example, truck mechanics have to pass a criminal records check

to get one; yet this isn’t foolproof as it’s often companies that get convicted,

and the wealthy owners of crooked truck-maintenance ﬁrms just set up new

ﬁrms. There’s no company licensing scheme, and although wrongdoers can be

blacklisted from acting as directors of licensed ﬁrms, crooks just hide behind

nominee directors.

Various technical attacks are possible. When assessing the security of the

proposed design in the late 1990s, I was concerned that villains might physically

reverse-engineer a card, extracting its master key and enabling a powerful

workshop or police card to be forged. Since then, tamper-resistance has got

better, so attacks are more expensive; and my 1998 critique helped move the

design from shared-key to public-key crypto, limiting the damage from a

single card compromise. But the most recent attacks on smartcard systems

are deﬁnitely a concern. Consider for example relay attacks, in which a bogus

card, connected to a bogus reader using mobile phones, enables a smartcard in

country A to appear to be in country B [401]. In Chapter 10, I discussed this as

a means of bank fraud. In the tachograph world, its implications are different.

If any police card, or workshop card, can be used to erase evidence of a crime,

then what’s to stop a corrupt mechanic or policeman in Sicily or in Romania

12.3 Taxi Meters, Tachographs and Truck Speed Limiters 407

from using his card to destroy evidence in London or in Berlin? This really is

arbitrage with a vengeance. It’s no longer enough for British coppers or the

German Polizei to be honest, if ofﬁcials from less well governed countries can

indulge in telepresence. Perhaps we’ll need region coding for policemen just

as we have for DVDs.

This helps illustrate that key management is, as always, difﬁcult. This is a

pervasive problem with security systems in vehicles — not just tachographs

and taxi meters, but even such simple devices as card door locks and the

PIN codes used to protect car radios against theft. If the garage must always

be able to override the security mechanisms, and a third of garage mechan-

ics have criminal records, then what sort of protection are you buying? (In

my own experience, PIN-protected radios are just a protection racket for the

garages — you end up paying a garage £20 to tell you the PIN after you get a

ﬂat battery.)

12.3.3.3 The Resurrecting Duckling

In the late 1990s, a European Union regulation decreed that, in order to

frustrate the use of interruptors of the kind shown in Figure 12.4 above, all

digital tachographs had to encrypt the pulse train from the gearbox sensor to

the vehicle unit. As both of these devices contain a microcontroller, and the

data rate is fairly low, this shouldn’t in theory have been a problem. But how

on earth could we distribute the keys? If we just set up a hotline that garages

could call, it is likely to be abused. There’s a long history of ﬁtters conspiring

with truck drivers to defeat the system, and of garage staff abusing helplines

to get unlocking data for stolen cars and even PIN codes for stolen radios.

One solution is given by the resurrecting duckling security policy model. This

is named after the fact that a duckling emerging from its egg will recognize

as its mother the ﬁrst moving object it sees that makes a sound: this is called

imprinting. Similarly, a ‘newborn’ vehicle unit, just removed from the shrink

wrap, can recognize as its owner the ﬁrst gearbox sensor that sends it a secret

key. The sensor does this on power-up. As soon as this key is received, the

vehicle unit is no longer a newborn and will stay faithful to the gearbox

sensor for the rest of its ‘life’. If the sensor fails and has to be replaced, a

workshop card can be used to ‘kill’ the vehicle unit’s key store and resurrect

it as a newborn, whereupon it can imprint on the new sensor. Each act of

resurrection is indelibly logged in the vehicle unit to make abuse harder. (This

at least was the theory — the implementation fell somewhat short in that in

one unit the error code for sensor rekeying is the same as the error code for a

power outage.)

The resurrecting duckling model of key management was originally devel-

oped to deal with the secure imprinting of a digital thermometer or other

piece of medical equipment to a doctor’s PDA or a bedside monitor. It can

408 Chapter 12 ■Monitoring and Metering

also be used to imprint consumer electronics to a remote control in such a way

as to make it more difﬁcult for a thief who steals the device but not the

controller to make use of it [1218].

Another possible application is weapon security. Many of the police ofﬁcers

who are shot dead on duty are killed with their own guns, so there has been

some interest in safety mechanisms. One approach is to design the gun so

it will ﬁre only when within a foot or so of a signet ring the ofﬁcer wears.

The problem is managing the relationship between rings and guns, and a

possible solution is to let the gun imprint on any ring, but with a delay of a

minute or so. This is not a big deal for the policeman signing a gun out of

the armory, but is a problem for the crook who snatches it. (One may assume

that if the policeman can’t either overpower the crook or run for it within a

minute, then he’s a goner in any case.) Such mechanisms might also mitigate

the effects of battleﬁeld capture of military weapons, for which passwords are

often unacceptable [175].

However, one last problem with the idea of a secure sensor has emerged

in the last two years, since digital tachographs started shipping. The folks in

Italy who brought you the interruptor now have a new product. This is a

black box containing electromagnets and electronics to simulate a gearbox.

The errant truck driver unscrews his gearbox sensor and places it in this virtual

gearbox. The box comes with its own cable and a sensor that he plugs into his

actual gearbox. The system now operates as before; on command it will either

relay impulses faithfully, or discard them, or ﬁlter some of them out. The

dodgy pulse-train arrives at the tachograph as before, but this time beautifully

encrypted using triple-DES. Secure sensing is harder than it looks!

12.4 Postage Meters

My third case history of metering is the postage meter. Postage stamps were

introduced in Britain 1840 by Sir Rowland Hill to simplify charging for post,

and developed into a special currency that could be used for certain purposes,

from paying for postage to paying certain taxes and topping up the value of

postal money orders. Bulk users of the postal system started to ﬁnd stamps

unsatisfactory by the late 19th century, and the postage meter was invented in

1889 by Josef Baumann. Its ﬁrst commercial use was in Norway in 1903; in the

USA Arthur Pitney and Walter Bowes had a meter approved for use in 1920

and built a large business on it. Early postal meters were analogue, and would

print a stamp (known as an indicium) on a letter, or on a tape to stick on a

parcel. The indicium had a date so that old indicia couldn’t be peeled off and

reused. Each meter had a mechanical value counter, protected by a physical

seal; every so often you’d take your meter into the post ofﬁce to be read and

12.4 Postage Meters 409

reset. Fraud prevention relied on users taking their mail to the local post ofﬁce,

which knew them; the clerk could check the date and the meter serial number.

In 1979, Pitney Bowes introduced a ‘reset-by-phone’ service, which enabled

ﬁrms to buy an extra $500 worth of credit over the phone; the implementation

involved a mechanical one-time pad, with the meter containing a tape with

successive recharge codes [328]. In 1981, this was upgraded to a DES-based

system that enabled a meter to be recharged with any sum of money. The

recharge codes were calculated in part from the value counter — so if the ﬁrm

lied about how much postage they’d used, they couldn’t recharge the device.

However, these meters still produced inked indicia.

In 1990, Jos´

e Pastor of Pitney Bowes suggested replacing stamps and indicia

with digital marks protected by digital signatures [1007]. This caught the

attention of the U.S. Postal Service, which started a program to investigate

whether cryptography could help produce better postage meters. One concern

was whether the availability of color scanners and copiers would make stamps

and indicia too easy to forge. A threat analysis done for them by Doug Tygar,

Bennett Yee and Nevin Heintze revealed that the big problem was not so

much the forging or copying of stamps, or even tampering with meters to get

extra postage. It was bulk mailers corrupting Postal Service employees so as to

insert truckloads of junk mail into the system without paying for them [1265].

As a bulk mailer who was ﬁddling his meter seriously would risk arousing

the suspicion of alert staff, there was a temptation to cut them in on the deal;

and then it was natural to forge a meter plate whose inducting post ofﬁce was

elsewhere. By 1990 U.S. Postal service losses were in nine ﬁgures, and through

the 1990s there were a number of high-proﬁle convictions of bulk mailers who

had manipulated their meters, and got away with millions of dollars of free

postage [190].

This led to a development programme to produce a design based on digital

signatures, generated by tamper-resistant processors in the postage meters,

that were developed from Pastor’s ideas into an open standard available to

multiple competing manufacturers. The basic idea is that the indicium, which

is machine-readable, contains both the sender and recipient postal codes, the

meter number, the date, the postage rate, the amount of postage ever sold

by the meter and the amount of credit remaining in it, all protected with a

digital signature. The private signature key is kept in the meter’s processor

while its corresponding public signature veriﬁcation key is kept in a Postal

Service directory, indexed by the meter serial number. In this way, postal

inspectors can sample mail in bulk at sorting ofﬁces, checking that each item

is not only franked but on a logical route from its ostensible source to its

destination.

The USA introduced the technology in 2000, with Germany next in 2004 and

Canada in 2006; other countries are moving to follow suit. By 2006, the USA

had about 450,000 digital meters out of a total market of 1.6 million, and it’s

410 Chapter 12 ■Monitoring and Metering

expected that digital devices will have taken over completely by 2012. Also, by

2006, all U.S. postal facilities had the scanners needed to read the new indicia,

of which an example is illustrated in Figure 12.5 below.

Such indicia can be produced by postage meters that are drop-in replace-

ments for the old-fashioned devices; you weigh a letter, frank it, and get

billed at the end of the month. You don’t have to take the meter in to be

read though, as that can be done over the Internet for a credit meter, while

if you buy a prepayment meter you replenish it by phoning a call center and

buying a magic number with your credit card. This works in much the same

way as the prepayment electricity meters discussed earlier in this chapter. The

tamper-resistance is used to implement what’s in effect prepaid digital cash

(or preauthorized digital credit) that’s kept in the meter on the customer’s

premises.

Indicia can also be produced without any special equipment locally; you can

buy a stamp over the Internet by simply specifying the sender and destination

postal codes. This facility, ‘online postage’, is aimed at small businesses and

people working from home who don’t send enough mail for it to be worth

their while buying a meter. Both metered and online postage are cheaper than

stamps to distribute. Also, it becomes possible to manage the system much

better, by tracking volumes and proﬁtability of mail down to local level. This

matters as many countries’ postal systems are deregulated and open up to

competition. And despite predictions that email would displace physical mail,

post volumes continue to grow by 1% a year, and it turns out that households

with broadband actually receive more post.

FIM Mark

Town Circle or Postmark

Licensing Post Office/Mailed From ZIP Code

Date of Mailing (as required)

Postage Amount

Device ID

Rate Category

•Signature Algorithm Flag

•Device ID

•Date of Mailing

•Postage

•Licensing ZIP Code

•Destination ZIP Code

•Software ID

•Ascending Register

•Descending Register

•Digital Signature

•Rate Category

•Reserve Field

•Version

Two Dimensional Barcode

DEVICE12345678901234

MERRIFIELDVA 22180

US POSTAGE

$0.32

FIRST-CLASS

March 31, 1998

optional

advertising

art area

Figure 12.5: One of the new formats for U.S. postal meters (courtesy of Symbol

Technologies)

12.4 Postage Meters 411

So, all told, digital postal meters offer more ﬂexibility to both users and

postal services than the old analogue ones. But what about security?

One way of looking at postage meters is that they’re a slight extension of

the utility metering model. There is a tamper-resistant processor, either in the

meter itself, or attached to a web server in the case of online postage; this has

a value counter and a crypto key. It dispenses value, by creating indicia in the

requested denominations, until the value counter is exhausted. It then requires

replenishment via a cryptographically-protected message from a control unit

higher up in the chain. There are some additional features in each case. Many

postage meters include a ‘Clark-Wilson’ feature whereby the value counter

actually consist of two counters, an Ascending Register (AR) containing

the total value ever dispensed by the meter, and a Descending Register (DR)

indicating the remaining credit. The balancing control is AR + DR = TS, the ‘total

setting’, that is, the total of all the sales made by or authorised for that device.

If the balance fails, the meter locks up and can only be accessed by inspectors.

An alternative way of looking at postage meters is that they constitute a

distributed mint. In Sir Rowland Hill’s original British postage system, the

penny black stamps were printed at the mint; and postage engineers to this

day refer to the step of generating an indicium as ‘minting’, whether it’s done

in a tamper-resistant processor in a meter or in an online server. These systems

can provide a lot of useful experience for anyone wanting to design novel

payment and e-money protocols.

The full threat model includes stolen postage meters, meters that have been

tampered with to provide free postage, genuine meters used by unauthorised

people, mail pieces with indicia of insufﬁcient value to cover the weight and

service class, and straightforward copies of valid indicia. Various sampling

and other tests are used to control these risks. Subtleties include how you

deal with features like certiﬁed mail and reply mail. There are also national

differences on matters ranging from which authentication algorithms are used

to what sort of usage data the meters have to upload back to the postal service.

One interesting development is that, as operators get real experience of

digital postal metering, the industry is moving away from the initial design

of using digital signatures to one of using message authentication codes.

Signatures appealed because they were elegant; but in real life, signature

veriﬁcation is expensive, and has also turned out to be unnecessary. Equipment

at major sorting ofﬁces must process thousands of mail pieces a minute, and

even using low-exponent RSA, this entails a lot of computation. One argument

for signatures was that indicia could be veriﬁed even when central servers

are ofﬂine; but in real operations, postal services usually verify indicia as

an ofﬂine batch operation. This means that forged mail pieces go through

initially and are only intercepted once a pattern of abuse emerges. Once the

veriﬁcation is done ofﬂine, this can just as easily be MAC veriﬁcation as

signature veriﬁcation. (The central servers have hardware security modules

412 Chapter 12 ■Monitoring and Metering

with master keys that were diversiﬁed to a MAC key in each meter.) It turns

out that only two digits of the MAC are needed, as this is enough to detect any

systematic abuse before it becomes signiﬁcant [328].

The most recent optimisation is for the postal service not to do any cryp-

tography at all, but to contract it out to the meter vendors. This in turn means

that indicia are veriﬁed only in the home postal system, as overseas systems

will often use different vendors. (So if you want to bribe a postal employee

to let a few tons of junk mail into the system, the place to do it is now at

a border crossing.) The upshot of the move away from public-key cryptog-

raphy is a diversity of architectures, and sometimes multiple architectures

in one country. Canada, for example, uses both signatures and MACs on its

indicia.

How stuff actually breaks in real life is — as always — instructive. In the

German post ofﬁce’s ‘Stampit’ scheme, a user buys ‘smart pdf’ ﬁles that contact

the post ofﬁce to say they’re being printed, without any interaction with the

user or her software. If the paper jams, or the printer is out of toner, then tough.

So users arrange to photocopy the stamp, or to copy it to a ﬁle from which it

can be printed again if need be. The UK system has learnt from this: although

a stamp is grey-listed when a user PC phones home and says it’s been printed,

the grey doesn’t turn to black until the stamp appears at the sorting ofﬁce. The

difference in syntax is subtle: the German system tried to stop you printing

the stamp more than once, while the British system more realistically tries to

stop you using it more than once [592].

All told, moving to digital postal meters involves a nontrivial investment,

but enables much better control than was possible in the old days, when postal

inspectors had to refer to paper records of mechanical meter readings. The

hardware tamper-resistance also facilitates prepayment business models that

extend the service’s scope to many more customers and that also improve a

service’s cash ﬂow and credit control. Unlike the case of digital tachographs,

digital postal meters appear to be a success story.

12.5 Summary

Many security systems are concerned one way or another with monitoring or

metering some aspect of the environment. They range from utility meters to taxi

meters, tachographs, and postal meters. We’ll come across further metering

and payment systems in later chapters, ranging from the mechanisms used to

stop printer cartridges working once they have printed a certain number of

pages, to prepay scratch cards for mobile phone use, which may be the world’s

largest application-speciﬁc payment mechanism.

Many monitoring, metering and payment systems are being redesigned as

the world moves from analogue to digital technology. Some of the redesigns

Research Problems 413

are a success, and others aren’t. The new digital prepayment electricity meters

have been a success, and are being introduced around the developing world

as an enabling technology that lets utility companies sell power to people who

don’t even have addresses, let alone credit ratings. Digital tachographs have

been much less impressive; they just do what the old analogue systems did,

but less well. Our third example, postage meters, appear to be a success.

As with burglar alarms, the protection of these systems is tied up with

dependability; if you’re designing such a thing, you have to think long and hard

about what sort of service denial attacks are possible on system components.

Key management can be an issue, especially in low cost widely distributed

systems where a central key management facility can’t be justiﬁed or an

adequate base of trustworthy personnel doesn’t exist. Systems may have to deal

with numerous mutually suspicious parties, and must often be implemented

on the cheapest possible microcontrollers. Many of them are routinely in the

hands of the opponent. And there are all sorts of application-level subtleties

that had better be understood if you want your design to succeed.

Research Problems

We’re gradually acquiring a set of worked examples of secure embedded

metering, thanks to the kinds of systems described here. We don’t yet have

a really general set of tools for building them. At the component level,

we have crypto algorithms as seen in Chapter 5, protocols as described in

Chapter 3, security policies like Clark-Wilson which I described in Chapter

10, and tamper resistant processors, which I’ll describe later. However we

don’t have many concepts, abstractions, or middleware that help us pull these

components together into larger building blocks. Although the mechanisms

(and products) developed for automatic teller machine networks can be

adapted (and are), much of the design work has to be redone and the end

result often has vulnerabilities. Top level standards for ways in which crypto

and other mechanisms can be built into a range of monitoring and ticketing

systems might save engineers a lot of effort. Meanwhile we have to rely on

case histories like those presented here. Metering applications are particularly

useful because of the pervasive mutual mistrust caused not just by competing

commercial entities but by the presence of dishonest staff at every level, as

well as dishonest customers; and the fact that most of the equipment is in the

custody of the attackers.

Again, there are questions for the security economist, and the business school

researchers. Why did some digitisations of existing metering systems work

well (utilities, postage) while others were much less impressive (tachographs)?

Why were some disruptive, in that new entrants successfully challenged

the previous incumbent suppliers, while in other cases (such as postage) the

414 Chapter 12 ■Monitoring and Metering

existing suppliers managed the transition to better digital systems and largely

saw off competition from dotcom startups?

Further Reading

Prepayment electricity meters are described in [59]. Tachographs are written

up in [45]; other papers relevant to transport appear in the annual ESCAR

conference on electronic security in cars. The early work on postal meters is

in [1265] and the U.S. regulations can be found in [894]. However by far the

most detailed exposition of postage meter security is in a book written by

Gerrit Bleumer, a scientist at Francotyp-Postalia that took a leading role in the

program [190].

CHAPTER

Nuclear Command and Control

In Germany and Turkey they viewed scenes that were particularly distressing. On the

runway stood a German (or Turkish) quick-reaction alert airplane loaded with

nuclear weapons and with a foreign pilot in the cockpit. The airplane was ready to

take off at the earliest warning, and the nuclear weapons were fully operational.

The only evidence of U.S. control was a lonely 18-year-old sentry armed with a

carbine and standing on the tarmac. When the sentry at the German airﬁeld was

asked how he intended to maintain control of the nuclear weapons should the pilot

suddenly decide to scramble (either through personal caprice or through an order

from the German command circumventing U.S. command), the sentry replied that he

would shoot the pilot; Agnew directed him to shoot the bomb.

— Jerome Wiesner, reporting to President

Kennedy on nuclear arms command

and control after the cuban crisis

13.1 Introduction

The catastrophic harm that could result from the unauthorized use of a nuclear

weapon, or from the proliferation of nuclear technology to unsuitable states

or substate groups, has led the U.S. and other nuclear powers to spend

colossal amounts of money protecting not just nuclear warheads but also the

supporting infrastructure, industry and materials. The growing concern about

global warming makes nuclear protection all the more critical: how do we

build new nuclear power stations without greatly increasing the risk that bad

people get hold of weapons or ﬁssile materials?

A surprising amount of nuclear security know-how has been published. In

fact, severe limits have been placed on how much could be kept secret even if

this was thought desirable. Many countries are capable of producing nuclear

415

416 Chapter 13 ■Nuclear Command and Control

weapons but have decided not to (Japan, Australia, Switzerland, ...)andso

maintain controls on nuclear materials in a civilian context. Much of the real

force of nonproliferation is cultural, built over the years through diplomacy

and through the restraint of nuclear powers who since 1945 forebore use of

these weapons even when facing defeat at the hands of non-nuclear states.

The culture is backed by international nonproliferation agreements, such as

the Convention on the Physical Protection of Nuclear Material [640], enforced

by the International Atomic Energy Agency (IAEA).

Eleven tons of plutonium are produced by civil reactors each year, and if the

human race is to rely on nuclear power long-term then we’ll be burning it in

reactorsaswellasjustmakingitasaside-effectofburninguranium.Soways

have to be found to guard the stuff, and these have to inspire international

conﬁdence — not just between governments but from an increasingly sceptical

public1.

So a vast range of security technology has spun off from the nuclear

program. The U.S. Department of Energy weapons laboratories — Sandia,

Lawrence Livermore and Los Alamos — have worked for two generations to

make nuclear weapons and materials as safe as can be achieved, using almost

unlimited budgets. I’ve already mentioned some of their more pedestrian

spin-offs, from the discovery that passwords of more than twelve digits were

not usable under battleﬁeld conditions to high-end burglar alarm systems. The

trick of wrapping an optical ﬁber round the devices to be protected and using

interference effects to detect a change in length of less than a micron, is also

one of theirs — it was designed to loop round the warheads in an armoury

and alarm without fail if any of them are moved.

In later chapters, we’ll see still more technology of nuclear origin. For

example, iris recognition — the most accurate system known for biomet-

ric identiﬁcation of individuals — was developed using U.S. Department of

Energy funds to control entry to the plutonium store, and much of the exper-

tise in tamper-resistance and tamper-sensing technology originally evolved

to prevent the abuse of stolen weapons or control devices. The increased

tension since 9/11 has led to further spread of controls, especially once it was

realised that for terrorist purposes it isn’t necessary to get ﬁssile materials like

plutonium or uranium-235. A ‘dirty bomb’ — a device that would disperse

radioactive material over a city block — is also a real threat, and one that

jihadists have talked about. It might not kill anyone but it could lead to panic,

and in a ﬁnancial center it could cause great economic damage. For example,

in March 2007, GAO investigators set up a bogus company and got a license

from the Nuclear Regulatory Commission authorizing them to buy isotopes

with which they could have built such a radiological dispersion device. What’s

1For example, the British government was seriously embarrassed in 2007 when its safety

arrangements for its 100-ton plutonium stockpile were criticised by eminent scientists [1089].

13.2 The Evolution of Command and Control 417

more, the license was printed on ordinary paper; the investigators altered it to

change the quantity of material they were allowed to buy, then used it

to order dozens of moisture density gauges containing americium-241 and

cesium-137 [757]. This incident suggests that materials control may spread

quite widely in the economy, and it may involve the wider deployment of

many of the technologies described in this book.

Nuclear safety continually teaches us lessons about the limits of assurance.

For example, it’s tempting to assume that if a certain action that you don’t

want to happen has a probability of 1 in 10 of happening through human error,

then by getting ﬁve different people to check, you can reduce the probability

to 1 in 100,000. The U.S. Air Force thought so too. Yet in October 2007, six

U.S. hydrogen bombs went missing for 36 hours after a plane taking cruise

missiles from Minot Air Force Base in North Dakota to Barksdale in Louisiana

was mistakenly loaded with six missiles armed with live warheads. This was

supposed to be prevented by the handlers inspecting all the missiles in the

storage area and checking them against a schedule (which was out of date), by

ground crew waiting for the inspection to ﬁnish before moving any missiles,

(they didn’t), by ground crew inspecting the missiles (they didn’t look in little

glass portholes to see whether the warheads were real or dummy), by the

driver calling in the identiﬁcation numbers to a control center (nobody there

bothered to check), and ﬁnally by the navigator during his preﬂight check

(he didn’t look at the wing with the live missiles). The plane took off, ﬂew to

Louisiana, landed, and sat unguarded on the runway for nine hours before the

ground crew there arrived to unload the missiles and discovered they were

live [127, 380]. This illustrates one of the limits to shared control. People will

rely on others and slack off — a lesson also known in the world of medical

safety. Indeed, in the USAF case it turned out that the airmen had replaced the

ofﬁcial procedures with an ‘informal’ schedule of their own. So how can you

design systems that don’t fail in this way?

In this chapter I’m going to describe the nuclear safety environment and

some of the tricks that might still ﬁnd applications (or pose threats) elsewhere.

This chapter has been assembled from public sources — but even from the

available material there are useful lessons to be drawn.

13.2 The Evolution of Command and Control

The ﬁrst atomic bomb to be used in combat was the ‘Little Boy’ dropped on

Hiroshima. It came with three detonators, and the weapons ofﬁcer was sup-

posed to replace green dummy ones with red live ones once the plane was

airborne. However, a number of heavily loaded B-29s had crashed on takeoff

from Tinian, the base that was used. The Enola Gay weapon ofﬁcer, Navy

Captain Deak Parsons, reckoned that if the Enola Gay, crashed, the primer

418 Chapter 13 ■Nuclear Command and Control

might explode, detonating the bomb and wiping out the island. So he spent

the day before the raid practicing removing and reinstalling the primer — a

gunpowder charge about the size of a loaf of bread — so he could install it

after takeoff instead.

Doctrine has rather moved away from improvization of weapon safety

procedures in the ﬁeld. If anything we’re at the other extreme now, with

mechanisms and procedures tested and drilled and exercised and analysed by

multiple experts from different agencies. It has of course been an evolutionary

process. When weapons started being carried in single-seat tactical aircraft

in the 1950s, and also started being slung under the wings rather than in a

bomb bay, it was no longer possible for someone to manually insert a bag

of gunpowder. There was a move to combination locks: the pilot would arm

the bomb after takeoff by entering a 6-digit code into a special keypad with

a wired-seal lid. This enabled some measure of control; the pilot might only

receive the code once airborne. However both the technical and procedural

controls in early strategic systems were primitive.

13.2.1 The Kennedy Memorandum

The Cuban missile crisis changed all that. U.S. policymakers (and many others)

suddenly became very concerned that a world war might start by accident.

Hundreds of U.S. nuclear weapons were kept in allied countries such as Greece

and Turkey, which were not particularly stable and occasionally fought with

each other. These weapons were protected by only token U.S. custodial forces,

so there was no physical reason why the weapons couldn’t be seized in time of

crisis. There was also some worry about possible unauthorized use of nuclear

weapons by U.S. commanders — for example, if a local commander under

pressure felt that ‘if only they knew in Washington how bad things were

here, they would let us use the bomb.’ These worries were conﬁrmed by three

emergency studies carried out by presidential science adviser Jerome Wiesner.

In [1223] we ﬁnd the passage quoted at the head of this chapter.

President Kennedy’s response was National Security Action Memo no. 160

[153]. This ordered that America’s 7,000 nuclear weapons then dispersed to

NATO commands should be got under positive U.S. control using technical

means, whether they were in the custody of U.S. or allied forces. Although

this policy was sold to Congress as protecting U.S. nuclear weapons from

foreigners, the worries about a psychotic ‘Dr Strangelove’ were also real:

they were actually at the top of Wiesner’s list, although of course they were

downplayed politically.

The Department of Energy was already working on safety devices for nuclear

weapons. The basic principle was that a unique aspect of the environment had

to be sensed before the weapon would arm. For example, missile warheads

and some free-fall bombs had to experience zero gravity, while artillery shells

had to experience an acceleration of thousands of G. There was one exception:

13.2 The Evolution of Command and Control 419

atomic demolition munitions. These are designed to be taken to their targets by

ground troops and detonated using time fuses. There appears to be no scope for

a unique environmental sensor to prevent accidental or malicious detonation.

The solution then under development was a secret arming code that acti-

vated a solenoid safe lock buried deep in the plutonium pit at the heart of the

weapon. The main engineering problem was maintenance. When the lock was

exposed, for example to replace the power supply, the code might become

known. So it was not acceptable to have the same code in every weapon.

Group codes were one possibility — ﬁring codes shared by only a small batch

of warheads.

Following the Kennedy memo, it was proposed that all nuclear bombs

should be protected using code locks, and that there should be a ‘universal

unlock’ action message that only the president or his legal successors could

give. The problem was to ﬁnd a way to translate this code securely to a

large number of individual ﬁring codes, each of which enabled a small batch

of weapons. The problem became worse in the 1960s and 1970s when the

doctrine changed from massive retaliation to ‘measured response’. Instead of

arming all nuclear weapons or none, the President now needed to be able to

arm selected batches (such as ‘all nuclear artillery in Germany’). This clearly

starts to lead us to a system of some complexity, especially when we realise

that we need disarming codes too, for maintenance purposes, and that we

need some means of navigating the trade-offs between weapons safety and

effective command.

13.2.2 Authorization, Environment, Intent

So the deep question was the security policy that nuclear safety systems, and

command systems, should enforce. What emerged was the rule of ‘authoriza-

tion, environment, intent’. For a warhead to detonate, three conditions must

be met.

Authorization: the use of the weapon in question must have been authorized

by the national command authority (i.e., the President and his lawful

successors in ofﬁce).

Environment: the weapon must have sensed the appropriate aspect of the

environment. (With atomic demolition munitions, this requirement is

replaced by the use of a special container.)

Intent: the ofﬁcer commanding the aircraft, ship or other unit must unam-

biguously command the weapon’s use.

In early systems, ‘authorization’ meant the entry into the device of a

four-digit authorization code.

The means of signalling ‘intent’ depended on the platform. Aircraft typically

use a six-digit arming or ‘use control’ code. The command consoles for

420 Chapter 13 ■Nuclear Command and Control

intercontinental ballistic missiles are operated by two ofﬁcers, each of whom

must enter and turn a key to launch the rocket. Whatever the implementation,

the common concept is that there must be a unique signal; the effectively 22 bits

derived from a six-digit code is believed to be a good tradeoff between a number

of factors from usability to minimising the risk of accidental arming [908].

13.3 Unconditionally Secure Authentication

Nuclear command and control led to the development of a theory of one-time

authentication codes. These are similar in concept to the test keys which were

invented to protect telegraphic money transfers, in that a keyed transformation

is applied to the message in order to yield a short authentication code, also

known as an authenticator or tag. As the keys are only used once, authentication

codes can be made unconditionally secure. So they do for authentication what

the one-time pad does for conﬁdentiality.

Recall from Chapter 5, ‘Cryptography’, that while a computationally secure

system could be broken by some known computation and depends on this

being too hard, the perfect security provided by the one-time pad is indepen-

dent of the computational resources available to the attacker.

There are differences though between authentication codes and the one-

time pad. As the authentication code is of ﬁnite length, it’s always possible

for the opponent to guess it, and the probability of a successful guess might

be different depending on whether the opponent was trying to guess a valid

message from scratch (impersonation) or modify an existing valid message so

as to get another one (substitution).

An example should make this clear. Suppose a commander has agreed an

authentication scheme with a subordinate under which an instruction is to be

encoded as a three digit number from 000 to 999. The instruction may have

two values: ‘Attack Russia’ and ‘Attack China’. One of these will be encoded

as an even number, and the other by an odd number: which is which will be

part of the secret key. The authenticity of the message will be vouched for by

making its remainder, when divided by 337, equal to a secret number which

is the second part of the key.

Suppose the key is that:

‘Attack Russia’ codes to even numbers, and ‘Attack China’ to odd

an authentic message is one which has the remainder 12 when divided

by 337.

So ‘Attack Russia’ is ‘686’ (or ‘12’) and ‘Attack China’ is ‘349’.

An enemy who has taken over the communications channel between the

commander and the subordinate, and who knows the scheme but not the key,

has a probability of only 1 in 337 of successfully impersonating the commander.

13.3 Unconditionally Secure Authentication 421

However, once he sees a valid message (say ‘12’ for ‘Attack Russia’), then he

can easily change it to the other by adding 337, and so (provided he understood

what it meant) he can send the missiles to the other country. So the probability

of a successful substitution attack in this case is 1.

As with computationally secure authentication, the unconditional variety

can provide message secrecy or not: it might work like a block cipher, or like

a MAC on a plaintext message. Similarly, it can use an arbitrator or not. One

might even want multiple arbitrators, so that they don’t have to be trusted indi-

vidually. If the ﬁrst arbitrator wrongfully ﬁnds in favor of the cheated party,

then a multi-arbitrator scheme lets his victim denounce him. Schemes may

combine unconditional with computational security. For example, an uncondi-

tional code without secrecy could have computationally secure secrecy added

by simply enciphering the message and the authenticator using a conventional

cipher system.

Authentication is in some sense the dual of coding in that in the latter,

given an incorrect message, we want to ﬁnd the nearest correct message

efﬁciently; in the former, we want ﬁnding a correct message to be impossible

unless you’ve seen it already or are authorized to construct it. And just as

the designer of an error-correcting code wants the shortest length of code for

a given error recovery capability, so the designer of an authentication code

wants to minimize the key length required to achieve a given bound on the

deception probabilities.

One application that’s worth noting is the new GCM mode of operation for

block ciphers, described brieﬂy in Chapter 5, ‘Cryptography’. In effect this uses

the user-supplied key to generate an unconditionally-secure authentication

code on the plaintext; it’s just a polynomial function of the key and the

plaintext. Combined with the counter-mode encryption of the plaintext, this

gives an authenticated encryption mode that requires only one pass through

the block cipher, rather than the two passes required for CBC plus MAC.

The authentication terminology used in civil and military applications is

slightly different [1172]. More importantly, the threat models are different.

Soldiers are in general not too worried about non-repudiation — except when

enforcing treaties with other countries, which might later repudiate a message

claiming that the key had been leaked by a ‘defector’. In business, the majority

of frauds are carried out by insiders, so shared control systems are the main

issue when designing authentication mechanisms.

Quite a few more details have to be ﬁxed before you have a fully-functioning

command and control system. You have to work out ways to build the key

control mechanisms into warheads in ways that will resist disarming or

dismantling by people without disarming keys. You need mechanisms for

generating keys and embedding them in weapons and control devices. You

have to think of all the ways an attacker might social-engineer mainte-

nance staff, and what you’ll do to forestall this. And there is one element of

422 Chapter 13 ■Nuclear Command and Control

cryptographic complexity. How do you introduce an element of one-wayness,

so that a maintenance man who disarms a bomb to change the battery doesn’t

end up knowing the universal unlock code? You need to be able to derive

the code to unlock this one speciﬁc device from the universal unlock, but not

vice-versa. What’s more, you need serviceable mechanisms for recovery and

re-keying in the event that a crisis causes you to authorize some weapons,

that thankfully are stood down rather than used. U.S. systems now use public-

key cryptography to implement this one-wayness, but you could also use

one-way functions. In either case, you will end up with an interesting mix of

unconditional and computational security.

13.4 Shared Control Schemes

The nuclear command and control business became even more complex with

the concern, from the late 1970s, that a Soviet decapitation strike against the U.S.

national command authority might leave the arsenal intact but useless. There

was also concern that past a certain threshold of readiness, it wasn’t sensible

to assume that communications between the authority and ﬁeld commanders

could be maintained, because of the damage that electromagnetic pulse could

do (and other possible attacks on communications).

The solution was found in another branch of cryptomathematics known as

secret sharing, whose development it helped to inspire. The idea is that in time of

tension a backup control system will be activated in which combinations

of ofﬁce holders or ﬁeld commanders can jointly allow a weapon to be

armed. Otherwise the problems of maintaining detailed central control of a

large number of weapons would likely become insoluble. There was some

precedent for this in submarine-launched ballistic missiles. These exist in part

to provide a second-strike capability — that is, to take vengeance on a country

that has destroyed your country with a ﬁrst strike. In such circumstances it is

impossible for the submarine commander to be left unable to arm his weapons

unless he gets a code from the President. So arming material is kept in safes

under the control of the boat’s ofﬁcers, along with orders from the command

authority on the circumstances in which weapons are to be used.

Now there is an obvious way to do shared control — just give half of the

authentication key to each of two people. The drawback is that you need twice

the length of key, assuming that the original security parameter must apply

even if one of them is suborned. An alternative approach is to give each of

them a number and have the two of them add up to the key. This is how

keys for automatic teller machines are managed2. But this still may not be

2Combining keys using addition or exclusive-or turns out to be a bad idea for ATMs as it opens

up the system to attacks that I’ll discuss later under the rubric of ‘API security’. However in the

context of unconditionally-secure authentication codes, addition is often OK.

13.4 Shared Control Schemes 423

enough in command applications, as one cannot be sure that the personnel

operating the equipment will consent, without discussion or query, to unleash

Armageddon. So a more general approach was invented independently by

Blakley and Shamir in 1979 [181, 1146]. Their basic idea is illustrated in the

following diagram (Figure 13.1).

G1x

G2x

xG3

Figure 13.1: Shared control using geometry

Suppose the rule Britain wants to enforce if the Prime Minister is assassinated

is that a weapon can be armed either by any two cabinet ministers, or by any

three generals, or by a cabinet minister and two generals. To implement this,

let the point Con the zaxis be the unlock code that has to be supplied to

the weapon. We now draw a line at random through Cand give each cabinet

minister a random point on the line. Now any two of them can together work

out the coordinates of the line and ﬁnd the point Cwhere it meets the z

axis. Similarly, we embed the line in a random plane and give each general a

random point on the plane. Now any three generals, or two generals plus a

minister, can reconstruct the plane and thence the ﬁring code C.

By generalizing this simple construction to geometries of ndimensions, or to

general algebraic structures rather than lines and planes, this technique enables

weapons, commanders and options to be linked together with a complexity

limited only by the available bandwidth. An introduction to secret sharing

can be found in [1226] and a more detailed exposition in [1173]. This inspired

the development of threshold signature schemes, as described in Chapter 5,

‘Cryptography’, and can be used in products that enforce a rule such as ‘Any

two vice-presidents of the company may sign a check’.

424 Chapter 13 ■Nuclear Command and Control

As with authentication codes, there is a difference between civil and military

views of shared secrets. In the typical military application, two-out-of-ncontrol

is used; nmust be large enough that at least two of the keyholders will be ready

and able to do the job, despite combat losses. Many details need attention. For

example, the death of a commander shouldn’t give his deputy both halves of

the key, and there are all sorts of nitty-gritty issues such as who shoots whom

when (on the same side).

In many civilian applications, however, many insiders may conspire to break

your system. The classic example is pay-TV where a pirate may buy several

dozen subscriber cards and reverse engineer them for their secrets. So the

pay-TV operator wants a system that’s robust against multiple compromised

subscribers. I’ll talk about this traitor tracing problem more in the chapter on

13.5 Tamper Resistance and PALs

In modern weapons the solenoid safe locks have been superseded by prescribed

action links, more recently renamed permissive action links (either way, PALs),

which are used to protect most U.S. nuclear devices. A summary of the

published information about PALs can be found in [153]. PAL development

started in about 1961, but deployment was slow. Even twenty years later, about

half the U.S. nuclear warheads in Europe still used four-digit code locks3.As

more complex arming options were introduced, the codes increased in length

from 4 to 6 and ﬁnally to 12 digits. Devices started to have multiple codes, with

separate ‘enable’ and ‘authorize’ commands and also the ability to change

codes in the ﬁeld (to recover from false alarms).

The PAL system is supplemented by various coded switch systems and

operational procedures, and in the case of weapons such as atomic demolition

munitions, which are not complex enough for the PAL to be made inaccessible

in the core of the device, the weapon is also stored in tamper sensing containers

called PAPS (for prescribed action protective system). Other mechanisms used

to prevent accidental detonation include the deliberate weakening of critical

parts of the detonator system, so that they will fail if exposed to certain

abnormal environments.

Whatever combination of systems is used, there are penalty mechanisms to

deny a thief the ability to obtain a nuclear yield from a stolen weapon. These

mechanisms vary from one weapon type to another but include gas bottles

3Bruce Blair says that Strategic Air Command resisted the new doctrine and kept Minuteman

authorization codes at ’00000000’ until 1977, lying to a succession of Presidents and Defense

Secretaries [180]. Other researchers have claimed this was not the authorization code but just the

use control code.

13.5 Tamper Resistance and PALs 425

to deform the pit and hydride the plutonium in it, shaped charges to destroy

components such as neutron generators and the tritium boost, and asymmetric

detonation that results in plutonium dispersal rather than yield. Indeed most

weapons have a self-destruct procedure that will render them permanently

inoperative, without yield, if enemy capture is threatened. It is always a priority

to destroy the code. It is assumed that a renegade government prepared to

deploy ‘terrorists’ to steal a shipment of bombs would be prepared to sacriﬁce

some of the bombs (and some technical personnel) to obtain a single serviceable

weapon.

To perform authorized maintenance, the tamper protection must be dis-

abled, and this requires a separate unlock code. The devices that hold the

various unlock codes — for servicing and ﬁring — are themselves protected

in similar ways to the weapons.

The assurance target is summarized in [1223]:

It is currently believed that even someone who gained possession

of such a weapon, had a set of drawings, and enjoyed the technical

capability of one of the national laboratories would be unable to

successfully cause a detonation without knowing the code.

Meeting such an ambitious goal requires a very substantial effort. There are

several examples of the level of care needed:

after tests showed that 1 mm chip fragments survived the protective det-

onation of a control device carried aboard airborne command posts, the

software was rewritten so that all key material was stored as two sep-

arate components, which were kept at addresses more than 1 mm apart

on the chip surface;

the ‘football’, the command device carried around behind the President,

is said to be as thick as it is because of fear that shaped charges might be

used to disable its protective mechanisms. (This may or may not be an

urban myth.) Shaped charges can generate a plasma jet with a velocity

of 8000m/s, which could in theory be used to disable tamper sensing cir-

cuitry. So some distance may be needed to give the alarm circuit enough

time to zeroize the code memory.

This care must extend to many details of implementation and operation.

The weapons testing process includes not just independent veriﬁcation and

validation, but hostile ‘black hat’ penetration attempts by competing agencies.

Even then, all practical measures are taken to prevent access by possible

opponents. The devices (both munition and control) are defended in depth by

armed forces; there are frequent zero-notice challenge inspections; and staff

may be made to re-sit the relevant examinations at any time of the day or

night.

426 Chapter 13 ■Nuclear Command and Control

I’ll discuss tamper resistance in much more detail in a later chapter, as

it’s becoming rather widely used in applications from pay-TV to bank cards.

However, tamper resistance, secret sharing and one-time authenticators aren’t

the only technologies to have beneﬁtted from the nuclear industry’s interest.

There are more subtle system lessons too.

13.6 Treaty Veriﬁcation

A variety of veriﬁcation systems are used to monitor compliance with nuclear

nonproliferation treaties. For example, the IAEA and the U.S. Nuclear Regu-

latory Commission (NRC) monitor ﬁssile materials in licensed civilian power

reactors and other facilities.

An interesting example comes from the tamper resistant seismic sensor

devices designed to monitor the Comprehensive Test Ban Treaty [1170]. The

goal in this application was to have sufﬁciently sensitive sensors emplaced in

each signatory’s test sites that any violation of the treaty (such as by testing

too large a device) can be detected with high probability. The tamper sensing

here is fairly straightforward: the seismic sensors are ﬁtted in a steel tube and

inserted into a drill hole that is backﬁlled with concrete. The whole assembly

is so solid that the seismometers themselves can be relied upon to detect

tampering events with a fairly high probability. This physical protection is

reinforced by random challenge inspections.

The authentication process becomes somewhat more complex because one

has to make an assumption of pervasive deceit. Because of the lack of a third

party trusted by both sides, and because the quantity of seismic data being

transmitted is of the order of 108bits per day, a digital signature scheme (RSA)

was used instead of one-time authentication tags. But this is only part of the

answer. One party might, for example, disavow a signed message by saying

that the ofﬁcial responsible for generating it had defected, and so the signature

was forged. So it is necessary for keys to be generated within the seismic

package itself once it has been sealed by both sides. Also, if one side builds the

equipment, the other will suspect it of having hidden functionality. Several

protocols were proposed of the cut and choose variety, in which one party would

produce several devices of which the other party would dismantle a sample

for inspection. A number of these issues have since resurfaced in electronic

commerce. (Many system designers since could have saved themselves a lot of

grief if they’d read Gus Simmons’ account of these treaty monitoring systems

in [1170].)

13.7 What Goes Wrong 427

13.7 What Goes Wrong

Despite the huge amounts of money invested in developing high-tech pro-

tection mechanisms, nuclear control and safety systems appear to suffer from

just the same kind of design bugs, implementation blunders and careless

operations as any others.

Britain’s main waste reprocessing plant at Sellaﬁeld, which handles pluto-

nium in multiple-ton quantities, has been plagued with a series of scandals.

Waste documentation has been forged; radiation leaks have been covered up;

workers altered entry passes so they could bring their cars into restricted

areas; and there have been reports of sabotage. The nuclear police force only

managed to clear up 17 out of 158 thefts and 3 out of 20 cases of criminal

damage [776]. The situation in the former Soviet Union appears to be very

much worse. A survey of nuclear safekeeping describes how dilapidated their

security mechanisms have become following the collapse of the USSR, with ﬁs-

sile materials occasionally appearing on the black market and whistleblowers

being prosecuted [644].

There are also a number of problems relating to the reliability of communi-

cations and other systems under attack. How can communication between the

President and many sites round the world be assured? I’ll discuss these later

in the chapter on ‘Electronic and Information Warfare’.

There have also been a number of interesting high-tech security failures.

One example is a possible attack discovered on a nuclear arms reduction treaty

which led to the development of a new branch of cryptomathematics — the

study of subliminal channels — and is relevant to later discussions of copyright

marking and steganography.

The story is told in [1176]. During the Carter administration, the USA

proposed a deal with the USSR under which each side would cooperate with

the other to verify the number of intercontinental ballistic missiles. In order to

protect U.S. Minuteman missiles against a possible Soviet ﬁrst strike, it was

proposed that 100 missiles be moved randomly around a ﬁeld of 1000 silos

by giant trucks, which were designed so that observers couldn’t determine

whether they were moving a missile or not. So the Soviets would have had to

destroy all 1,000 silos to make a successful ﬁrst strike, and in the context of the

proposed arms controls this was thought impractical.

This raised the interesting problem of how to assure the Soviets that there

were at most 100 missiles in the silo ﬁeld, but without letting them ﬁnd out

which silos were occupied. The proposed solution was that the silos would

have a Russian sensor package that would detect the presence or absence of a

428 Chapter 13 ■Nuclear Command and Control

missile, sign this single bit of information, and send it via a U.S. monitoring

facility to Moscow. The sensors would be packaged and randomly shufﬂed

by the USA before emplacement, so that the Russians could not correlate ‘full’

or ‘empty’ signals with particular silos. The catch was that only this single

bit of information could be sent; if the Russians could smuggle any more

information into the message, they could quickly locate the full silos — as it

would take only ten bits of address information to specify a single silo in the

ﬁeld. (There were many other security requirements to prevent either side

cheating, or falsely accusing the other of cheating: for more details, see [1175].)

To see how subliminal channels work, consider the Digital Signature Algo-

rithm described in the chapter on cryptography. The system-wide values are a

prime number p, a prime number qdividing p−1, and a generator gof a

subgroup of F∗

pof order q. The signature on the message Mis r,swhere r=(gk

(mod p)) (mod q), and kis a random session key. The mapping from kto

ris fairly random, so a signer who wishes to hide ten bits of information

in this signature for covert transmission to an accomplice can ﬁrstly agree

a convention about how the bits will be hidden (such as ‘bits 72–81’) and

secondly, try out one value of kafter another until the resulting value rhas the

desired value in the agreed place.

This could have caused a disastrous failure of the security protocol as there

had been an agreement that the monitoring messages would be authenticated

ﬁrst with a Russian scheme, using Russian equipment, and then by an Ameri-

can scheme using American equipment. Had the Russians speciﬁed a signature

scheme like DSA then they could have leaked the location of the occupied

silos and acquired the capability to make a ﬁrst strike against the Minuteman

force.

In the end, the ‘missile shell game’, as it had become known in the popular

press, wasn’t used. The cooling of relations following the 1980 election put

things on hold. Eventually with the medium range ballistic missile treaty

(MRBM) statistical methods were used. The Russians could say ‘we’d like

to look at the following 20 silos’ and they would be uncapped for the Soviet

satellites to take a look. With the end of the Cold War, inspections have become

much more intimate with inspection ﬂights in manned aircraft, with observers

from both sides, rather than satellites.

Still, the discovery of subliminal channels was signiﬁcant. Ways in which

they might be abused include putting HIV status, or the fact of a felony

conviction, into a digital passport or identity card. Where this is unacceptable,

and the card issuer isn’t sufﬁciently trusted not to do it, then the remedy is to

use a completely deterministic signature scheme such as RSA instead of one

that uses a random session key like DSA.

13.8 Secrecy or Openness? 429

13.8 Secrecy or Openness?

Finally, the nuclear industry provides a nice case history of secrecy. In the

1930s, physicists from many countries had freely shared the scientiﬁc ideas

that led to the development of the bomb, but after the ‘atomic spies’ (Fuchs,

the Rosenbergs and others) had leaked the designs of the Hiroshima and

Nagasaki devices to the Soviet Union, things swung to the other extreme. The

U.S. adopted a policy that atomic knowledge was born classiﬁed. That meant

that if you were within U.S. jurisdiction and had an idea relevant to nuclear

weapons, you had to keep it secret regardless of whether you held a security

clearance or even worked in the nuclear industry. This was clearly in tension

with the Constitution. Things have greatly relaxed since then, as the protection

issues were thought through in detail.

‘We’ve a database in New Mexico that records the physical and chemical

properties of plutonium at very high temperatures and pressures’, a former

head of U.S. nuclear security once told me. ‘At what level should I classify

that? Who’s going to steal it, and will it do them any good? The Russians,

they’ve got that data for themselves. The Israelis can ﬁgure it out. Gaddaﬁ?

What the hell will he do with it?’

As issues like this got worked though, a surprising amount of the technology

has been declassiﬁed and sometimes published, at least in outline. Starting

from early publication at scientiﬁc conferences of results on authentication

codes and subliminal channels in the early 1980s, the beneﬁts of public design

review have been found to outweigh the possible advantage to an opponent

of knowing broadly the system in use.

Many implementation details are kept secret, though; information that

could facilitate sabotage, such as which of a facility’s ﬁfty buildings contains

the alarm response force, gets marked unclassiﬁed controlled nuclear information

(UCNI) adding yet another layer of complexity to the security policy model.

Yet the big picture is open (or so we’re assured), and command and control

technologies used to be explicitly offered to other states, including hostile

ones like the USSR. The beneﬁts of reducing the likelihood of an accidental

war were considered to outweigh the possible beneﬁts of secrecy. Post-9/11,

it’s clear that we’d rather have decent nuclear command and control systems

in Pakistan rather than risk having one of their weapons used against us

by some mid-level ofﬁcer suffering from an attack of religious zealotry. This

is a modern reincarnation of Kerckhoffs’ doctrine, ﬁrst put forward in the

nineteenth century, that the security of a system must depend on its key, not

on its design remaining obscure [713].

430 Chapter 13 ■Nuclear Command and Control

Indeed, the nuclear lessons should be learned more widely. Post-9/11, a

number of governments (including those of the UK and the European Union)

are talking up the possibility of terrorists using biological weapons, and

imposing various controls on research and teaching in bacteriology, virology,

toxicology and indeed medicine. My faculty colleagues in these disciplines

are deeply unimpressed. ‘You just shouldn’t worry about anthrax’, one of the

UK’s top virologists told me. ‘The real nasties are the things Mother Nature

dreams up like HIV and SARS and bird ﬂu. If these policies mean that there

aren’t any capable public health people in Khartoum next time a virus comes

down the Nile, we’ll be sorry’.

13.9 Summary

The control of nuclear weapons, and subsidiary activities from protecting the

integrity of the national command system through physical security of nuclear

facilities to monitoring international arms control treaties, has made a huge

contribution to the development of security technology.

The rational decision that weapons and ﬁssile material had to be protected

almost regardless of the cost drove the development of a lot of mathematics and

science that has found application elsewhere. The particular examples we’ve

looked at in this chapter are authentication codes, shared control schemes and

subliminal channels. There are other examples scattered through the rest of

this book, from alarms to iris biometrics and from tamper-resistant electronic

devices to seals.

Research Problems

The research problem I set at the end of this chapter in the ﬁrst edition in 2001

was ‘Find interesting applications for technologies developed in this area, such

as authentication codes.’ The recently standardised Galois Counter mode of

operation is a pretty good response to that challenge. What else might there be?

Further Reading

As my own experience of this subject is rather indirect, being limited to work-

ing in the 1970s on the avionics of nuclear-capable aircraft, this chapter has

been assembled from published sources. One of the best sources of public infor-

mation on nuclear weapons is the Federation of American Scientists [460]. The

rationale for the recent declassiﬁcation of many nuclear arms technologies is

Further Reading

The deﬁnitive textbook on security printing is van Renesse [1279] which

goes into not just the technical tricks such as holograms and kinegrams, but

how they work in a variety of applications from banknote printing through

passports to packaging. This is very important background reading.

The essential writing on seals can be found in the many publications by

Roger Johnston’s seal vulnerability assessment team at Los Alamos National

Laboratory (e.g., [668]).

The history of counterfeiting is fascinating. From Independence to the

Civil War, Americans used banknotes issued by private banks rather than by

the government, and counterfeiting was pervasive. Banks could act against

local forgers, but by about 1800 there had arisen a network of engravers,

papermakers, printers, wholesalers, retailers and passers, with safe havens in

the badlands on the border between Vermont and Canada; neither the U.S.

nor the Canadian government wanted to take ownership of the problem [887].

It was in many ways reminiscent of the current struggle against phishing.

CHAPTER

Biometrics

And the Gileadites took the passages of Jordan before the Ephraimites: and it was

so, that when those Ephraimites which were escaped said, Let me go over; that the

men of Gilead said unto him, Art thou an Ephraimite? If he said, Nay; Then said they

unto him, Say now Shibboleth: and he said Sibboleth: for he could not frame to

pronounce it right. Then they took him, and slew him at the passages of the Jordan:

and there fell at that time of the Ephraimites forty and two thousand.

— Judges 12:5–6

15.1 Introduction

The above quotation may be the ﬁrst recorded military use of a security protocol

in which the authentication relies on a property of the human being — in this

case his accent. (There had been less formal uses before this, as when Isaac

tried to identify Esau by his bodily hair but got deceived by Jacob, or indeed

when people recognized each other by their faces — which I’ll discuss later.)

Biometrics identify people by measuring some aspect of individual anatomy

or physiology (such as your hand geometry or ﬁngerprint), some deeply

ingrained skill or behavior (such as your handwritten signature), or some

combination of the two (such as your voice).

Over the last quarter century or so, people have developed a large number

of biometric devices. Since 9/11 the market has really taken off, with a

number of large-scale programs including the international standards for

biometric travel documents, the US-VISIT program which ﬁngerprints visitors

to the USA, Europe’s Schengen visa, assorted ID card initiatives, and various

registered traveler programs. Some large systems already existed, such as

the FBI’s ﬁngerprint database, which is now being expanded to contain a

457

458 Chapter 15 ■Biometrics

range of biometric data for both identiﬁcation and forensic purposes. The

Biometric systems market was reportedly worth over $1.5bn in 2005 [675], a

massive increase from $50 m in 1998 [655]. I already mentioned the use of

hand geometry to identify staff at a nuclear reactor in the late 1970s. But the

best established biometric techniques predate the computer age altogether —

namely the use of handwritten signatures, facial features and ﬁngerprints. I

will look at these ﬁrst, then go on to the fancier, more ‘high-tech’ techniques.

15.2 Handwritten Signatures

Handwritten signatures had been used in classical China, but carved personal

seals came to be considered higher status; they are still used for serious

transactions in China, Japan and Korea. Europe was the other way round: seals

had been used in medieval times, but as writing spread after the Renaissance

people increasingly just wrote their names to signify assent to documents.

Over time the signature became accepted as the standard. Every day, billions

of dollars’ worth of contracts are concluded by handwritten signatures on

documents; how these will be replaced by electronic mechanisms remains a

hot policy and technology issue.

Handwritten signatures are a very weak authentication mechanism by

themselves (in that they’re easy to forge) but have worked well for centuries

because of the context of their use. An important factor is the liability for

forgery. UK law provides that a forged handwritten signature is completely

null and void, and this has survived in the laws of many countries that were

part of the British Empire at the time. It means that the risk from a forged

signature falls on the party who relies on it, and it’s not possible for a bank

to use its standard terms and conditions to dump the risk on the customer.

So manuscript signatures are better for the customer, while the PINs and

electronic tokens that are now replacing them can be better for the bank.

This is not the case everywhere; some Swiss banks make customers liable for

forged cheques. In the USA, Regulation E makes banks liable for the electronic

systems they deploy, so the introduction of electronics doesn’t change the

game much. Needless to say, European banks have moved much further than

U.S. banks in moving customers away from handwritten signatures.

Now the probability that a forged signature will be accepted as genuine

mainly depends on the amount of care taken when examining it. Many

bank card transactions in stores are accepted without even a glance at the

specimen signature on the card — so much so that many Americans do not

even bother to sign their credit cards1. But even diligent signature checking

1Indeed it’s not in the cardholder’s interest to give a specimen signature to a thief — if the thief

makes a random signature on a voucher, it’s easier for the real cardholder to disown it. Signing

the card is in the bank’s interest but not the customer’s.

15.2 Handwritten Signatures 459

doesn’t reduce the risk of fraud to zero. An experiment showed that 105

professional document examiners, who each did 144 pairwise comparisons,

misattributed 6.5% of documents. Meanwhile, a control group of 34 untrained

people of the same educational level got it wrong 38.3% of the time [682],

and the nonprofessionals’ performance couldn’t be improved by giving them

monetary incentives [683]. Errors made by professionals are a subject of

continuing discussion in the industry but are thought to reﬂect the examiner’s

preconceptions [137] and context [403]. As the participants in these tests were

given reasonable handwriting samples rather than just a signature, it seems

fair to assume that the results for verifying signatures on checks or credit card

vouchers would be even worse.

So handwritten signatures are surrounded by a number of conventions

and special rules that vary from one country to another, and these extend

well beyond banking. For example, to buy a house in England using money

borrowed from a bank of which you’re not an established customer, the

procedure is to go to a lawyer’s ofﬁce with a document such as a passport,

sign the property transfer and loan contract, and get the contract countersigned

by the lawyer. The requirement for government issued photo-ID is imposed

by the mortgage lender to keep its insurers happy, while the requirement that

a purchase of real estate be in writing was imposed by the government some

centuries ago in order to collect tax on property transactions. Other types of

document (such as expert testimony) may have to be notarized in particular

ways. Many curious anomalies go back to the nineteenth century, and the

invention of the typewriter. Some countries require that machine written

contracts be initialled on each page, while some don’t, and these differences

have sometimes persisted for over a century. Clashes in conventions still cause

serious problems. In one case, a real estate transaction in Spain was held to be

invalid because the deal had been concluded by fax, and a UK company went

bust as a result.

In most of the English speaking world, however, most documents do not

need to be authenticated by special measures. The essence of a signature is

the intent of the signer, so an illiterate’s ‘X’ on a document is just as valid

as the ﬂourish of an educated man. In fact, a plaintext name at the bottom of

an email message also has just as much legal force [1358], except where there

are speciﬁc regulations to the contrary. There may be many obscure signature

regulations scattered through each country’s laws.

It’s actually very rare for signatures to be disputed in court cases, as the

context mostly makes it clear who did what. So we have a very weak biometric

mechanism that works fairly well in practice — except that it’s choked by

procedural rules and liability traps that vary by country and by application.

Sorting out this mess, and imposing reasonably uniform rules for electronic

documents, is a subject of much international activity. A summary of the issues

can be found in [1359], with an analysis by country in [109]. I’ll discuss some

460 Chapter 15 ■Biometrics

of the issues further in Part III. Meanwhile, note that the form of a signature,

the ease with which it can be forged, and whether it has legal validity in a

given context, are largely independent questions.

There is one application where better automatic recognition of handwritten

signatures could be valuable. This is check clearing.

A bank’s check processing center will typically only verify signatures on

checks over a certain amount — perhaps $1,000, perhaps $10,000, perhaps a

percentage of the last three months’ movement on the account. The signature

veriﬁcation is done by an operator who is simultaneously presented on screen

with the check image and the customer’s reference signature. Verifying checks

for small amounts is not economic unless it could be automated.

So a number of researchers have worked on systems to compare handwritten

signatures automatically. This turns out to be a very difﬁcult image processing

task because of the variability between one genuine signature and another. A

much easier option is to use a signature tablet. This is a sensor surface on which

the user does a signature; it records not just the shape of the curve but also

its dynamics (the velocity of the hand, where the pen was lifted off the paper,

and so on). Tablets are used by delivery drivers to collect receipts for goods;

there have been products since the early 1990s that will compare captured

signatures against specimens enrolled previously.

Like alarm systems, most biometric systems have a trade-off between false

accept and false reject rates, often referred to in the banking industry as the

fraud and insult rates and in the biometric literature as type 1 and type 2 errors.

Many systems can be tuned to favor one over the other. The trade-off is known

as the receiver operating characteristic, a term ﬁrst used by radar operators; if you

turn up the gain on your radar set too high, you can’t see the target for clutter,

while if it’s too low you can’t see it at all. It’s up to the operator to select a

suitable point on this curve. The equal error rate is when the system is tuned so

that the probabilities of false accept and false reject are equal. For tablet-based

signature recognition systems, the equal error rate is at best 1%; for purely

optical comparison it’s several percent. This is not fatal in an operation such

as a check processing center, as the automated comparison is used as a ﬁlter

to preselect dubious checks for scrutiny by a human operator. However, it

is a show-stopper in a customer-facing application such as a retail store. If

one transaction in a hundred fails, the aggravation to customers would be

unacceptable. So UK banks set a target for biometrics of a fraud rate of 1% and

an insult rate of 0.01%, which is beyond the current state of the art in signature

veriﬁcation and indeed ﬁngerprint scanning [500].

What can be done to bridge the gap? An interesting experiment was

conducted by the University of Kent, England, to cut fraud by welfare claimants

who were drawing their beneﬁts at a post ofﬁce near Southampton. The novel

feature of this system is that, just as in a check processing center, it was used to

screen signatures and support human decisions rather than to take decisions

15.3 Face Recognition 461

itself. So instead of being tuned for a low insult rate, with a correspondingly

high fraud rate, it had fraud and insult rates approximately equal. When a

signature is rejected, this merely tells the staff to look more closely, and ask

for a driver’s license or other photo-ID. With 8500 samples taken from 343

customers, 98.2% were veriﬁed correctly at the ﬁrst attempt, rising to 99.15%

after three attempts [452]. But this rate was achieved by excluding goats —a

term used by the biometric community for people whose templates don’t

classify well. With them included, the false reject rate was 6.9% [453]. Because

of this disappointing performance, sales of signature recognition technology

are only 1.7% of the total biometric market; automation has cost it its leadership

of the biometric market.

In general, biometric mechanisms tend to be much more robust in attended

operations where they assist a guard rather than replacing him. The false

alarm rate may then actually help by keeping the guard alert.

15.3 Face Recognition

Recognizing people by their facial features is the oldest identiﬁcation mech-

anism of all, going back at least to our early primate ancestors. Biologists

believe that a signiﬁcant part of our cognitive function evolved to provide efﬁ-

cient ways of recognizing other people’s facial features and expressions [1076].

For example, we are extremely good at detecting whether another person is

looking at us or not. In normal social applications, humans’ ability to identify

people by their faces appears to be very much better than any automatic

facial-recognition system produced to date.

The human ability to recognize faces is important to the security engineer

because of the widespread reliance placed on photo ID. Drivers’ licenses,

passports and other kinds of identity card are not only used to control entry to

computer rooms directly, they are also used to bootstrap most other systems.

The issue of a password, or a smartcard, or the registration of a user for a

biometric system using some other technique such as iris recognition, is often

the end point of a process which was started by that person presenting photo

ID when applying for a job, opening a bank account or whatever.

But even if we are good at recognising friends in the ﬂesh, how good are we

at identifying strangers by photo ID?

The simple answer is that we’re not. Psychologists at the University of West-

minster conducted a fascinating experiment with the help of a supermarket

chain and a bank [705]. They recruited 44 students and issued each of them

with four credit cards each with a different photograph on it:

one of the photos was a ‘good, good’ one. It was genuine and recent;

the second was a ‘bad, good one’. It was genuine but a bit old, and

the student now had different clothing, hairstyle or whatever. In other

462 Chapter 15 ■Biometrics

words, it was typical of the photo that most people have on their

photo ID;

the third was a ‘good, bad one’. From a pile of a hundred or so random

photographs of different people, investigators chose the one which most

looked like the subject. In other words, it was typical of the match that

criminals could get if they had a stack of stolen cards;

the fourth was a ‘bad, bad’ one. It was chosen at random except that

it had the same sex and race as the subject. In other words, it was typ-

ical of the match that really lazy, careless criminals would get.

The experiment was conducted in a supermarket after normal business

hours, but with experienced cashiers on duty, and aware of the purpose of the

experiment. Each student made several trips past the checkout using different

cards. It transpired that none of the checkout staff could tell the difference

between ‘good, bad’ photos and ‘bad, good’ photos. In fact, some of them

could not even tell the difference between ‘good, good’ and ‘bad, bad’. Now

this experiment was done under optimum conditions, with experienced staff,

plenty of time, and no threat of embarrassment or violence if a card was

rejected. Real life performance can be expected to be worse. In fact, many

stores do not pass on to their checkout staff the reward offered by credit

card companies for capturing stolen cards. So even the most basic incentive

is absent.

The response of the banking industry to this experiment was ambivalent. At

least two banks who had experimented with photos on credit cards had expe-

rienced a substantial drop in fraud — to less than one percent of the expected

amount in the case of one Scottish bank [107]. The overall conclusion was that

the beneﬁt to be had from photo ID is essentially its deterrent effect [471].

So maybe people won’t use their facial recognition skills effectively in iden-

tiﬁcation contexts, or maybe the information we use to identify people in social

contexts is stored differently in our brains from information we get by looking

at a single photo. (Recognising passing strangers is in any case much harder

than recognising people you know. It’s reckoned that misidentiﬁcations are

the main cause of false imprisonment, with 20% of witnesses making mistakes

in identity parades [1360] — not as bad as the near-random outcomes when

comparing faces with photos, but still not good.)

But in any case, photo-ID doesn’t seem to work, and this is one of the

reasons for trying to automate the process. Attempts go back to the nineteenth

century, when Francis Galton devised a series of spring-loaded ‘mechani-

cal selectors’ for facial measurements [510]. But automated face recognition

actually subsumes a number of separate problems, and in most of them we

don’t have the luxury of taking careful 3-d measurements of the subject. In a

typical identity veriﬁcation application, the subject looks straight at the camera

under controlled lighting conditions, and his face is compared with the one

15.3 Face Recognition 463

on ﬁle. A related but harder problem is found in forensics, where we may be

trying to establish whether a suspect’s face ﬁts a low-quality recording on a

security video. The hardest of all is surveillance, where the goal may be to

scan a moving crowd of people at an airport and try to pick out anyone who

is on a list of thousands of known suspects. Yet automatic face recognition

was one of the technologies most hyped by the security-industrial complex

after 9/11 [1084].

Even picking out faces from an image of a crowd is a non-trivial compu-

tational task [798]. An academic study of the robustness of different facial

feature extraction methods found that given reasonable variations in lighting,

viewpoint and expression, no method was sufﬁcient by itself and error rates

were up to 20% [13]. Systems that use a combination of techniques can get

the error rate down but not to the levels possible with many other biomet-

rics [898, 1370]. Field trials by the U.S. Department of Defense in 2002 found

that a leading face-recognition product correctly recognized one individual

out of 270 only 51% of the time, and identiﬁed one person correctly to within

a range of 10 participants 81% of the time [852]. (The vendor in question had

put out a press release on the afternoon of September 11th and seen a huge

rise in its stock price in the week after trading resumed [782].) By 2003, the

technology had improved somewhat, with one vendor recognising 64% of

subjects against a database of over 30,000, although performance outdoors

was poorer. Tests done in 2001 by the UK National Physical Laboratory (NPL)

of a number of biometric technologies found that face recognition was almost

the worst, outperforming only vein patterns; its single-attempt equal-error

rate was almost ten percent [834]. A UK Passport Ofﬁce trial in 2005, that was

a better approximation to ﬁeld conditions, found it recognised only 69% of

users (though this fell to 48% for disabled participants) [1274].

So the technology still does not work very well in engineering terms. But

there are applications where it can have an effect. For example, the Illinois

Department of Motor Vehicles uses it to detect people who apply for extra

drivers’ licenses in false names [454]. Where wrongdoers can be punished, it

may be worthwhile to try to detect them even if you only catch a quarter of

them (that’s still better than the 8% or so of house burglars we catch).

Face recognition has also been used as what Bruce Schneier calls ‘security

theater’. In 1998, the London borough of Newham placed video cameras

prominently in the high street and ran a PR campaign about how their

new computer system constantly scanned the faces in the crowd for several

hundred known local criminals. They managed to get a signiﬁcant reduction

in burglary, shoplifting and street crime. The system even worries civil

libertarians — but it worked entirely by the placebo effect [1227]. The police

have since admitted that they only ever had 20 or 25 villains’ faces on the

system, and it never recognised any of them [871]. In Tampa, Florida, a

similar system was abandoned after an ACLU freedom of information request

464 Chapter 15 ■Biometrics

discovered that it had recognised no villains [1072]. The ACLU welcomed

its demise, remarking that ‘every person who walked down the street was

subjected to an electronic police line-up without their consent’. (Given that the

technology just didn’t work, this was maybe a tad zealous.) Face recognition

was also tried at Boston’s Logan airport; passengers passing through security

screening were observed and matched. The system was found to be impractical,

with no useful balance between false matches and false alarms [222].

Yet facial recognition is already the second largest-selling biometric with a

nominal 19% of the market. However, much of this relates to the automated

storage of facial images that are compared by humans — for example, the

photos stored in the chips on the new biometric passports. The market for

automated recognition is much smaller. Maybe as time passes and technology

improves, both its potential (and the privacy worries) will increase.

15.4 Bertillonage

Inventors in the nineteenth century spent quite a lot of effort trying to identify

people by their bodily measurements. The most famous of these, Alphonse

Bertillon, started out as a clerk in the police records department in Paris, where

an important task was to identify serial offenders. In 1882 he published a

system based on bodily measurements, such as height standing and sitting,

the length and width of the face, and the size and angle of the ear. These were

principally used to index a collection of record cards that also held mugshots

and thumbprints, which could be used to conﬁrm an identiﬁcation. This

system was known as ‘anthropometry’, and also as ‘Bertillonage’ in honour of

its creator. Eventually it fell out of favour, once police forces understood how

to index and search for ﬁngerprints.

This technique has made a comeback in the form of hand-geometry readers.

In addition to its use since the 1970s in nuclear premises entry control, hand

geometry is now used at airports by the U.S. Immigration and Naturalization

Service to provide a ‘fast track’ for frequent ﬂyers. It is simple to implement

and fairly robust, and the NPL trials found a single-attempt equal error rate of

about one percent [834]. (Passport inspection is a less critical application than

one might initially think, as airline staff also check passports against passenger

lists and provide these lists to the homeland security folks.) Hand geometry is

now reported to have 8.8% of the biometric market.

15.5 Fingerprints

Automatic ﬁngerprint identiﬁcation systems (AFIS) are by far the biggest

single technology. In 1998, AFIS products accounted for a whopping 78%

15.5 Fingerprints 465

of the $50 m sales of biometric technology; the huge growth of the industry

since then has cut this in percentage terms to 43.5% of $1,539m by 2005, but

it leads all other automated recognition options. AFIS products look at the

friction ridges that cover the ﬁngertips and classify patterns of minutiae such

as branches and end points of the ridges. Some also look at the pores in the

skin of the ridges. A recent technical reference book on automatic ﬁngerprint

identiﬁcation systems is [832].

The use of ﬁngerprints to identify people was discovered independently

a number of times. Mark Twain mentions thumbprints in 1883 in Life on the

Mississippi where he claims to have learned about them from an old Frenchman

who had been a prison-keeper; his 1894 novel Pudd’nhead Wilson made the idea

popular in the States. Long before that, ﬁngerprints were accepted in a seventh

century Chinese legal code as an alternative to a seal or a signature, and

required by an eighth century Japanese code when an illiterate man wished

to divorce his wife. They were also used in India centuries ago. Following

the invention of the microscope, they were mentioned by the English botanist

Nathaniel Grew in 1684, by Marcello Malpighi in Italy in 1686; in 1691, 225

citizens of Londonderry in Ireland used their ﬁngerprints to sign a petition

asking for reparations following the siege of the city by King William.

The ﬁrst modern systematic use was in India from 1858, by William

Herschel, grandson of the astronomer and a colonial magistrate. He intro-

duced handprints and then ﬁngerprints to sign contracts, stop impersonation

of pensioners who had died, and prevent rich criminals paying poor people

to serve their jail sentences for them. Henry Faulds, a medical missionary

in Japan, discovered them independently in the 1870s, and came up with

the idea of using latent prints from crime scenes to identify criminals. Faulds

brought ﬁngerprints to the attention of Charles Darwin, who in turn motivated

Francis Galton to study them. Galton wrote an article in Nature [510]; this got

him in touch with the retired Herschel, whose data convinced Galton that

ﬁngerprints persisted throughout a person’s life. Galton went on to collect

many more prints and devise a scheme for classifying their patterns [511]. The

Indian history is told by Chandak Sengoopta, whose book also makes the point

that ﬁngerprinting saved two somewhat questionable Imperial institutions,

namely the indentured labor system and the opium trade [1145].

The practical introduction of the technique owes a lot to Sir Edward

Henry, who had been a policeman in Bengal. He wrote a book in 1900

describing a simpler and more robust classiﬁcation, of loops,whorls,arches

and tents, that he had developed with his assistants Azizul Haque and

Hem Chandra Bose, and that is still in use today. In the same year he

became Commissioner of the Metropilitan Police in London from where the

466 Chapter 15 ■Biometrics

technique spread round the world2. Henry’s real scientiﬁc contribution was

to develop Galton’s classiﬁcation into an indexing system. By assigning one

bit to whether or not each of a suspect’s ten ﬁngers had a whorl — a type of

circular pattern — he divided the ﬁngerprint ﬁles into 1024 bins. In this way,

it was possible to reduce the number of records that have to be searched by

orders of magnitude. Meanwhile, as Britain had stopped sending convicted

felons to Australia, there was a perceived need to identify previous offenders,

so that they could be given longer jail sentences.

Fingerprints are now used by the world’s police forces for essentially two

different purposes: identifying people (the main use in the USA), and crime

scene forensics (their main use in Europe).

I’ll now look at these two technologies in turn.

15.5.1 Verifying Positive or Negative Identity Claims

In America nowadays — as in nineteenth-century England — quite a few

criminals change their names and move somewhere new on release from

prison. This is ﬁne when offenders go straight, but what about fugitives

and recidivists? American police forces have historically used ﬁngerprints

to identify arrested suspects to determine whether they’re currently wanted

by other agencies, whether they have criminal records and whether they’ve

previously come to attention under other names. The FBI maintains a large

online system for this purpose; it identiﬁes about eight thousand fugitives

a month [1208]. It is also used to screen job applicants; for example, anyone

wanting a U.S. government clearance at Secret or above must have an FBI

ﬁngerprint check, and checks are also run on some people applying to work

with children or the elderly. Up to 100,000 ﬁngerprint checks are made a

day, and 900,000 federal, local and state law enforcement ofﬁcers have access.

There’s now a project to expand this to contain other biometrics, to hold data

on foreign nationals, and to provide a ‘rap-back’ service that will alert the

employer of anyone with a clearance who gets into trouble with the law — all

of which disturbs civil-rights groups [927]. Since 9/11, ﬁngerprints are also

used in immigration. The US-VISIT program ﬁngerprints all aliens arriving at

U.S. ports and matches them against a watch list of bad guys, compiled with

the help of other police forces and intelligence services worldwide.

These are examples of one type of identity veriﬁcation — checking an

(implicit) claim not to be on a blacklist. The other type is where the system

2In the Spanish version of history, they were ﬁrst used in Argentina where they secured a murder

conviction in 1892; while Cuba, which set up its ﬁngerprint bureau in 1907, beat the USA whose

ﬁrst conviction was in Illinois in 1911. The Croation version notes that the Argentinian system

was developed by one Juan Vucetich, who had emigrated from Dalmatia. The German version

refers to Professor Purkinje of Breslau, who wrote about ﬁngerprints in 1828. Success truly has

many fathers!

15.5 Fingerprints 467

checks a claim to have a certain known identity. Fingerprints are used for this

purpose in the USA for building entry control and welfare payment [405]; and

banks use them to identify customers in countries such as India and Saudi

Arabia, where the use of ink ﬁngerprints was already common thanks to high

levels of illiteracy.

Fingerprints have not really taken off in banking systems in North America

or Europe because of the association with crime, though a few U.S. banks do

ask for ﬁngerprints if you cash a check there and are not a customer. They ﬁnd

this cuts check fraud by about a half. Some have gone as far as ﬁngerprinting

new customers, and found that customer resistance is less than expected, espe-

cially if they use scanners rather than ink and paper [497]. These applications

are not routine identity veriﬁcation, though, so much as an attempt to identify

customers who later turn out to be bad — another example being the large

British van-hire company that demands a thumbprint when you rent a van. If

the vehicle isn’t returned, or if it’s used in a crime and then turns out to have

been rented with a stolen credit card, the thumbprint is given to the police.

They are thus really a ‘crime scene forensics’ application, which I’ll discuss in

the following section.

So how good are automatic ﬁngerprint identiﬁcation systems? A good rule

of thumb (if one might call it that) is that to verify a claim to identity, it may

be enough to scan a single ﬁnger, while to check someone against a blacklist

of millions of felons, you had better scan all ten. In fact, the US-VISIT program

set out to scan just the two index ﬁngers of each arriving visitor, and has

been overwhelmed by false matches. With 6,000,000 bad guys on the database,

the false match rate in 2004 was 0.31% and the missed match rate 4% [1347].

Although these numbers could be improved somewhat by using the best

algorithms we have now in 2007, the program is now moving to ‘10-prints’, as

they’re called, where each visitor will present the four ﬁngers of each hand,

and then both thumbs, in three successive scans.

This is all about the trade-off between false negatives and false positives

— the receiver operating characteristic, described in the previous section.

In 2001, the NPL study found a 1% false match and 8% false accept rate

for common products; by now, the better ones have an equal error rate

of slightly below 1% per ﬁnger. False accepts happen because of features

incorporated to reduce the false reject rate — such as allowance for distortion

and ﬂexibility in feature selection [1080]. Spotting returning fugitives with

high enough probability to deter them and high enough certainty to detain

them (which means keeping false alarms at manageable levels) will require

several ﬁngers to be matched — perhaps eight out of ten. But requiring every

ﬁnger of every passenger to be scanned properly at immigration may cause

delays; a UK Passport Ofﬁce study found that about 20% of participants failed

to register properly when taking a 10-print, and that 10-print veriﬁcation took

over a minute [1274]. This will come down with time, but with even an extra

468 Chapter 15 ■Biometrics

30 seconds per passenger, an airport getting a planeload of 300 international

arrivals every 15 minutes would need an extra 10 working immigration lanes.

The extra building and stafﬁng costs could swamp anything spent on hardware

and software. (For more on algorithms and systems, see [832, 656, 831].)

Errors are not uniformly distributed. A number of people such as manual

workers and pipe smokers damage their ﬁngerprints frequently, and both

the young and the old have faint prints [275]. Automated systems also have

problems with amputees, people with birth defects such as extra ﬁngers, and

the (rare) people born without conventional ﬁngerprint patterns at all [764].

Fingerprint damage can also impair recognition. When I was a kid, I slashed

my left middle ﬁnger while cutting an apple, and this left a scar about half

an inch long. When I presented this ﬁnger to the system used in 1989 by the

FBI for building entry control, my scar crashed the scanner. (It worked OK

with the successor system from the same company when I tried again ten

years later.) Even where scars don’t cause gross system malfunctions, they still

increase the error rate.

Fingerprint identiﬁcation systems can be attacked in a number of ways.

An old trick was for a crook to distract (or bribe) the ofﬁcer ﬁngerprinting

him, so that instead of the hand being indexed under the Henry system

as ‘01101’ it becomes perhaps ‘01011’, so his record isn’t found and he

gets the lighter sentence due a ﬁrst offender [764]. The most recent batch

of headlines was in 2002, when Tsutomu Matsumoto caused much alarm

in the industry; he and his colleagues showed that ﬁngerprints could be

molded and cloned quickly and cheaply using cooking gelatin [845]. He tested

eleven commercially available ﬁngerprint readers and easily fooled all of

them. This prompted the German computer magazine C’T to test a number

of biometric devices that were offered for sale at the CeBIT electronic fair in

Hamburg — nine ﬁngerprint readers, one face-recognition system and one iris

scanner. They were all easy to fool — the low-cost capacitative sensors often by

such simple tricks as breathing on a ﬁnger scanner to reactivate a latent print

left there by a previous, authorized, user [1246]. Latent ﬁngerprints can also

be reactivated — or transferred — using adhesive tape. The more expensive

thermal scanners could still be defeated by rubber molded ﬁngers.

However, ﬁngerprint systems still dominate the biometric market, and are

rapidly expanding into relatively low-assurance applications, from entry into

golf club car parks to automatic book borrowing in school libraries. (Most

European countries’ privacy authorities have banned the use of ﬁngerprint

scanners in schools; Britain allows it, subject to government guidelines, with

the rationale that ﬁngerprints can’t be reverse engineered from templates and

thus privacy is protected [132]. As I’ll discuss later, this reasoning is bogus.)

An important aspect of the success of ﬁngerprint identiﬁcation systems is

not so much their error rate, as measured under laboratory conditions, but

their deterrent effect. This is particularly pronounced in welfare payment

15.5 Fingerprints 469

systems. Even though the cheap ﬁngerprint readers used to authenticate

welfare claimants have an error rate as much as 5% [267], they have turned

out to be such an effective way of reducing the welfare rolls that they have

been adopted in one place after another [890].

15.5.2 Crime Scene Forensics

The second use of ﬁngerprint recognition is in crime scene forensics. In Europe,

forensics are the main application. Prints found at a crime scene are matched

against database records, and any that match to more than a certain level are

taken as hard evidence that a suspect visited the crime scene. They are often

enough to secure a conviction on their own. In some countries, ﬁngerprints

are required from all citizens and all resident foreigners.

The error rate in forensic applications has become extremely controversial

in recent years, the critical limitation being the size and quality of the image

taken from the crime scene. The quality and procedure rules vary from one

country to another. The UK used to require that ﬁngerprints match in sixteen

points (corresponding minutiae), and a UK police expert estimated that this

will only happen by chance somewhere between one in four billion and one in

ten billion matches [764]. Greece accepts 10, Turkey 8, while the USA has no

set limit (it certiﬁes examiners instead). This means that in the USA, matches

can be found with poorer quality prints but they can be open to doubt.

In the UK, ﬁngerprint evidence went for almost a century without a

successful challenge; a 16-point ﬁngerprint match was considered to be

incontrovertible evidence. The courts’ conﬁdence in this was shattered by

the notorious McKie case [867]. Shirley McKie, a Scottish policewoman, was

prosecuted on the basis of a ﬁngerprint match on the required sixteen points,

veriﬁed by four examiners of the Scottish Criminal Records Ofﬁce. She denied

that it was her ﬁngerprint, and found that she could not get an independent

expert in Britain to support her; the profession closed ranks. She called two

American examiners who presented testimony that it is not an identiﬁcation.

The crime scene print is in Figure 15.1, and her ﬁle print is at Figure 15.2.

Figure 15.1: Crime scene print Figure 15.2: Inked print

470 Chapter 15 ■Biometrics

She was acquitted [866], which led to a political drama that ran on for years.

The ﬁrst problem was the nature of the case against her [867]. A number of

senior police ofﬁcers had tried to persuade her to make a false statement

in order to explain the presence, at the scene of a gruseome murder, of the

misidentiﬁed print. Her refusal to do so led to her being prosecuted for perjury,

as a means of discrediting her. Her acquittal said in effect that Glasgow police

ofﬁcers were not reliable witnesses. An immediate effect was that the man

convicted of the murder, David Asbury, was acquitted on appeal and sued the

police for compensation. A longer term effect was to undermine conﬁdence

in ﬁngerprints as forensic evidence. The government then prosecuted its four

ﬁngerprint experts for perjury, but this didn’t get anywhere either. The issue

went back to the Scottish parliament again and again. The police refused

to reinstate Shirley, the ofﬁcers involved got promoted, and the row got

ever more acrimonious. Eventually she won £750,000 compensation from the

government [130].

The McKie case led to wide discussion among experts of the value of ﬁnger-

print identiﬁcation [522]. It also led to ﬁngerprint evidence being successfully

challenged in a number of other countries. Two high-proﬁle cases in the USA

were Stephan Cowans and Brandon Mayﬁeld. Cowans had been convicted of

shooting a police ofﬁcer in 1997 following a robbery, but was acquitted on

appeal six years later after he argued that his print was a misidentiﬁcation

and saved up enough money to have the evidence tested for DNA. The DNA

didn’t match, which got the Boston and State police to reanalyze the ﬁnger-

print, whereupon they realised it was not a match after all. Brandon Mayﬁeld

was an Oregon lawyer who was mistakenly identiﬁed by the FBI as one of

the perpetrators of the Madrid bombing, and held for two weeks until the

Madrid police arrested another man whose ﬁngerprint was a better match.

The FBI, which had called their match ‘absolutely incontrovertible’, agreed to

pay Mayﬁeld $2 m in 2006.

In a subsequent study, psychologist Itiel Dror showed ﬁve ﬁngerprint

examiners a pair of prints, told them they were from the Mayﬁeld case, and

asked them where the FBI had gone wrong. Three of the examiners decided

that the prints did not match and pointed out why; one was unsure; and

one maintained that they did match. He alone was right. The prints weren’t

the Mayﬁeld set, but were in each case a pair that the examiner himself had

matched in a recent criminal case [402]. Dror repeated this with six experts

who each looked at eight prints, all of which they had examined for real in

the previous few years. Only two of the experts remained consistent; the other

four made six inconsistent decisions between them. The prints had a range of

difﬁculty, and in only half of the cases was misleading contextual information

supplied [403].

How did we get to a point where law enforcement agencies insist to juries

that forensic results are error-free when FBI proﬁciency exams have long had

15.5 Fingerprints 471

an error rate of about one percent [141], and misleading contextual information

can push this up to ten percent or more?

Four comments are in order.

As Figure 15.1 should make clear, ﬁngerprint impressions are often very

noisy, being obscured by dirt. So mistakes are quite possible, and the

skill (and prejudices) of the examiner enter into the equation in a much

bigger way than was accepted until the McKie case, the Mayﬁeld case,

and the general uproar that they have caused. Dror’s work conﬁrmed

that the cases in which misidentiﬁcations occur tend to be the difﬁcult

ones [403]. Yet the forensic culture was such that only certainty was

acceptable; the International Association for Identiﬁcation, the largest

forensic group, held that testifying about ‘‘possible, probable or likely

identiﬁcation shall be deemed ... conduct unbecoming.’’ [141]

Even if the probability of a false match on sixteen points were one in

ten billion (10−10) as claimed by police optimists, once many prints are

compared against each other, probability theory starts to bite. A system

that worked ﬁne in the old days as a crime scene print would be com-

pared manually with the records of a hundred and ﬁfty-seven known

local burglars, breaks down once thousands of prints are compared

every year with an online database of millions. It was inevitable that

sooner or later, enough matches would have been done to ﬁnd a 16-

point mismatch. Indeed, as most people on the ﬁngerprint database are

petty criminals who will not be able to muster the resolute defence that

Shirley McKie did, I would be surprised if there hadn’t already been

other wrongful convictions. Indeed, things may get worse, because of a

2007 agreement between European police forces that they will link up

their biometric databases (both ﬁngerprints and DNA) so that police

forces can search for matches across all EU member states [1261]. I expect

they will ﬁnd they need to develop a better understanding of proba-

bility, and much more robust ways of handling false positives.

The belief that any security mechanism is infallible creates the compla-

cency and carelessness needed to undermine its proper use. No

consideration appears to have been given to increasing the number

of points required from sixteen to (say) twenty with the introduction

of computer matching. Sixteen was tradition, the system was infallible,

and there was certainly no reason to make public funds available for

defendants’ experts. In the UK, all the experts were policemen or former

policemen, so there were no independents available for hire. Even so,

it would have been possible to use randomised matching with multiple

experts; but if the ﬁngerprint bureau had had to tell the defence in the

perhaps 5–10% of cases when (say) one of four experts disagreed, then

472 Chapter 15 ■Biometrics

many more defendants would have been acquitted and the ﬁngerprint

service would have been seen as less valuable.

A belief of infallibility ensures that the consequences of the eventual fail-

ure will be severe. As with the Munden case described in section 10.4.3,

which helped torpedo claims about cash machine security, an assump-

tion that a security mechanism is infallible causes procedures, cultural

assumptions and even laws to spring up which ensure that its eventual

failure will be denied for as long as possible, and will thus have serious

effects when it can no longer be postponed. In the Scottish case, there

appears to have arisen a hierarchical risk-averse culture in which no-

one wanted to rock the boat, so examiners were predisposed to conﬁrm

identiﬁcations made by colleagues (especially senior colleagues). This

risk aversion backﬁred when four of them were tried for perjury.

However, even when we do have a correct match its implications are not

always entirely obvious. It is possible for ﬁngerprints to be transferred using

adhesive tape, or for molds to be made — even without the knowledge of the

target — using techniques originally devised for police use. So it is possible

that the suspect whose print is found at the crime scene was framed by another

criminal (or by the police — most ﬁngerprint fabrication cases involve law

enforcement personnel rather than other suspects [179]). Of course, even if the

villain wasn’t framed, he can always claim that he was and the jury might

believe him.

In the USA, the Supreme Court’s Daubert judgment [350] ruled that trial

judges should screen the principles and methodology behind forensic evidence

to ensure it is relevant and reliable. The judge ought to consider the refereed

scientiﬁc literature — and in the case of ﬁngerprints this has been somewhat

lacking, as law enforcement agencies have been generally unwilling to submit

their examination procedures to rigorous double-blind testing. A number of

Daubert hearings relating to forensic ﬁngerprint evidence have recently been

held in U.S. trials, and the FBI has generally prevailed [523]. However, the

bureau’s former line that ﬁngerprint examination has a zero error rate is now

widely ridiculed [1208].

15.6 Iris Codes

We turn now from the traditional ways of identifying people to the modern

and innovative. Recognizing people by the patterns in the irises of their

eyes is far and away the technique with the best error rates of automated

systems when measured under lab conditions. Research on the subject was

funded by the Department of Energy, which wanted the most secure possible

way of controlling entry to premises such as plutonium stores, and the

15.6 Iris Codes 473

technology is now being used in applications such as immigration. The latest

international standards for machine-readable travel documents mandate the

use of photographs, and permit both ﬁngerprints and irises.

So far as is known, every human iris is measurably unique. It is fairly easy

to detect in a video picture, it does not wear out, and it is isolated from

the external environment by the cornea (which in turn has its own cleaning

mechanism). The iris pattern contains a large amount of randomness, and

appears to have many times the number of degrees of freedom of a ﬁngerprint.

It is formed between the third and eighth month of gestation, and (like the

ﬁngerprint pattern) is phenotypic in that there appears to be limited genetic

inﬂuence; the mechanisms that form it appear to be chaotic. So the patterns are

different even for identical twins (and for the two eyes of a single individual),

and they appear to be stable throughout life.

John Daugman found signal processing techniques that extract the informa-

tion from an image of the iris into a 256 byte iris code. This involves a circular

wavelet transform taken at a number of concentric rings between the pupil

and the outside of the iris (Figure 15.3). The resulting iris codes have the neat

property that two codes computed from the same iris will typically match

in 90% of their bits [351]. This is much simpler than in ﬁngerprint scanners

where orienting and classifying the minutiae is a hard task. The speed and

accuracy of iris coding has led to a number of commercial iris recognition

products [1327]. Iris codes provide the lowest false accept rates of any known

veriﬁcation system — zero, in tests conducted by both the U.S. Department of

Figure 15.3: An iris with iris code (courtesy John Daugman)

474 Chapter 15 ■Biometrics

Energy and the NPL [834]. The equal error rate has been shown to be better

than one in a million, and if one is prepared to tolerate a false reject rate of one

in ten thousand then the theoretical false accept rate would be less than one in

a trillion. In practice, the false reject rate is signiﬁcantly higher than this; many

things, from eyelashes to hangovers, can cause the camera to not see enough

of the iris. The U.S. Department of Defense found a 6% false reject rate in its

2002 ﬁeld trials [852]; the Passport Ofﬁce trial found 4% for normal users and

9% for disabled users [1274]. A further problem is failure to enrol; the Passport

Ofﬁce trial failed to enrol 10% of participants, and the rate was higher among

black users, the over-60s and the disabled.

One practical problem with iris scanning used to be getting the picture

cheaply without being too intrusive. The iris is small (less than half an inch)

and an image including several hundred pixels of iris is needed. A cooperative

subject can place his eye within a few inches of a video camera, and the

best standard equipment will work up to a distance of two or three feet.

Cooperation can be assumed with entry control to computer rooms. But it

is less acceptable in general retail applications as some people ﬁnd being so

close to a camera uncomfortable. All current iris scanning systems use infrared

light, and some people feel uncomfortable when this is shone in their eyes.

(The Chinese government gave this as an excuse for rejecting iris scanning

for the latest Hong Kong identity cards, going for a thumbprint instead [771].)

Given more sophisticated cameras, with automatic facial feature recognition,

pan and zoom, it is now possible to capture iris codes from airline passengers

covertly as they walk along a corridor [841], and no doubt the cost will come

down in time (especially once the key patent runs out in 2011). This is likely

to make overt uses less objectionable; but covert identiﬁcation of passers-

by has Orwellian overtones, and in Europe, data protection law could be a

show-stopper.

Possible attacks on iris recognition systems include — in unattended

operation at least — a simple photograph of the target’s iris. This may not

be a problem in entry control to supervised premises, but if everyone starts

to use iris codes to authenticate bank card transactions, then your code will

become known to many organizations. There are terminals available that will

detect such simple fakes, for example by measuring hippus — a natural ﬂuc-

tuation in the diameter of the pupil that happens at about 0.5 Hz. But the

widely-sold cheap terminals don’t do this, and if liveness detection became

widespread then no doubt attackers would try more sophisticated tricks, such

as printing the target’s iris patterns on a contact lens.

As iris recognition is fairly new, we don’t have as much experience with it

as we have with ﬁngerprints. The biggest deployment so far is in the United

Arab Emirates where it’s used to screen incoming travelers against a blacklist

of people previously deported for illegal working. The blacklist has 595,000

people as of July 2007 — 1.19 million irises — and so far 150,000 deportees have

15.7 Voice Recognition 475

been caught trying to re-enter the country. The typical arrestee is a lady with a

previous conviction for prostitution, who returns with a genuine (but corruptly

issued) passport, in a new name, from a low or middle income Asian country.

A typical attack was for the returning deportee to take atropine eyedrops on

the plane, dilating her pupils; nowadays such travelers are held in custody

until their eyes return to normal. Nonetheless, the atropine trick might be a

problem for blacklist applications in developed countries. There might also

be evidentiary problems, as iris recognition depends on computer processing;

there are no ‘experts’ at recognising eyes, and it’s doubtful whether humans

could do so reliably, as the information that John Daugman’s algorithms

depend on is mostly phase information, to which the human eye is insensitive.

(In developed countries, however, the typical application is a frequent-traveler

program that allows enrolees to bypass passport control at an airport; there

the users want to be recognised, rather than wanting not to be. The UK, for

example, has such a scheme with 200,000 enrolees. Here, evidence isn’t really

an issue.)

Despite the difﬁculties, iris codes remain a very strong contender as they

can, in the correct circumstances, provide much greater certainty than any

other method that the individual in front of you is the same human as the one

who was initially registered on the system. They alone can meet the goal of

automatic recognition with zero false acceptances.

15.7 Voice Recognition

Voice recognition — also known as speaker recognition — is the problem of iden-

tifying a speaker from a short utterance. While speech recognition systems

are concerned with transcribing speech and need to ignore speech idiosyn-

crasies, voice recognition systems need to amplify and classify them. There

are many subproblems, such as whether the recognition is text dependent or

not, whether the environment is noisy, whether operation must be real time

and whether one needs only to verify speakers or to recognize them from a

large set.

As with ﬁngerprints, the technology is used for both identiﬁcation and

forensics. In forensic phonology, the task is usually to match a recorded telephone

conversation, such as a bomb threat, to speech samples from a number of

suspects. Typical techniques involve ﬁltering and extracting features from

the spectrum; for more details see [721]. A more straightforward biometric

authentication objective is to verify a claim to identity in some telephone

systems. These range from telephone banking to the identiﬁcation of military

personnel, with over a dozen systems on the market. Campbell describes

a system that can be used with the U.S. government STU-III encrypting

telephone and that achieves an equal error rate of about 1% [264]; and the NSA

476 Chapter 15 ■Biometrics

maintains a standard corpus of test data for evaluating speaker recognition

systems [655]. A recent application is the use of voice recognition to track

asylum seekers in the UK; they will be required to ring in several times every

week [1260]. Such systems tend to use caller-ID to establish where people are,

and are also used for people like football hooligans who’re under court orders

not to go to certain places at certain times.

There are some interesting attacks on these systems, quite apart from the

possibility that a villain might somehow manage to train himself to imitate

your voice in a manner that the equipment ﬁnds acceptable. In [506] there

is a brief description of a system ﬁelded in U.S. EP-3 aircraft that breaks up

intercepted messages from enemy aircraft and ground controllers into quarter

second segments that are then cut and pasted to provide new, deceptive

messages. This is primitive compared with what can now be done with digital

signal processing. Some informed observers expect that within a few years,

there will be products available that support real-time voice and image forgery.

Crude voice morphing systems already exist, and enable female victims of

telephone sex pests to answer the phone with a male sounding voice. There

has been research aimed at improving them to the point that call centers can

have the same ‘person’ always greet you when you phone; and audio remixing

products improve all the time. Remote voice biometrics look less and less able

to withstand a capable motivated opponent.

15.8 Other Systems

Many other biometric technologies have been proposed [890]. Typing patterns,

were used in products in the 1980s but don’t appear to have been successful

(typing patterns, also known as keystroke dynamics, had a famous precursor

in the wartime technique of identifying wireless telegraphy operators by their

ﬁst, the way in which they used a Morse key). Vein patterns have been used

in one or two systems but don’t seem to have been widely sold (in the NPL

trials, the vein recognition ROC curve was almost all outside the other curves;

it was the worst of the lot) [834].

There has been growing interest recently in identifying anonymous authors

from their writing styles. Literary analysis of course goes back many years;

as a young man, the famous cryptologist William Friedman was hired by

an eccentric millionaire to study whether Bacon wrote Shakespeare. (He

eventually debunked this idea but got interested in cryptography in the

process.) Computers make it possible to run ever more subtle statistical tests;

applications range from trying to identify people who post to extremist web

fora to such mundane matters as plagiarism detection [3]. It’s possible that

such software will move from forensic applications to real-time monitoring, in

which case it would become a biometric identiﬁcation technology.

15.9 What Goes Wrong 477

Other proposals include facial thermograms (maps of the surface temperature

of the face, derived from infrared images), the shape of the ear, gait, lip

prints and the patterns of veins in the hand. Bertillon used the shape of the

ear in nineteenth century Paris, but most of the rest of these exotica don’t

seem to have been marketed as products. Other technologies may provide

opportunities in the future. For example, the huge investment in developing

digital noses for quality control in the food and drink industries may lead to a

‘digital doggie’ which recognizes its master by scent.

One ﬁnal biometric deserves passing mention — DNA typing. This has

become a valuable tool for crime scene forensics and for determining par-

enthood in child support cases, but it is still too slow for applications like

building entry control. Being genotypic rather than phenotypic, its accuracy is

also limited by the incidence of monozygotic twins: about one white person in

120 has an identical twin. There’s also a privacy problem in that it should soon

be possible to reconstruct a large amount of information about an individual

from his DNA sample. There have been major procedural problems, with

false matches resulting from sloppy lab procedure. And there are also major

data quality problems; the UK police have the biggest DNA database in the

world, with records on about four million people, but have got the names

misspelled or even wrong for about half a million of them [588]. The processes

that work for local policing don’t always scale nationally — small errors from

mistyped records, to suspects giving false names that were never discovered

because they weren’t prosecuted, accumulate along with lab errors until the

false-positive rate becomes a serious operational and political issue. For a

survey of forensic DNA analysis, and suggestions of how to make national

DNA databases consistent with privacy law, see [1124].

15.9 What Goes Wrong

As with other aspects of security, we ﬁnd the usual crop of failures due to

bugs, blunders and complacency. The main problem faced by DNA typing, for

example, was an initially high rate of false positives, due to careless laboratory

procedure. This scared off some police forces which sent in samples from

different volunteers and got back false matches, but also led to disputed court

cases and miscarriages of justice. This is reminiscent of the ﬁngerprint story,

and brings to mind the quote from Lars Knudsen at the head of Chapter 5: ‘if

it’s provably secure, it probably isn’t’. Any protection measure that’s believed to

be infallible will make its operators careless enough to break it.

Biometrics are also like many other physical protection mechanisms (alarms,

seals, tamper sensing enclosures, ...) in that environmental conditions can

cause havoc. Noise, dirt, vibration and unreliable lighting conditions all take

their toll. Some systems, like speaker recognition, are vulnerable to alcohol

478 Chapter 15 ■Biometrics

intake and stress. Changes in environmental assumptions, such as from

closed to open systems, from small systems to large ones, from attended to

stand-alone, from cooperative to recalcitrant subjects, and from veriﬁcation to

identiﬁcation, can all undermine a system’s viability.

There are a number of interesting attacks that are more speciﬁc to biometric

systems and that apply to more than one type of biometric.

Forensic biometrics often don’t tell as much as one might assume. Apart

from the possibility that a ﬁngerprint or DNA sample might have been

planted by the police, it may just be old. The age of a ﬁngerprint can’t be

determined directly, and prints on areas with public access say little. A

print on a bank door says much less than a print in a robbed vault. So in

premises vulnerable to robbery, cleaning procedures may be critical for

evidence. If a suspect’s prints are found on a bank counter, and he claims

that he had gone there three days previously, he may be convicted by

evidence that the branch counter is polished every evening. Putting this

in system terms, freshness is often a critical issue, and some quite unex-

pected things can ﬁnd themselves inside the ‘trusted computing base’.

Another aspect of freshness is that most biometric systems can, at least

in theory, be attacked using suitable recordings. We mentioned direct

attacks on voice recognition, attacks on iris scanners by photos on a

contact lens, and moulds of ﬁngerprints. Even simpler still, in countries

like South Africa where ﬁngerprints are used to pay pensions, there are

persistent tales of ‘Granny’s ﬁnger in the pickle jar’ being the most valu-

able property she bequeathed to her family. The lesson to be learned

here is that unattended operation of biometric authentication devices

is tricky. Attacks aren’t always straightforward; although it’s easy to

make a mold from a good ﬁngerprint [281], the forensic-grade prints

that people leave lying around on doorknobs, beer glasses and so on are

often too smudged and fragmentary to pass an identiﬁcation system.

However, attacks are deﬁnitely possible, and deﬁnitely happen.

Most biometrics are not as accurate for all people, and some of the pop-

ulation can’t be identiﬁed as reliably as the rest (or even at all). The

elderly, and manual workers, often have damaged or abraded ﬁnger-

prints. People with dark eyes, and large pupils, give poorer iris codes.

Disabled people with no ﬁngers, or no eyes, risk exclusion if such sys-

tems become widespread. Illiterates who make an ‘X’ are more at risk

from signature forgery.

Biometric engineers sometimes refer to such subjects dismissively as

goats, but this is foolish and offensive. A biometric system that is (or is

seen to be) socially regressive — that puts the disabled, the poor, the old

and ethnic minorities at greater risk of impersonation — may meet with

15.9 What Goes Wrong 479

principled resistance. In fact a biometric system might be defeated by

legal challenges on a number of grounds [1046]. It may also be vulner-

able to villains who are (or pretend to be) disabled. Fallback modes of

operation will have to be provided. If these are less secure, then forcing

their use may yield an attack, and if they are at least as secure, then why

use biometrics at all?

A point that follows from this is that systems may be vulnerable to col-

lusion. Alice opens a bank account and her accomplice Betty withdraws

money from it; Alice then complains of theft and produces a watertight

alibi. Quite apart from simply letting Betty take a rubber impression of

her ﬁngertip, Alice might voluntarily decrease handwriting ability; by

giving several slightly different childish sample signatures, she can force

the machine to accept a lower threshold than usual. She can spend a cou-

ple of weeks as a bricklayer, building a wall round her garden, and wear

her ﬁngerprints ﬂat, so as to degrade registration in a ﬁngerprint sys-

tem. She might register for a voice recognition system when drunk.

The statistics are often not understood by system designers, and the

birthday theorem is particularly poorly appreciated. With 10,000 biomet-

rics in a database, for example, there are about 50,000,000 pairs. So even

with a false accept rate of only one in a million, the likelihood of there

being at least one false match will rise above one-half as soon as there

are somewhat over a thousand people (in fact, 1609 people) enrolled. So

identiﬁcation is a tougher task than veriﬁcation [352]. The practical con-

sequence is that a system designed for authentication may fail when you

try to rely on it for evidence.

Another aspect of statistics comes into play when designers assume that

by combining biometrics they can get a lower error rate. The curious

and perhaps counter-intuitive result is that a combination will typically

result in improving either the false accept or the false reject rate, while

making the other worse. One way to look at this is that if you install two

different burglar alarm systems at your home, then the probability that

they will be simultaneously defeated goes down while the number of

false alarms goes up. In some cases, such as when a very good biometric

is combined with a very imprecise one, the effect can be worse over-

all [352].

Many vendors have claimed that their products protect privacy, as

what’s stored is not the image of your face or ﬁngerprint or iris, but

rather a template that’s derived from it, somewhat like a one-way hash,

and from which you can’t be identiﬁed. It’s been argued from this that

biometric data are not personal data, in terms of privacy law, and can

thus be passed around without restriction. These claims were exploded

480 Chapter 15 ■Biometrics

by Andy Adler who came up with an interesting hill-climbing attack on

face recognition systems. Given a recogniser that outputs how close an

input image is to a target template, the input face is successively altered

to increase the match. With the tested systems, this led rapidly to a rec-

ognizable image of the target — a printout of which would be accepted

as the target’s face [14]. He then showed how this hill-climbing tech-

nique could be used to attack other biometrics, including some based on

ﬁngerprints [15].

Automating biometrics can subtly change the way in which security pro-

tocols work, so that stuff that used to work now doesn’t. An example is

the biometric passport or identity card that contains your digital photo,

and perhaps your ﬁngerprint and iris data, on an RFID chip. The chip

can be cloned by copying the contents to another RFID chip (or replay-

ing them through a phone with an NFC interface.) The world’s passport

ofﬁces took the view that this wasn’t a big deal as the data are signed

and so the chip can’t be altered. However, the police have another use

for passports — if you’re on bail they insist that you leave your pass-

port with them. That protocol now breaks if you can leave the coun-

try via the fast track channel by replaying your iris data through your

mobile phone. There was also some embarrassment when researchers

discovered that despite the digital signature, they could modify the

RFID contents after all — by replacing the JPEG facial image with a bit-

string that crashed the reader [1374]. This in turn raises the question of

whether a more cunningly designed bitstring could modify the reader’s

behaviour so that it accepted forged passports. I suppose the moral is

that when passport ofﬁces digitized their systems they should have

read all of this book, not just the chapters on biometrics and crypto.

It’s worth thinking what happens when humans and computers

disagree. Iris data can’t be matched by unaided humans at all; that tech-

nology is automatic-only. But what happens when a guard and a

program disagree on whether a subject’s face matches a ﬁle photo, or

handwriting-recognition software says a bank manager’s writing looks

like a scrawled ransom note when they look quite different to the human

eye? Psychologists advise that biometric systems should be used in ways

that support and empower human cognition and that work within our

social norms [404]. Yet we engineers often ﬁnd it easier to treat the users

as a nuisance that must adapt to our technology. This may degrade

the performance of the humans. For example when an automated ﬁn-

gerprint database pulls out what it thinks is the most likely print and

presents it to the examiner: is he not likely to be biased in its favour?

Yet if the computer constantly tested the examiner’s alertness by giving

15.10 Summary 481

him the three best matches plus two poor matches, would that work any

better?

Finally, Christian fundamentalists are uneasy about biometric technol-

ogy. They ﬁnd written of the Antichrist in Revelation 13:16-18: ‘And he

causes all, both small and great, rich and poor, free and slave, to receive a

mark on their right hand or on their foreheads, and that no one may buy

or sell except one who has the mark or the name of the beast, or the num-

ber of his name.’ So biometrics may arouse political opposition on the

right as well as the left.

So there are some non-trivial problems to be overcome as biometrics tiptoe

towards mass-market use. But despite the cost and the error rates, they have

proved their worth in a number of applications — most notably where their

deterrent effect is useful.

15.10 Summary

Biometric measures of one kind or another have been used to identify people

since ancient times, with handwritten signatures, facial features and ﬁnger-

prints being the traditional methods. Systems have been built that automate

the task of recognition, using these methods and newer ones such as iris pat-

terns and voiceprints. These systems have different strengths and weaknesses.

In automatic operation, most have error rates of the order of 1% (though iris

recognition is better, hand geometry slightly better, and face recognition much

worse). There is always a trade-off between the false accept rate (the fraud

rate) and the false reject rate (the insult rate). The statistics of error rates are

deceptively difﬁcult.

If any biometric becomes very widely used, there is increased risk of forgery

in unattended operation: voice synthesisers, photographs of irises, ﬁngerprint

moulds and even good old-fashioned forged signatures must all be thought

of in system design. These do not rule out the use of biometrics, as traditional

methods such as handwritten signatures are usable in practice despite very

large error rates. That particular case teaches us that context matters; even a

weak biometric can be effective if its use is well embedded in the social and

legal matrix.

Biometrics are usually more powerful in attended operation, where with

good system design the relative strengths and weaknesses of the human guard

and the machine recognition system may complement one another. Forensic

uses are problematic, and courts are much less blindly trusting of even

ﬁngerprint evidence than they were ten years ago. Finally, many biometric

systems achieve most or all of their result by deterring criminals rather than

actually identifying them.

482 Chapter 15 ■Biometrics

Research Problems

Many practical research problems relate to the design, or improvement, of

biometric systems. Is it possible to build a system — other than iris scanning

— which will meet the banks’ goal of a 1% fraud rate and a 0.01% insult rate?

Is it possible to build a static signature veriﬁcation system which has a good

enough error rate (say 1%) for it to be used for screening images of all checks,

rather than just as a pre-screening stage to human inspection of high-value

checks? Are there any completely new biometrics that might be useful in some

circumstances?

One I thought up while writing this chapter for the ﬁrst edition in 2000,

in a conversation with William Clocksin and Alan Blackwell, was instrument-

ing a car so as to identify a driver by the way in which he operated the gears

and the clutch. If your car thinks it’s been stolen, it phones a GPS ﬁx to a

control center which then calls you to check. Recently this has come to pass;

there is now research showing that users of haptic systems can be recognised

by the way in which they use tools [990].

Further Reading

The history of ﬁngerprints is good reading. The standard reference is

Lambourne [764], while Block has a good collection of U.S. case histories [195]

and the history of ﬁngerprints in India is told by Sengoopta [1145]. The McKie

case is described in a book by Ian McKie and Michael Russella [867]. A good

technical reference on automated ﬁngerprint identiﬁcation systems is the book

by Maltoni, Maio, Jain and Prabhakar [832]; there’s also an earlier book by

Jain, Bolle and Pankanti [655]. As for facial and handwriting recognition in the

text, there’s also an IBM experimental system described at [684] and a survey

of the literature at [288]. The standard work on iris codes is Daugman [351].

For voice recognition, there is a tutorial in [264] which focuses on speaker

identiﬁcation while for the forensic aspects, see Klevans and Rodman [721].

Snapshots of the state of the technical art can be found in two journal special

issues of the Proceedings of the IEEE on biometric systems — volume 85 no 9

(September 1997) and volume 94 no 11 (November 2006).

CHAPTER

Physical Tamper Resistance

It is relatively easy to build an encryption system that is secure if it is working as

intended and is used correctly but it is still very hard to build a system that does not

compromise its security in situations in which it is either misused or one or more of

its sub-components fails (or is ’encouraged’ to misbehave) ... this is now the only

area where the closed world is still a long way ahead of the open world and the

many failures we see in commercial cryptographic systems provide some evidence

for this.

— Brian Gladman

The amount of careful, critical security thinking that has gone into a given security

device, system or program is inversely proportional to the amount of

high-technology it uses.

— Roger Johnston

16.1 Introduction

Low-cost tamper-resistant devices are becoming almost ubiquitous. Examples

I’ve discussed so far include:

smartcards used as SIMs in mobile phones and as bank cards in Europe;

accessory control chips used in printer toner cartridges, mobile phone

batteries and games-console memory modules;

the TPM chips being shipped in PCs and Macs to support hard-disk

encryption, DRM and software registration;

security modules used to manage bank PINs, not just in bank server

farms but in ATMs and point-of-sale terminals;

483

484 Chapter 16 ■Physical Tamper Resistance

security modules buried in vending machines that sell everything from

railway tickets through postage stamps to the magic numbers that acti-

vate your electricity meter.

Many of the devices on the market are simply pathetic, like the banking

terminals whose failures I described in section 10.6.1.1: those terminals could

be trivially compromised in under a minute using simple tools, despite having

been evaluated by VISA and also using the Common Criteria framework.

Yet some tamper-resistant processors are getting pretty good. For example, I

know of one ﬁrm that spent half a million dollars trying, and failing, to reverse-

engineer the protocol used by a games console vendor to stop competitors

making memory modules compatible with its equipment1. But a few years

ago this was not the case. Serious tamper resistance emerged out of an arms

race between ﬁrms that wanted to lock down their products, and others who

wanted to unlock them. Some of the attackers were respectable companies

exercising their legal rights to reverse engineer for compatibility. Others were

lawyers, reverse engineering products to prove patent infringements. There

are half a dozen specialist ﬁrms that work for the lawyers, and the legal

reverse engineers. There are academics who hack systems for glory, and to

push forward the state of the art. There are bad guys like the pay-TV pirates

who clone subscriber cards. And ﬁnally there are lots of grey areas. If you ﬁnd

a way to unlock a particular make of mobile phone, so that it can be used on

any network, is that a crime? The answer is, it depends what country you’re in.

There are now many products on the market that claim to be tamper-

resistant, from cheap microcontrollers through smartcards to expensive cryp-

toprocessors. Some of them are good; many are indifferent; and some are

downright awful. It is increasingly important for the security engineer to

understand what tamper resistance is, and what it can and can’t do. In this

chapter I’m going to take you through the past ﬁfteen years or so, as ever more

clever attacks have been met with successively more sophisticated defenses.

It has long been important to make computers resist physical tampering, as

an attacker who can get access can in principle change the software and get

the machine to do what he wants. While computers were massive objects, this

involved the techniques discussed in the previous few chapters — physical

barriers, sensors and alarms. In some applications, a computer is still made into

a massive object: an ATM is basically a PC in a safe with banknote dispensers

and alarm sensors, while the sensor packages used to detect unlawful nuclear

tests may be at the bottom of a borehole several hundred feet deep and

backﬁlled with concrete.

Where tamper resistance is needed purely for integrity and availability, it can

sometimes be implemented using replication instead of physical protection. A

1Eventually the memory module was cracked, but it took a custom lab with chip testing

equipment and a seven ﬁgure budget.

16.2 History 485

service may be implemented on different servers in different sites that perform

transactions simultaneously and vote on the result; and the threshold schemes

discussed in section 13.4 can also provide conﬁdentiality for key material. But

tamper-resistant devices can provide conﬁdentiality for the data too. This is

one respect in which the principle that many things can be done either with

mathematics or with metal, breaks down.

16.2 History

The use of tamper resistance in cryptography goes back centuries [676]. Naval

codebooks were weighted so they could be thrown overboard if capture

was imminent; to this day, the dispatch boxes used by British government

ministers’ aides to carry state papers are lead lined so they will sink. Codes

and, more recently, the keys for wartime cipher machines have been printed

in water soluble ink; Russian one-time pads were printed on cellulose nitrate,

so that they would burn furiously if lit; and one U.S. wartime cipher machine

came with self-destruct thermite charges so it could be destroyed quickly.

But such mechanisms depended on the vigilance of the operator, and key

material was often captured in surprise attacks. So attempts were made to

automate the process. Early electronic devices, as well as some mechanical

ciphers, were built so that opening the case erased the key settings.

Following a number of cases in which key material was sold to the other side

by cipher staff — such as the notorious Walker family in the USA, who sold

U.S. Navy key material to the Russians for over 20 years [587] — engineers

paid more attention to the question of how to protect keys in transit too.

The goal was ‘to reduce the street value of key material to zero’, and this

can be achieved either by tamper resistant devices from which the key cannot

be extracted, or tamper evident ones from which key extraction would be

obvious.

Paper keys were once carried in ‘tattle-tale containers’, designed to show

evidence of tampering. When electronic key distribution came along, a typical

solution was the ‘ﬁll gun’: a portable device that dispenses crypto keys

in a controlled way. Nowadays this function is usually performed using a

small security processor such as a smartcard; as with electricity meters, it

may be packaged as a ‘crypto ignition key’. Control protocols range from a

limit on the number of times a key can be dispensed, to mechanisms using

public key cryptography to ensure that keys are only loaded into authorized

equipment. The control of key material also acquired broader purposes. In

both the USA and the UK, it was centralized and used to enforce the use

of properly approved computer and communications products. Live key

material would only be supplied to a system once it had been properly

accredited.

486 Chapter 16 ■Physical Tamper Resistance

Once initial keys have been loaded, further keys may be distributed using

various authentication and key agreement protocols. I already talked about

many of the basic tools, such as key diversiﬁcation, in the chapter on protocols

in Part I, and I’ll have more to say on protocols later in the chapter in

API attacks. Here, I’m going to look ﬁrst at the physical defenses against

tampering.

16.3 High-End Physically Secure Processors

An example worth studying is the IBM 4758 (Figures 16.1 and 16.2). This

is important for three reasons. First, it was the ﬁrst commercially available

processor to have been successfully evaluated to the highest level of tamper

resistance (FIPS 140-1 level 4) [938] then set by the U.S. government. Second,

there is an extensive public literature about it, including the history of its design

evolution, its protection mechanisms, and the transaction set it supports [1195,

1328, 1330]. Third, as it was the ﬁrst level-4-evaluated product, it was the

highest proﬁle target in the world of tamper resistance, and from 2000–2005

my students and I put some effort into attacking it.

Figure 16.1: The IBM 4758 cryptoprocessor (courtesy of Steve Weingart)

16.3 High-End Physically Secure Processors 487

Figure 16.2: The 4758 partially opened showing (from top left downward) the circuitry,

aluminium electromagnetic shielding, tamper sensing mesh and potting material (courtesy

of Frank Stajano)

The evolution that led to this product is brieﬂy as follows. The spread

of multi-user operating systems, and the regularity with which bugs were

found in their protection mechanisms, meant that large numbers of people

might potentially have access to the data being processed. The reaction of

the military computing community, which I described in Chapter 9, was the

Anderson report and multilevel security. The reaction of the banking commu-

nity was to focus on particularly sensitive data — and speciﬁcally on long-term

cryptographic keys and the personal identiﬁcation numbers (PINs) used by

bank customers to identify themselves to cash machines. It was realized in the

early 1980s that the level of protection available from commercial operating

systems was likely to remain insufﬁcient for these ‘crown jewels’.

This led to the development of standalone security modules of which the ﬁrst

to be commercially successful were the IBM 3848 and the VISA security module.

Both of these were microcomputers encased in robust metal enclosures, with

encryption hardware and special key memory, which was static RAM designed

to be zeroized when the enclosure was opened. This was accomplished by

wiring the power supply to the key memory through a number of lid switches.

So whenever the maintenance crew came to replace batteries, they’d open the

lid and destroy the keys. Once they’d ﬁnished, the device operators would

then reload the key material. In this way, the device’s owner could be happy

that its keys were under the unique control of its own staff.

488 Chapter 16 ■Physical Tamper Resistance

How to hack a cryptoprocessor (1)

The obvious attack on such a device is for the operator to steal the keys. In early

banking security modules, the master keys were kept in PROMs that were

loaded into a special socket in the device to be read during initialization, or as

strings of numbers which were typed in at a console. The PROMs could easily

be pocketed, taken home and read out using hobbyist equipment. Cleartext

paper keys were even easier to steal.

The ﬁx was shared control — to have two or three PROMs with master

key components, and make the device master keys the exclusive-or of all the

components. The PROMs can then be kept in different safes under the control

of different departments. (With the virtue of hindsight, the use of exclusive-or

for this purpose was an error, and a hash function should have been used

instead. I’ll explain why shortly.)

However, this procedure is tedious and such procedures tend to degrade.

In theory, when a device is maintained, its custodians should open the lid

to erase the live keys, let the maintenance engineer load test keys, and then

re-load live keys afterwards. The managers with custodial responsibility will

often give their PROMs to the engineer rather than bothering with them. I’ve

even come across cases of the master keys for an automatic teller machine

being kept in the correspondence ﬁle in a bank branch, where any of the staff

could look them up.

Prudent cryptography designers try to minimize the number of times that a

key reload will be necessary, whether because of maintenance or power failure.

So modern security modules typically have batteries to back up the mains

power supply (at least to the key memory). Thus, in practice, the custodians

have to load the keys only when the device is ﬁrst installed, and after occasional

maintenance visits after that.

It has been debated whether frequent or infrequent key loading is best. If

key loading is very infrequent, then the responsible personnel will likely never

have performed the task before, and may either delegate it out of ignorance, or

be hoodwinked by a more technically astute member of staff into doing it in an

insecure way (see [33] for a case history of this). The modern trend is toward

devices that generate master keys (or have them loaded) in a secure facility

after manufacture but before distribution. But not all keys can be embedded

in the processor at the factory. Some keys may be kept on smartcards and

used to bootstrap key sharing and backup between processors; others may be

generated after distribution, especially signature keys that for legal reasons

should always be under the customer’s unique control.

How to hack a cryptoprocessor (2)

Early devices were vulnerable to attackers cutting through the casing, and to

maintenance engineers who could disable the lid switches on one visit and

16.3 High-End Physically Secure Processors 489

extract the keys on the next. Second generation devices dealt with the easier

of these problems, namely physical attack, by adding further sensors such as

photocells and tilt switches. These may be enough for a device kept in a secure

area to which access is controlled. But the hard problem is to prevent attacks

by the maintenance man.

The strategy adopted by many of the better products is to separate all

the components that can be serviced (such as batteries) from the core of the

device (such as the tamper sensors, crypto, processor, key memory and alarm

circuits). The core is then ‘potted’ into a solid block of a hard, opaque substance

such as epoxy. The idea is that any physical attack will be ‘obvious’ in that it

involves an action such as cutting or drilling, which can be detected by the

guard who accompanies the maintenance engineer into the bank computer

room. (That at least was the theory; my own experience suggests that it’s a

bit much to ask a minimum-wage guard to ensure that a specialist in some

exotic piece equipment repairs it using some tools but not others.) At least it

should leave evidence of tampering after the fact. This is the level of protection

needed for medium-level evaluations under the FIPS standard.

How to hack a cryptoprocessor (3)

However, if a competent person can get unsupervised access to the device for

even a short period of time — and, to be realistic, that’s what the maintenance

engineer probably has, even if the guard is breathing down his neck — then

potting the device core is inadequate. For example, it is often possible to scrape

away the potting with a knife and drop the probe from a logic analyzer on

to one of the bus lines in the core. Most common cryptographic algorithms,

such as RSA and DES, have the property that an attacker who can monitor

any bitplane during the computation can recover the key [580]. So an attacker

who can get a probe anywhere into the device while it is operating can likely

extract secret key material.

So the high-end products have a tamper-sensing barrier whose penetration

triggers destruction of the secrets inside. An early example appeared in IBM’s

μABYSS system in the mid 1980s. This used loops of 40-gauge nichrome wire

that were wound loosely around the device as it was embedded in epoxy,

and then connected to a sensing circuit [1328]. Bulk removal techniques such

as milling, etching and laser ablation break the wire, which erases the keys.

But the wire-in-epoxy technique can be vulnerable to slow erosion using sand

blasting; when the sensing wires become visible at the surface of the potting,

shunts can be connected round them. So the next major product from IBM, the

4753, used a metal shield combined with a membrane printed with a pattern

of conductive ink and surrounded by a more durable material of similar

chemistry. The idea was that any attack would break the membrane with high

probability.

490 Chapter 16 ■Physical Tamper Resistance

How to hack a cryptoprocessor (4)

The next class of methods an attacker can try involve the exploitation of memory

remanence, the fact that many kinds of computer memory retain some trace of

data that have been stored there. Sometimes all that is necessary is that the same

data were stored for a long time. An attacker might bribe the garbage truck

operator to obtain a bank’s discarded security modules: as reported in [69],

once a certain security module had been operated for some years using the

same master keys, the values of these keys were burned in to the device’s static

RAM. On power-up, about 90% of the relevant bits would assume the values of

the corresponding keybits, which was more than enough to recover the keys.

Memory remanence affects not just static and dynamic RAM, but other

storage media as well. For example, the heads of a disk drive change alignment

over time, so that it may be impossible to completely overwrite data that were

ﬁrst written some time ago. The relevant engineering and physics issues are

discussed in [566] and [568], while [1184] explains how to extract data from

Flash memory in microcontrollers, even after it has been ‘erased’ several times.

The NSA has published guidelines (the ‘Forest Green Book’) on preventing

remanence attacks by precautions such as careful degaussing of media that

are to be reused [378].

The better third generation devices have RAM savers which function in much

the same way as screen savers; they move data around the RAM to prevent it

being burned in anywhere.

How to hack a cryptoprocessor (5)

A further problem is that computer memory can be frozen by low temperatures.

By the 1980s it was realized that below about −20◦C, static RAM contents can

persist for several seconds after power is removed. This extends to minutes

at the temperatures of liquid nitrogen. So an attacker might freeze a device,

remove the power, cut through the tamper sensing barrier, extract the RAM

chips containing the keys and power them up again in a test rig. RAM contents

can also be burned in by ionising radiation. (For the memory chips of the

1980s, this required a serious industrial X-ray machine; but as far as I’m aware,

no-one has tested the current, much smaller, memory chip designs.)

So the better devices have temperature and radiation alarms. These can

be difﬁcult to implement properly, as modern RAM chips exhibit a wide

variety of memory remanence behaviors, with some of them keeping data for

several seconds even at room temperature. What’s worse, remanence seems

to have got longer as feature sizes have shrunk, and in unpredictable ways

even within standard product lines. The upshot is that although your security

module might pass a remanence test using a given make of SRAM chip, it

might fail the same test if ﬁtted with the same make of chip purchased a

year later [1182]. This shows the dangers of relying on a property of some

component to whose manufacturer the control of this property is unimportant.

16.3 High-End Physically Secure Processors 491

Temperature sensors are also a real bugbear to security module vendors, as

a device that self-destructs if frozen can’t be sent reliably through normal

distribution channels. (We’ve bought cryptoprocessors on eBay and found

them dead on arrival.)

How to hack a cryptoprocessor (6)

The next set of attacks on cryptographic hardware involve either monitoring

the RF and other electromagnetic signals emitted by the device, or even

injecting signals into it and measuring their externally visible effects. This

technique, which is variously known as ‘Tempest’, ‘power analysis,’ ‘side-

channel attacks’ or ‘emission security’, is such a large subject that I devote

the next chapter to it. As far as the 4758 is concerned, the strategy is to have

solid aluminium shielding and to low-pass ﬁlter the power supply to block

the egress of any signals at the frequencies used internally for computation.

The 4758 also has an improved tamper sensing membrane in which four

overlapping zig-zag conducting patterns are doped into a urethane sheet,

which is potted in a chemically similar substance so that an attacker cutting

into the device has difﬁculty even detecting the conductive path, let alone con-

necting to it. This potting surrounds the metal shielding which in turn contains

the cryptographic core. The design is described in more detail in [1195].

How to hack a cryptoprocessor (7)

I don’t know how to attack the hardware of the 4758. My students and I found

a number of novel software vulnerabilities, which I’ll describe later in the

chapter on API Attacks. But here are a few ideas for keen grad students who

want to have a go at the hardware:

The straightforward approach would be to devise some way to erode the

protective potting, detect mesh lines, and connect shunts round them. A

magnetic force microscope might be worth a try.

One could invent a means of drilling holes eight millimeters long and

only 0.1 millimeters wide (that is, much less than the mesh line diame-

ter). This isn’t straightforward with standard mechanical drills, and the

same holds for laser ablation and ion milling. However I speculate that

some combination of nanotechnology and ideas from the oil industry

might make such a drill possible eventually. Then one could drill right

through the protective mesh with a fair probability of not breaking the

circuit.

Having dismantled a few instances of the device and understood its

hardware, the attacker might attempt to destroy the tamper responding

circuitry before it has time to react. One possibility is to use an

industrial X-ray machine; another would be to use shaped explosive

charges to send plasma jets of the kind discussed in section 13.5 into the

device.

492 Chapter 16 ■Physical Tamper Resistance

The success of such attacks is uncertain, and they are likely to remain beyond

the resources of the average villain for some time.

So by far the attacks on 4758-based systems involve the exploitation of

logical rather than physical ﬂaws. The device’s operating system has been

subjected to formal veriﬁcation, so the main risk resides in application design

errors that allow an opponent to manipulate the transactions provided by the

device to authorized users. Most users of the 4758 use an application called

CCA that is described in [619] and contains many features that make it difﬁcult

to use properly (these are largely the legacy of previous encryption devices

with which 4758 users wished to be backward compatible.) Starting in 2000, we

discovered that the application programming interface (API) which the 4758

exposed to the host contained a number of serious ﬂaws. (Most of the other

security modules on the market were worse.) The effect was that a programmer

with access to the host could send the security module a series of commands

that would cause it to leak PINs or keys. I’ll discuss these API attacks in

Chapter 18.

Finally, it should be mentioned that the main constraints on the design and

manufacture of security processors are remarkably similar to those we encoun-

tered with more general alarms. There is a trade-off between the false alarm

rate and the missed alarm rate, and thus between security and robustness.

Vibration, power transients and electromagnetic interference can be a problem,

but temperature is the worst. I mentioned the difﬁculty of passing security

processors that self-destruct at −20◦C through normal shipping channels,

where goods are often subjected to −40◦C in aircraft holds. Military equip-

ment makers have the converse problem: their kit must be rated from −55◦to

+155◦C. Some military devices use protective detonation; memory chips are

potted in steel cans with a thermite charge precisely calculated to destroy the

chip without causing gas release from the can. Meeting simultaneous targets

for tamper resistance, temperature tolerance, radiation hardening and weight

can be expensive.

16.4 Evaluation

A few comments about the evaluation of tamper-resistant devices are in order

before I go on to discuss cheaper devices.

The IBM paper which describes the design of the 4753 [6] proposed the

following classiﬁcation of attackers, which has been widely used since:

1. Class 1 attackers — ‘clever outsiders’ — are often very intelligent but

may have insufﬁcient knowledge of the system. They may have access to

only moderately sophisticated equipment. They often try to take advan-

tage of an existing weakness in the system, rather than try to create one.

16.4 Evaluation 493

2. Class 2 attackers — ‘knowledgeable insiders’ — have substantial spe-

cialized technical education and experience. They have varying degrees

of understanding of parts of the system but potential access to most of it.

They often have highly sophisticated tools and instruments for analysis.

3. Class 3 attackers — ‘funded organizations’ — are able to assemble teams

of specialists with related and complementary skills backed by great

funding resources. They are capable of in-depth analysis of the system,

designing sophisticated attacks, and using the most advanced analysis

tools. They may use Class 2 adversaries as part of the attack team.

Within this scheme, the typical SSL accelerator card is aimed at blocking

clever outsiders; the early 4753 aimed at stopping clever insiders, and the 4758

was aimed at (and certiﬁed for) blocking funded organizations. (By the way,

this classiﬁcation is becoming a bit dated; we see class 1 attackers renting access

to class 3 equipment and so on. It’s better to create an attacker proﬁle for your

application, rather than try to reuse one of these old industry standard ones.)

The FIPS certiﬁcation scheme is operated by laboratories licenced by the

U.S. government. The original standard, FIPS 140-1, set out four levels of

protection, with level 4 being the highest, and this remained in the next

version, FIPS 140-2, which was introduced in 2001. At the time of writing,

only three vendors — IBM, AEP and Thales — have managed to get products

certiﬁed at level 4. There is a large gap between level 4 and the next one down,

level 3, where only potting is required and attacks that exploit electromagnetic

leakage, memory remanence, drilling, sandblasting and so on may still be

possible. (I have handled a level-3 certiﬁed device from which I could scrape

off the potting with my Swiss army knife.) So while FIPS 140-1 level 3 devices

can be (and have been) defeated by class 1 attackers in the IBM sense, the

next step up — FIPS 140-1 level 4 — is expected to keep out an IBM class 3

opponent. There is no FIPS level corresponding to a defence against IBM’s

class 2.

In fact, the original paper on levels of evaluation was written by IBM

engineers and proposed six levels [1330]; the FIPS standard adopted the ﬁrst

three of these as its levels 1–3, and the proposed level 6 as its level 4 (the 4758

designer Steve Weingart tells the story in [1329]). The gap, commonly referred

to as ‘level 3.5’, is where many of the better commercial systems are aimed.

Such equipment certainly attempts to keep out the class 1 attack community,

while making life hard for class 2 and expensive for class 3.

At the time of writing (2007) there is a revised Federal standard, FIPS 140-3,

out from NIST for consultation. This increases the number of levels from

four to ﬁve, by adding a ﬁfth level with additional testing required. However

the standard does not deal with the problem of API attacks, and indeed

neither did FIPS 140-1 or 140-2. The FIPS standard unfortunately covers only

the device’s resistance against direct invasive and semi-invasive hardware

494 Chapter 16 ■Physical Tamper Resistance

attacks, of the kind discussed in this chapter, and against noninvasive attacks

using techniques like power analysis, which I discuss in the next.

16.5 Medium Security Processors

Good examples of ‘level 3.5’ products are the iButton and 5002 security

processors from Dallas Semiconductor, and the Capstone chip used to protect

U.S. military communications up to ‘Secret’. While the 4758 costs $2000, these

products cost of the order of $10–20. Yet mounting an attack on them is far

from trivial.

16.5.1 The iButton

The original iButton from Dallas Semiconductor was designed to be a minimal,

self-contained cryptographic processor for use in applications such as postal

meters. It has a microcontroller, static RAM for keys and software, a clock

and tamper sensors encased in a steel can with a lithium battery that can

maintain keys in the RAM for a design life of ten years (see Figure 16.3).

It is small enough to be worn in a signet ring or carried as a key fob. An

early application was as an access token for the ‘Electronic Red Box’, a secure

laptop system designed for use by UK government ministers. To access secret

documents, the minister had to press his signet ring into a reader at the side

of the laptop. (One of the design criteria had been: ‘Ministers shall not have

to use passwords’.) Other applications include ticketing for the ˙

Istanbul mass

transit system, parking meters in Argentina, and tokens used by bar staff to

logon rapidly to tills.

The iButton was a pure security product when I wrote the ﬁrst edition in 2000

and some versions even had a cryptoprocessor to do public-key operations.

In December 1999 the ﬁrst break of an iButton took place; the printer vendor

Lexmark had started incorporating iButtons in some of its printer cartridges in

1998, to stop aftermarket vendors selling compatible ones, and Static Control

Components ﬁnally broke the system and produced a compatible chip for

the aftermarket in December 1999 [1221]. Since then Dallas has become a

subsidiary of Maxim and the iButton has evolved into a broader product

range. There are now versions containing temperature sensors that provide a

dependable temperature history of perishable goods in transit, simple versions

that are in effect just RFID tokens, and complex versions that contain a JVM.

So it’s no longer the case that all iButtons are ‘secure’: any given device might

not be designed to be, or it might have been programmed (deliberately or

otherwise) not to be. However the range still does include tamper-resistant

devices that use SHA-1 to support cryptographic authentication, and it’s still

widely used in security applications.

16.5 Medium Security Processors 495

Stainless Can

Micro Switch

Crypto iButton Assembly

Metalurgical Bond

Micro Switch

Quartz

Energy Reservoir

Stainless Can Lid

Lithium Backup

Grommet

Barricade

ESD Suppressor Crypto

Computer Chip

Figure 16.3: iButton internals (courtesy of Dallas Semiconductor Inc.)

How might a secure iButton be attacked? The most obvious difference from

the 4758 is the lack of a tamper-sensing barrier. So one might try drilling in

through the side and then either probing the device in operation, or disabling

the tamper sensing circuitry. As the iButton has lid switches to detect the can

being opened, and as the processor is mounted upside-down on the circuit

board and is also protected by a mesh in the top metal layer of the chip, this

is unlikely to be a trivial exercise and might well involve building custom jigs

and tools. I wrote in the ﬁrst edition, ‘it’s a tempting target for the next bright

graduate student who wants to win his spurs as a hardware hacker’, and SCC

appear to have risen to the challenge. (Lexmark then sued them under the

DMCA and lost; I’ll discuss this further in section 22.6 on accessory control.)

16.5.2 The Dallas 5000 Series

Another medium-grade security device from Maxim / Dallas is the 5000 series

secure microcontroller, which is widely used in devices such as point-of-sale

terminals where it holds the keys used to encrypt customer PINs.

The ingenious idea behind this device is bus encryption. The chip has added

hardware which encrypts memory addresses and contents on the ﬂy as data

are loaded and stored. This means that the device can operate with external

memory and is not limited to the small amount of RAM that can be ﬁtted into a

low-cost tamper-sensing package. Each device has a unique master key, which

is generated at random when it is powered up. The software is then loaded

496 Chapter 16 ■Physical Tamper Resistance

through the serial port, encrypted and written to external memory. The device

is then ready for use. Power must be maintained constantly, or the internal

tampering event is sensed (like the iButton, the DS5002 has a tamper-sensing

mesh built into the top metal layer of the chip).

An early version of the 5002 (1995) fell to an ingenious protocol attack by

Markus Kuhn, the cipher instruction search attack [748]. The idea is that some

of the processor’s instructions have a visible external effect, such as I/O. In

particular, there is one instruction which causes the next byte in memory to

be output to the device’s parallel port. So the trick is to intercept the bus

between the processor and memory using a test clip, and feed in all possible

8-bit instruction bytes at some point in the instruction stream. One of them

should decrypt to the parallel output instruction, and output the plaintext

version of the next ‘encrypted memory’ byte. By varying this byte, a table

could be built up of corresponding plaintext and ciphertext. After using this

technique to learn the encryption function for a sequence of seven or eight

bytes, the attacker could encipher and execute a short program to dump the

entire memory contents.

The full details are a bit more intricate. The problem has since been ﬁxed,

and Dallas is selling successor products such as the 5250. However, the attack

on bus encryption is a good example of the completely unexpected things that

go wrong when trying to implement a clever new security concept for the ﬁrst

time.

16.5.3 FPGA Security, and the Clipper Chip

In 1993, the security world was convulsed when the US government introduced

the Clipper chip as a proposed replacement for DES. Clipper, also known as the

Escrowed Encryption Standard (EES), was a tamper-resistant chip that imple-

mented the Skipjack block cipher in a protocol designed to allow the U.S.

government to decrypt any trafﬁc encrypted using Clipper. The idea was that

when a user supplied Clipper with a string of data and a key with which to

encrypt it, the chip returned not just the ciphertext but also a ‘Law Enforce-

ment Access Field’, or LEAF, which contained the user-supplied key encrypted

under a key embedded in the device and known to the government. To prevent

people cheating and sending along the wrong LEAF with a message, the LEAF

has a cryptographic checksum computed with a ‘family key’ shared by all

interoperable Clipper chips. This functionality was continued into the next

generation chips, called Capstone, which incorporate ARM processors to do

public key encryption and digital signature operations.

Almost as soon as Capstone chips hit the market, a protocol vulnerability

was found [183]. As the cryptographic checksum used to bind the LEAF to

the message was only 16 bits long, it was possible to feed message keys into

16.5 Medium Security Processors 497

the device until you got one with a given LEAF, thus enabling a message to

be sent with a LEAF that would remain impenetrable to the government. The

Clipper initiative was abandoned and replaced with other policies aimed at

controlling the ‘proliferation’ of cryptography, but Capstone quietly entered

government service and is now widely used in the Fortezza card, a PCMCIA

card used in PCs to encrypt data at levels up to Secret. The Skipjack block cipher,

which was initially classiﬁed, has since been placed in the public domain [939].

Of more interest here are the tamper protection mechanisms used, which

were perhaps the most sophisticated in any single-chip device at the time,

and were claimed at the time to be sufﬁcient to withstand a ‘very sophisticated,

well funded adversary’ [937]. Although it was claimed that the Clipper chip

would be unclassiﬁed and exportable, I was never able to get hold of one for

dismantling despite repeated attempts.

Its successor is the QuickLogic military FPGA. This is designed to enable

its users to conceal proprietary algorithms from their customers, and it is

advertised as being ‘virtually impossible to reverse engineer’. Like Clipper,

it uses Vialink read only memory (VROM) in which bits are set by blowing

antifuses between the metal 1 and metal 2 layers on the chip. A programming

pulse at a sufﬁciently high voltage is used to melt a conducting path through

the polysilicon which separates the two metal layers. Further details and

micrographs can be found in a paper in the relevant data book [547].

FPGA security has since become a thriving research ﬁeld. There are basically

four approaches to reverse engineering an antifuse device.

The easiest way to read out an FPGA is usually to use the test circuit

that’s provided in order to read back and verify the bitstream during

programming. Until recently, many chips disabled this by melting a

single fuse after programming. If you could get sample devices and a

programmer, you could ﬁnd out where this fuse was located, for

example by differential optical probing [1186]. You could then use a

focussed ion beam workstation to repair the fuse. Once you have re-

enabled the test circuit you can read out the bitstream. Turning this

into a netlist isn’t entirely straightforward but it’s not impossible given

patience. This attack technique works not just for antifuse FPGAs but

also for the Flash and EEPROM varieties.

With antifuse FPGAs, it’s sometimes possible to abuse the programming

circuit. This circuit is designed to send a pulse to melt the fuse, but stop

once the resistance drops, as this means the metal has melted and estab-

lished contact. If the pulse weren’t stopped then the metal might vapor-

ise and go open circuit again. So the programming circuit has sensors

to detect whether a fuse is open or short, and if these aren’t sufﬁciently

disabled after programming they can be used to read the bitstream out.

498 Chapter 16 ■Physical Tamper Resistance

The brute-force approach is to determine the presence of blown anti-

fuses using optical or electron microscopy, for example by removing the

top metal layer of the chip and looking at the vias directly. This can be

extremely tedious, but brute force attacks have been done on some secu-

rity processors.

Where the device implements a cryptographic algorithm, or some other

piece of logic with a relatively compact description, an inference attack

using side-channel data may be the fastest way in. Most devices man-

ufactured before about 2000 are rather vulnerable to power analysis,

which can be used to reconstruct not just the keys of block ciphers but

also the block cipher design by studying the device’s power consump-

tion. I’ll describe this in more detail in the next chapter. A more direct

attack is to drop microprobes directly on the gate array’s bus lines and

look at the signals. Other possible sensors include optical probing, elec-

tromagnetic coils, voltage contrast microscopy and a growing number

of other chip testing techniques. In any case, the point is that it may be

faster to reconstruct an algorithm from observing on-chip signals than

from doing a full circuit reconstruction.

So this technology isn’t infallible, but used intelligently it certainly has

potential. Its main use nowadays is in FPGAs that are used to protect designs

from reverse engineering.

In recent years, most FPGAs sold have had conventional memory rather

than antifuse, which means that they can be made reprogrammable. They

also have a higher memory density, albeit at some cost in power, size and

performance compared to ASICs or antifuse devices. The current market

leaders are volatile FPGAs that use SRAM, and many chips don’t really have

any protection beyond the obscurity of the bitstream encoding; you can read

the unencrypted bitstream in transit to the FPGA on power-up. Others have

one or more embedded keys kept in nonvolatile memory, and the bitstream

is uploaded and then decrypted on power-up. Non-volatile devices generally

use Flash for the bitstream, and reprogramming involves sending the device

an encrypted bitstream with a message authentication code to guarantee

integrity. In both cases, there may be a few fuses to protect the key material

and the protection state. The better designs are so arranged that an attacker

would have to ﬁnd several fuses and antifuses and wire them in a particular

way in order to extract the bitstream. Saar Drimer has a survey of FPGA design

security [400].

Such FPGAs are used in equipment from routers through printers to cam-

eras, and the main threat models are that a manufacturing subcontractor may

try to overbuild the design so as to sell the extra products on the grey market,

or that a competitor will try to make compatible accessories. In the former

case, the vendor can separate programming from assembly, and also use the

16.6 Smartcards and Microcontrollers 499

upgrade cycle to undermine the value of any black-market copies. In the latter

case the FPGA logic typically implements an authentication mechanism of

some kind.

One trap to watch out for when designing a ﬁeld-upgradeable product using

an FPGA is to prevent service denial attacks via the upgrade mechanism. For

example, a Flash FPGA usually only has enough memory for one copy of the

bitstream, not two; so the usual drill is to read in the bitstream once to decrypt

it and verify the MAC, and then a second time to actually reprogram the part.

If the bitstream supplied the second time is corrupt, then the likely outcome

is a dead product. The typical SRAM-based FPGAs doesn’t have bitstream

authentication, so people could usually replay old bitstreams and thus escape

an upgrade if they wish. It’s also possible that if an attacker gets your products

to load a random encrypted bitstream, this could cause short circuits and do

permanent damage. So it’s worth thinking whether anyone might be malicious

enough to try to destroy your installed product base via a corrupt upgrade,

and if so you might consider whether your product needs a secure bitstream

loader.

16.6 Smartcards and Microcontrollers

In volume terms, the most common secure processors nowadays are self-

contained one-ship security processors containing nonvolatile memory, I/O,

usually a CPU, often some specialised logic, and some mechanisms to prevent

memory contents from being read out. They range from microcontrollers with

a memory-protect bit at the low end, up to smartcards and the TPMs that now

ship with most computer motherboards. Smartcards have quite a range of

capabilities. At the low end are phone cards costing under a dollar; then there

are bank cards that will do conventional crypto, with 8/16 bit processors; at

the top end there are bank cards that will do public-key crypto and often have

32-bit processors. There are also ‘contactless’ smartcards — essentially RFID

devices that can perform cryptographic protocols rather than just emitting

static data — that are used in applications such as transport ticketing.

As these devices are cheap and sold in large volumes, an opponent can

often obtain many samples and take them away to probe at her leisure. So

many attacks on them have been developed2. Pay-TV subscriber cards in

particular have been subjected to many cloning attacks and nowadays have a

lot of special protection mechanisms. In this section, I’ll tell the story of how

smartcard security evolved with successive attacks and defenses.

2As this book was about to go to press at the end of 2007, there was an announcement at the

Chaos Computer Club conference in Berlin that the widely-used Mifare contactless system had

been reverse engineered and turned out to use a weak 48-bit stream cipher.

500 Chapter 16 ■Physical Tamper Resistance

16.6.1 History

Although smartcards are marketed as a ‘new’ security solution, they actually

go back a long way, with the early patents (which date from the late 1960s

through mid 1970s) having long since expired [383]. For a history of the

development of smartcards, see [563]. For many years, they were mostly used

in France, where much of the original development work was done with

government support. In the late 1980s and early 1990s, they started to be used

on a large scale outside France, principally as the subscriber identity modules

(SIMs) in GSM mobile phones, and as subscriber cards for pay-TV stations.

They started being used as bank cards in France in 1994, and in the rest of

Europe from about 2005.

A smartcard is a self-contained microcontroller, with a microprocessor,

memory and a serial interface integrated in a single chip and packaged in

a plastic card. Smartcards used in banking use a standard size bank card,

while in most mobile phones a much smaller size is used. Smartcard chips are

also packaged in other ways. For example, many UK prepayment electricity

meters use them packaged in a plastic key, as do Nagravision pay-TV set-top

boxes. In the STU-III secure telephones used in the U.S. government, each

user has a ‘crypto ignition key’ which is also packaged to look and feel like

a physical key. The TPM chips built into computer motherboards to support

‘Trusted Computing’ are in effect public-key smartcard chips but with added

parallel ports. Contactless smartcards contain a smartcard chip plus a wire-

loop antenna. In what follows I’ll mostly disregard the packaging form factor

and just refer to these products as ‘smartcards’ or ‘chipcards’.

Their single most widespread application is the GSM mobile phone system,

the standard in some U.S. networks and in almost all countries outside the

U.S. The telephone handsets are commodity items, and are personalized for

each user by the SIM, a smartcard that contains not just your personal phone

book, call history and so on, but also a cryptographic key with which you

authenticate yourself to the network.

The strategy of using a cheap smartcard to provide the personalisation and

authentication functions of a more expensive consumer electronic device has

a number of advantages. The expensive device can be manufactured in bulk,

with each unit being exactly the same, while the smartcard can be replaced

relatively quickly and cheaply in the event of a successful attack. This has

led many pay-TV operators to adopt smartcards. The satellite TV dish and

decoder become commodity consumer durables, while each subscriber gets

a personalized smartcard containing the key material needed to decrypt the

channels to which he has subscribed.

Chipcards are also used in a range of other applications ranging from hotel

keys to public payphones — though in such applications it’s still common to

ﬁnd low-cost devices that contain no microprocessor but just some EEPROM

16.6 Smartcards and Microcontrollers 501

memory to store a counter or certiﬁcate, and some logic to perform a simple

authentication protocol.

Devices such as prepayment electricity meters are typically built around

a microcontroller — a chip that performs the same kind of functions as a

smartcard but has less sophisticated protection. Typically, this consists of

setting a single ‘memory protection’ bit that prevents the EEPROM contents

being read out easily by an attacker. There have been many design defects

in particular products; for example, a computer authentication token called

iKey had a master password which was hashed using MD5 and stored on an

EEPROM external to the processor, so a user could overwrite it with the hash

of a known password and assume complete control of the device [719]. For

more details of attacks speciﬁc to microcontrollers, see [68, 1181, 1183].

16.6.2 Architecture

The typical smartcard consists of a single die of up to 25 square millimeters

of silicon containing a microprocessor (larger dies are more likely to break as

the card is ﬂexed). Cheap products have an 8-bit processor such as an 8051 or

6805, and the more expensive products either have a 32-bit processor such as

an ARM, or else a dedicated modular multiplication circuit to do public-key

cryptography. It also has serial I/O circuitry and a hierarchy of three classes of

memory — ROM or Flash to hold the program and immutable data, EEPROM

to hold customer-speciﬁc data such as the registered user’s name and account

number as well as crypto keys, value counters and the like; and RAM registers

to hold transient data during computation.

The memory is very limited by the standards of normal computers. A

typical card on sale in 2007 might have 64 Kbytes of ROM, 8–32 Kbytes of

EEPROM and 2K bytes of RAM. The bus is not available outside the device;

the only connections supplied are for power, reset, a clock and a serial port.

The physical, electrical and low-level logical connections, together with a

ﬁle-system-like access protocol, are speciﬁed in ISO 7816. These speciﬁcations

have not changed much since 2000; prices for volumes of a few hundred cards

haven’t changed much either (maybe $3 for an 8-bit card that will do DES and

AES, $7 for a card with a cryptoprocessor to do elliptic curve crypto, and $12

for one beefy enough for RSA). Large-volume users pay much less. The main

change over the past seven years is that development kits are now widely avail-

able, whereas in 2000 most vendors screened purchasers and imposed NDAs.

The implication is of course that if an opponent can reverse engineer your

smartcard application, she has no difﬁculty buying and programming cards.

16.6.3 Security Evolution

When I ﬁrst heard a sales pitch from a smartcard vendor — in 1986 when I

was working as a banker — I asked how come the device was secure. I was

502 Chapter 16 ■Physical Tamper Resistance

assured that since the machinery needed to make the card cost $20m, just as

for making banknotes, the system must be secure. I didn’t believe this but

didn’t then have the time or the tools to prove the claim wrong. I later learned

from industry executives that none of their customers were prepared to pay

for serious security until about 1995, and so until then they relied on the small

size of the devices, the obscurity of their design, and the rarity of chip testing

tools to make attacks more difﬁcult.

The application that changed all this was satellite TV. TV operators broadcast

their signals over a large footprint — such as all of Europe — and give each

subscriber a card that will compute the keys needed to decipher the channels

they’ve paid for. Since the operators had usually only purchased the rights to

the movies for one or two countries, they couldn’t sell the subscriber cards

elsewhere. This created a black market in pay-TV cards, into which forged

cards could be sold. A critical factor was that ‘Star Trek’, which people in

Europe had picked up from UK satellite broadcasts for years, was suddenly

encrypted in 1993. In some countries, such as Germany, it simply wasn’t

available legally at any price. This motivated a lot of keen young computer

science and engineering students to look for vulnerabilities.

There have since been large ﬁnancial frauds carried out with cloned cards.

The ﬁrst to be reported involved a smartcard used to give Portuguese farmers

rebates on fuel. The villain conspired with petrol stations that registered other

fuel sales to the bogus cards in return for a share of the proceeds. The fraud,

which took place in February/March 1995, is reported to have netted about

thirty million dollars [900].

How to hack a smartcard (1)

The earliest hacks targeted the protocols in which the cards were used. For

example, some early pay-TV systems gave each customer a card with access

to all channels, and then sent messages over the air to cancel those channels to

which the customer hadn’t subscribed after an introductory period. This

opened an attack in which a device was inserted between the smartcard and

the decoder which would intercept and discard any messages addressed to the

card. Subscribers could then cancel their subscription without the vendor

being able to cancel their service.

The same kind of attack was launched on the German phone card system. A

hacker tipped off Deutsche Telekom that it was possible to make phone cards

that gave unlimited free calls. He had discovered this by putting a laptop

between a card and a phone to analyze the trafﬁc. Telekom’s experts refused

to believe him, so he exploited his knowledge by selling handmade chip cards

in brothels and in hostels for asylum seekers [1210]. Such low-cost attacks

were particularly distressing to the phone companies as the main reason

for moving to smartcards was to cut the cost of having to validate cheaper

tokens online [124]. I’ll discuss these protocol failures further in the chapter

16.6 Smartcards and Microcontrollers 503

on copyright enforcement systems. There has also been a fairly full range of

standard computer attacks; such as stack overwriting by sending too long a

string of parameters. In what follows, we’ll concentrate on the attacks that are

peculiar to smartcards.

How to hack a smartcard (2)

As smartcards use an external power supply, and store security state such

as crypto keys and value counters in EEPROM, an attacker could freeze

the EEPROM contents by removing the programming voltage, VPP. Early

smartcards received VPP on a dedicated connection from the host interface.

This led to very simple attacks: by covering the VPP contact with sticky tape,

cardholders could prevent cancellation signals from affecting their card. The

same trick could be used with some payphone chipcards; a card with tape

over the appropriate contact had ‘inﬁnite units’.

The ﬁx was to generate VPP internally from the supply voltage VCC using a

voltage multiplier circuit. However, this isn’t entirely foolproof as the circuit

can be destroyed by an attacker. So a prudent programmer, having (for

example) decremented the retry counter after a user enters an incorrect PIN,

will read it back and check it. She will also check that memory writing actually

works each time the card is reset, as otherwise the bad guy who has shot away

the voltage multiplier can just repeatedly reset the card and try every possible

PIN, one after another.

How to hack a smartcard (3)

Another early attack was to slow down the card’s execution, or even single-

step it through a transaction by repeatedly resetting it and clocking it ntimes,

then n+1 times, and so on. In one card, it was possible to read out RAM

contents with a suitable transaction after reset, as working memory wasn’t

zeroized. With very many cards, it was possible to read the voltages on

the chip surface using an electron microscope. (The low-cost scanning elec-

tron microscopes generally available in universities can’t do voltage contrast

microscopy at more than a few tens of kilohertz, hence the need to slow down

the execution.)

So many smartcard processors have a circuit to detect low clock frequency

and either freeze or reset the card; others use dynamic logic to achieve the

same effect. But, as with burglar alarms, there is a trade-off between the false

alarm rate and the missed alarm rate. This leads to many of the alarm features

provided by smartcard chip makers simply not being used by the OEMs or

application developers. For example, with cheap card readers, there can be

wild ﬂuctuations in clock frequency when a card is powered up, causing so

many false alarms that some developers do not use the feature. So low clock

frequency detectors need careful design.

504 Chapter 16 ■Physical Tamper Resistance

How to hack a smartcard (4)

Once pay-TV operators had ﬁxed most of the simple attacks, pirates turned

to attacks using physical probing. Until a few years ago, most smartcards had

no protection against physical tampering except the microscopic scale of the

circuit, a thin glass passivation layer on the surface of the chip, and potting

which is typically some kind of epoxy. Techniques for depackaging chips are

well known, and discussed in detail in standard works on semiconductor

testing, such as [136]. In most cases, a few milliliters of fuming nitric acid are

all that’s required to dissolve the epoxy, and the passivation layer is removed

where required for probing.

Probing stations consist of microscopes with micromanipulators attached for

landing ﬁne probes on the surface of the chip. They are widely used in the

semiconductor manufacturing industry for manual testing of production line

samples, and can be obtained second hand for under $10,000 (see Figure 16.4).

They may have specialized accessories, such as a laser to shoot holes in the

chip’s passivation layer.

Figure 16.4: Low-cost probing station

16.6 Smartcards and Microcontrollers 505

The usual target of a probing attack is the processor’s bus. If the bus

trafﬁc can be recorded, this gives a trace of the program’s operation with

both code and data. If the attacker is lucky, the card designer will have

computed a checksum on memory immediately after reset (a recommended

defense industry practice) which will give him a complete listing of the card

memory contents. So the attacker will identify the bus and expose the bus

lines for probing (see Figure 16.5). Indeed, if the chip is performing a known

cryptographic protocol with well understood algorithms, then unless there’s

some defense mechanism such as lots of dummy instructions, a trace from a

single bus line is likely to give away the key [580].

The ﬁrst defense used by the pay-TV card industry against attacks of this

kind was to endow each card with multiple keys and/or algorithms, and

arrange things so that only those in current use would appear on the processor

bus. Whenever pirate cards appeared on the market, a command would be

issued over the air to cause legitimate cards to activate new keys or algorithms

from a previously unused area of memory. In this way, the pirates’ customers

would suffer a loss of service until the probing attack could be repeated and

either new pirate cards, or updates to the existing ones, could somehow be

distributed.

Figure 16.5: The data bus of an ST16 smartcard prepared for probing by excavating eight

trenches through the passivation layer with laser shots (Photo courtesy Oliver K ¨

ommerling)

506 Chapter 16 ■Physical Tamper Resistance

How to hack a smartcard (5)

The defeat for this strategy was Oliver K¨

ommerling’s memory linearization

attack in which the analyst damages the chip’s instruction decoder in such

a way that instructions which change the program address other than by

incrementing it — such as jumps and calls — become inoperable [733]. One

way to do this is to drop a grounded microprobe needle on the control line

to the instruction latch, so that whatever instruction happens to be there on

power-up is executed repeatedly. The memory contents can now be read off

the bus. In fact, once some of the device’s ROM and EEPROM is understood, the

attacker can skip over unwanted instructions and cause the device to execute

only instructions of his choice. So with a single probing needle, he can get the

card to execute arbitrary code, and in theory could get it to output its secret

key material on the serial port. But probing the memory contents off the bus is

usually more convenient.

In practice, there are often several places in the instruction decoder where a

grounded needle will have the effect of preventing programmed changes in the

control ﬂow. So even if the processor isn’t fully understood, memory lineariza-

tion can often be achieved by trial and error. Some of the more modern proces-

sors have traps which prevent memory linearization, such as hardware access

control matrices which prevent particular areas of memory being read unless

some speciﬁc sequence of commands is presented. But such circuits can often be

defeated by shooting away carefully chosen gates using a laser or an ion beam.

Memory linearization is an example of a fault induction attack. There are

quite a few examples of this; faults can be injected into smartcards and other

security processors in a number of ways, from hardware probing through

power transients to our old friend, the stack overﬂow. A variant on the theme

is given by attacks on the card’s test circuitry. A typical smartcard chip has

a self-test routine in ROM that is executed in the factory and allows all the

memory contents to be read and veriﬁed. As with FPGAs, a fuse is then blown

in the chip to stop an attacker using the same facility. All that the attacker

had to do was to cause a fault in this mechanism — by ﬁnding the fuse

and repairing it. This could be as simple as bridging it with two probing

needles [212]. A more careful design might put the test circuitry on the part of

the silicon that is sawn away when the wafer is diced into individual chips.

It’s also possible to exploit already existing hardware bugs. For example,

Adi Shamir pointed out that if a CPU has an error in its multiply unit — even

just a single computation ab =cwhose result is returned consistently wrong

in a single bit — then it’s possible to send it an RSA ciphertext for decryption

(or an RSA plaintext for signature) constructed so that the computation will

be done correctly mod pbut incorrectly mod q; if this result is returned to the

attacker, he can instantly work out the private key [1148]. For this reason, a

careful programmer will check the results of critical computations.

16.6 Smartcards and Microcontrollers 507

How to hack a smartcard (6)

The next thing the pay-TV card industry tried was to incorporate hardware

cryptographic processors, in order to force attackers to reconstruct hardware

circuits rather than simply clone software, and to force them to use more

expensive processors in their pirate cards. In the ﬁrst such implementation,

the crypto processor was a separate chip packaged into the card, and it had

an interesting protocol failure: it would always work out the key needed to

decrypt the current video stream, and then pass it to the CPU which would

decide whether or not to pass it on to the outside world. Hackers broke this

by developing a way to tap into the wiring between the two chips.

Later implementations have the crypto hardware built into the CPU itself.

Where this consists of just a few thousand gates, it is feasible for an attacker to

reconstruct the circuit manually from micrographs of the chip. But with larger

gate counts and deep submicron processes, a successful attack may require

automatic layout reconstruction: successively etching away the layers of the

chip, taking electron micrographs and using image processing software to

reconstruct a 3-d map of the chip, or at least identify its component cells [196].

However, assembling all the equipment, writing the software and integrating

the systems involves huge effort and expense.

A much simpler, and common, attack is for pirates to ask one of the half

dozen or so existing commercial reverse engineering labs to reconstruct the

relevant area of the chip. Such labs get much of their business from analyzing

integrated circuits on behalf of the chip maker’s competitors, looking for patent

infringements. They also reverse engineer chips used for accessory control in

consumer electronic devices, as doing this for compatibility rather than piracy

is quite lawful. So they are used to operating in conditions of some secrecy,

and there’s been at least one case in which a pirate had a pay-TV card reverse

engineered by an above-board company by pretending to be a bona ﬁde

business.

How to hack a smartcard (7)

The next defense the card industry thought up was to furnish the chip with

protective surface mesh, implemented in a top metal layer as a serpentine

pattern of ground, power and sensor lines. The idea was that any break or

short in the pattern would be sensed as soon as the chip was powered up and

trigger a self destruct mechanism.

We mentioned such meshes in connection with the Dallas processors; after

the usual initial crop of implementation blunders, they proved to be an effective

way of pushing up the cost of an attack. The appropriate tool to defeat them is

the Focused Ion Beam Workstation or FIB. For a detailed description of FIBs and

other semiconductor test equipment that can be used in reverse engineering,

508 Chapter 16 ■Physical Tamper Resistance

see Martin [837]; in brief, a FIB is a device similar to a scanning electron

microscope but uses a beam of ions instead of electrons. By varying the beam

current, it can be used either as a microscope or as a milling machine, with a

useful resolution under 10 nanometers. By introducing a suitable gas which is

broken down by the ion beam, it is possible to lay down either conductors or

insulators with a precision of a few tens of nanometers.

FIBs are such extremely useful devices in all sorts of applications, from

semiconductor testing through metallurgy and forensics to nanotechnology,

that they are rapidly becoming widely available and the prices are tumbling.

Many universities and industrial labs now have one, and time on them can be

rented from a number of agencies for a few hundred dollars an hour.

Given a FIB, it is straightforward to attack a sensor mesh that is not powered

up. One simply drills a hole through the mesh to the metal line that carries the

desired signal, ﬁlls it up with insulator, drills another hole through the center

of the insulator, ﬁlls it with metal, and plates a contact on top — typically a

platinum ‘L’ or ‘X’ a few microns wide, which is easy to contact with a needle

from the probing station (see Figure 16.6).

Figure 16.6: The protective mesh of an ST16 smartcard with a FIB cross for probing the

bus line visible underneath (Photo courtesy Oliver K ¨

ommerling)

16.6 Smartcards and Microcontrollers 509

Defeating a sensor mesh that is continually powered up is much harder, but

some tools exist. For example, there are techniques to mill through the back

side of a chip with a suitably equipped FIB and make contact directly to the

electronics without disturbing the sensor mesh at all.

Many other defensive techniques can force the attacker to do more work.

Some chips are packaged in much thicker glass than in a normal passivation

layer. The idea is that the obvious ways of removing this (such as hydroﬂuoric

acid) are likely to damage the chip. Other chips have protective coatings of

substances such as silicon carbide or boron nitride. (Chips with protective

coatings are on display at the NSA Museum at Fort Meade, Md). Such coatings

can force the FIB operator to go slowly rather than damage the chip through a

build-up of electrical charge.

How to hack a smartcard (8)

In 1998, the smartcard industry was shaken when Paul Kocher announced

a new attack known as differential power analysis. This relied on the fact that

different instructions consumed different amounts of power, so by measuring

the power consumed by a smartcard chip it was possible to extract the key.

This had always been known to be theoretically possible, but Kocher came

up with efﬁcient signal processing techniques that made it easy, and which

I’ll describe in the following chapter on emission security. He came up with

even simpler attacks based on timing; if cryptographic operations don’t take

the same number of clock cycles, this can leak key material too. On larger

processors, it can be even worse; a number of researchers have developed

attacks on crypto algorithms based on cache misses. Power and timing attacks

are examples of side-channel attacks, where the opponent can obtain and exploit

some extra information about the processor’s state while a cryptographic

computation is going on. Essentially all the smartcards on the market at that

time turned out to be vulnerable to power analysis or other side-channel

attacks, and this held up the industry’s development for a couple of years

while countermeasures were developed.

We classify mechanical probing as an invasive attack as it involves penetrat-

ing the passivation layer, and power analysis as a noninvasive attack as the

smartcard is left untouched; the attacker merely observes its operation. Nonin-

vasive attacks can be further classiﬁed into local attacks, where the opponent

needs access to the device, as with power analysis; and remote noninvasive

attacks, where she could be anywhere. Timing attacks, and protocol attacks,

are examples of the latter.

A lot of engineering effort was invested in the late 1990s and early 2000s

in making noninvasive attacks more difﬁcult. This is actually more complex

than it might seem, as the measures one can take to make one noninvasive

510 Chapter 16 ■Physical Tamper Resistance

attack more difﬁcult often turn out to make another one easier. I’ll discuss the

problems in more detail in the next chapter.

How to hack a smartcard (9)

Mechanical probing techniques worked until the early 2000s, but since then

they have been increasingly difﬁcult because of shrinking feature sizes. The

next attack technology to develop was optical probing. There was a ﬁrst

report from Sandia National Laboratories in 1995 of a way to read out

a voltage directly using a laser [17]. Since 2001 optical probing has been

developed into an effective and low-cost technology, largely by my colleague

Sergei Skorobogatov at Cambridge. In 2002 we reported using a photographic

ﬂashgun, mounted on the microscope of a probing station, to induce transient

faults in selected transistors of an IC [1187]. The light ionises the silicon, causing

transistors to conduct. Once you can focus the light on single transistors,

whether by using a metal mask or by upgrading from a ﬂashgun to a laser,

this enables direct attacks on many chips. For example, microcontrollers can

be opened by zapping the ﬂip-ﬂop that latches their protection state.

Later that year, Sergei reported using a laser mounted on the same

cheap microscope to read out the memory contents of a microcontroller

directly [1111]. The basic idea is simple: if you use a laser to make a transistor

conduct, this will increase the device’s power consumption unless it was con-

ducting already. By scanning the laser across the device, it is possible to make

a map of which transistors are off and which are on. We developed this into a

reasonably dependable way of reading out memory cells [1111].

How to hack a smartcard (10)

At the time of writing (2007), smartcard vendors are using 0.18 and 0.13 micron

processes which typically have seven metal layers. Direct optical probe attacks

on the logic from the surface of the chip are now difﬁcult because of the feature

size and because the metal layers get in the way, dispersing the laser light. The

faults that are induced are thus more random and less targeted. In addition,

the sheer size and complexity of the chips makes it difﬁcult to know where

to aim. This is made worse by the use of glue logic — essentially randomised

place-and-route.

As you can see in Figure 16.7, the older MC68 device has clearly distinguish-

able blocks, and quite a lot can be learned about its structure and organisation

just by looking at it. Bus lines can be picked out and targeted for attack.

However, the SX28 in Figure 16.8 just looks like a random sea of gates. The

only easily distinguishable features are the EEPROM (at top left) and the RAM

(at top right). And the SX28 is a relatively old design; more modern chips

are planarised by chemical-mechanical polishing so that even fewer details are

visible, unless infrared light is used.

16.6 Smartcards and Microcontrollers 511

Figure 16.7: MC68HC705P6A microcontroller (courtesy of Sergei Skorobogatov)

Figure 16.8: SX28 microcontroller with ‘glue logic’ (courtesy of Sergei Skorobogatov)

512 Chapter 16 ■Physical Tamper Resistance

The two current windows of vulnerability are the memory and the rear

side. Static RAM cells are still fairly large; even with 0.13 micron process an

SRAM cell is about 1.2 by 1.7 microns and can thus be individually targeted,

whether for fault induction or for read-out of its contents. Rear-side optical

attacks use the fact that silicon is transparent at wavelengths greater than 1.1

microns, while rear-side probing attacks use FIBs with special equipment for

the purpose.

16.6.4 The State of the Art

The levels of physical tamper-resistance available from the best vendors has

improved signiﬁcantly over the last few years. Much of this comes from the

smaller features, more metal layers, and greater design complexity of the latest

devices. In the 2001 edition of this book, I wrote, ‘there isn’t any technology,

or combination of technologies, known to me which can make a smartcard

resistant to penetration by a skilled and determined attacker’. This is still

almost true, but nowadays, it can take a lot of time and money to penetrate a

good design. If you can’t ﬁnd an easier way in and have to reverse engineer the

entire chip, you’ll have to cope with hundreds of thousands of transistors laid

out in randomized glue logic with perhaps seven metal layers. One approach

is to have sophisticated layout-reconstruction software; another is to send

micrographs to a subcontractor in China that just throws a lot of people at the

problem. Either way, you can be looking at a year’s delay, a budget of over a

million dollars, and no certainty of success.

I know of one case where companies spent a huge amount of money (and

several years) trying to reverse an authentication chip. That was the Sony

MagicGate, a chip found in Playstation accessories and which is recognised

by security logic in the Playstation’s graphics chip. This used some interesting

protection tricks, and an authentication protocol that was both simple (so

protocol attacks couldn’t be found) and randomised (so that attackers couldn’t

learn anything from repeating transactions). Most designs aren’t that good.

Many chips remain vulnerable to side-channel attacks such as power analysis,

which I’ll discuss in Chapter 17. Many are too complex, or become complex

over time, leading to logical vulnerabilities; I will discuss API attacks in

Chapter 18.

A ﬁnal problem is our old friend, the market for lemons. There are smartcard

products out there that give very high levels of protection, and others that

don’t. Needless to say, the vendors all claim that their products are secure.

Some of them have Common Criteria evaluations, but these are difﬁcult to

interpret; there was at least one case of a chip advertised with a very high

rating, that was easy to probe (a careful reading of the evaluation certiﬁcate

would have revealed that it applied only to the card’s operating system, not

to its hardware).

16.6 Smartcards and Microcontrollers 513

So although, at the silicon level, smartcards are usually much harder to

copy than magnetic stripe cards, the level of protection they give in an actual

application will depend critically on what products you select and how cleverly

you use them.

So what sort of strategies are available to you if you are designing a system

which depends on smartcards?

16.6.4.1 Defense in Depth

The ﬁrst, used by pay-TV companies, is defense in depth. Smartcards may

use nonstandard processors with custom instruction sets, hardware crypto-

processors, registers that use dynamic rather than static RAM to prevent

single-stepping, and obscure proprietary encryption algorithms. Normally,

using home-brewed encryption schemes is a bad thing: Kerckhoffs’ principle

almost always wins in the end, and a bad scheme, once published, can be fatal.

Defense in depth of pay-TV provides an interesting exception. The goal is to

minimize, insofar as possible, the likelihood of a shortcut attack, and to force

the attacker to go to the trouble of reverse engineering substantially the whole

system. If you are prepared to spend serious money on revenue protection,

and you have access to serious expertise, the full-custom route is the way to go.

Technical measures on their own are not enough, though. It’s important to

think through the business model. Over the last few years of the 20th century,

the pay-TV industry managed to reduce piracy losses from over 5% of revenue

to an almost negligible proportion. More complex smartcards played a role,

but much of the improvement came from legal action against pirates, and from

making technical and legal measures work together efﬁciently. (I’ll discuss this

further in the chapter on copyright.) And if you can be sure that the opponent

has to reconstruct your chip fully before he can threaten your revenue, and

you’re conﬁdent that will take him a year and a million dollars, then you can

keep him perpetutally a year behind, and this may be enough. If you also have

the ability to refresh your security quickly — say by sending your subscribers

a new smartcard — then you can undermine his revenue expectations still

further and deter him from entering into the game.

16.6.4.2 Stop Loss

Whether you go for the defense-in-depth approach will often depend on

the extent to which you can limit the losses that result from a single card’s

being successfully probed. In early pay-TV systems, the system architecture

forced all customer cards to contain the same master secret. Once this secret

became known, pirate cards can be manufactured at will, and the card base

had to be replaced. The pay-TV systems currently being deployed for digital

broadcasting use crypto protocols in which cards have different keys, so that

514 Chapter 16 ■Physical Tamper Resistance

cloned cards can be revoked. I’ll describe these protocols in section 22.2.4.3.

This makes the cards less attractive targets.

In other systems, such telephone SIM cards and EMV bank cards, each

device contains a key to authorise transactions — whether calls or debits — on

a single customer’s account. Probing a key out of a card is thus pretty well

equivalent to stealing a card, or setting up an account in the target’s name and

getting a genuine card issued against his credit rating. Attacks that cost millions

to research and many thousands per card to carry out are not usually the real

problem here. The main threat rather comes from noninvasive attacks, where

people are duped into putting their cards into wicked terminals. Although

it is conceivable that such a terminal could do a power analysis attack, in

practice it’s the protocol vulnerabilities that provide the weakest link. So it is

quite logical for banks and phone companies to use cheap commodity cards.

The system of merchant ﬂoor limits, random online authorizations, lists of hot

cards and so on, limits the possible damage. (It’s still objectionable, though, for

the banks to say ‘We use smartcards which are secure, therefore any disputed

transaction is your fault.’)

16.7 What Goes Wrong

There are failure modes of systems involving tamper-resistant processors that

are more or less independent of whether the device is a low-cost smartcard or

a high-end banking security module.

16.7.1 The Trusted Interface Problem

None of the devices described in the above sections has a really trustworthy

user interface. Some of the bank security modules have a physical lock

(or two) on the front to ensure that only the person with a given metal key (or

smartcard) can perform certain privileged transactions. But whether you use a

$2000 4758 or a $2 smartcard to do digital signatures, you still trust the PC that

drives them. If it shows you a text saying ‘Please pay amazon.com $37.99 for

a copy of Anderson’s Security Engineering’ while the message it actually sends

for signature is ‘Please remortgage my house at 13 Acacia Avenue and pay the

proceeds to Maﬁa Real Estate Inc’, then the tamper resistance has not bought

you much. Indeed, it may even make your situation worse. Banks in many

countries used to move to electronic systems as an opportunity to change their

contract terms and conditions, so as to undermine consumer protection [201].

In the UK in particular, smartcards have been more a liability engineering

technology than a protective one; complaints are answered with a standard

refrain of ‘your chip and PIN card was used, so it’s your fault.’

16.7 What Goes Wrong 515

This is usually a user interface issue, though not always. Recall the digital

tachographs we discussed in Chapter 12; after vendors made the sensor in the

truck’s gearbox into a secure processor, with its own unique crypto key, the at-

tackers build a gearbox emulator that surrounded it and fed tampered data into

the sensor. Even where you put some effort into building a user interface into a

tamper-resistant device, it may be compromised via the environment; the

classic example here is the skimmer that is ﬁtted to the card slot of an ATM,

reads your card on the way in, and captures your PIN using a tiny camera.

Tamper-resistant processors are often able to add more value where they

do not have to authenticate users or sense the environment (beyond detecting

tampering attempts). Recall the example of prepayment electricity metering, in

Chapter 13: there, tamper-resistant processors are used to limit the loss when

a token vending machine is stolen. Tokens are generated using keys kept in a

cryptoprocessor that maintains and decrypts a value counter, enforcing a credit

limit on each vending machine. Postal meters work in exactly the same way.

The critical difference is that the banking application needs to authenticate

the customer, while the electricity and postal metering applications don’t.

Other applications that use crypto chips but don’t care who uses them range

from accessory control in printer ink cartridges and games consoles to prepaid

phone cards. There, the vendor only cares that only one person uses the

product and often that they only use it so much.

16.7.2 Conﬂicts

A further set of issues is that where an application is implemented on a number

of devices, under the control of different parties, you have to consider what

happens when each party attacks the others. In banking, the card issuer, the

terminal owner and the customer are different; all the interactions of cloned

cards, bogus terminals and cheating banks need to be thought through. This is

quite different from a building entry control application, where one ﬁrm owns

the cards and the terminals; phone cards are somewhere in the middle.

Bruce Schneier and Adam Shostack suggest a framework for analysing this

in [1136]. Imagine that the customer started off with a secure PC; say something

like a palmtop computer with software to generate digital signatures. Now

consider what happens when you split this up, for example by requiring her

to use a remote keyboard and screen controlled by somebody else, or by

requiring her to have the crypto done using a smartcard whose software is

controlled by somebody else. Then think of what happens when any one of

these components is temporarily subverted: the customer’s card is stolen, or

her computer gets infected. This gives a lot of abuse cases to consider, and

designers habitually miss some of them (especially the cases that are hard to

solve with their product, or that are embarrassing to their client). And every

time you introduce a split into a system, the interface creates new possibilities

516 Chapter 16 ■Physical Tamper Resistance

for mayhem; a survey of ﬂaws found by a commercial evaluation laboratory

showed that most of them were at the interfaces between physical, logical and

organisational measures [213]. I’ll discuss this at greater length in the chapter

on API attacks.

There are further problems with displacement. It’s common for designers

to think that, just because their product contains a security processor, its

protection needs have somehow been taken care of. The security box has been

ticked. But because of interface and other issues, this is rarely so.

A further source of conﬂict and vulnerability is that many of the users of

tamper resistance are ﬁrms implementing business models that restrict their

customers’ use of their product — such as rights management and accessory

control. Their customers are therefore in some sense their enemies. They have

not only access to the product, but the incentive to tamper with it if they can.

16.7.3 The Lemons Market, Risk Dumping and Evaluation

Each of the product categories discussed here, from hardware security modules

down through smartcards to microcontrollers, has a wide range of offerings

with an extremely wide variability in the quality of protection provided.

Although quite a few products have evaluations, these don’t really mean very

much.

First, there are very few offerings at the highest protection level — FIPS-140

level 4 or Common Criteria levels above 4. There are very many at lower

levels, where the tests are fairly easy to pass, and where vendors can also shop

around for a lab that will give them an easy ride. This leads to a lemons market

in which all but the best informed and motivated players will be tempted to

go for the cheapest level 3 product, or even an unevaluated offering.

Second, evaluation certiﬁcates don’t mean what they seem. Someone buying

a 4758 in 2001 might have interpreted its level 4 evaluation to mean that it

was unbreakable — and then been startled when we broke it. In fact, the FIPS

certiﬁcate referred only to the hardware, and we found vulnerabilities in the

software. It’s happened the other way too: there’s been a smartcard with a

Common criteria level 6 evaluation, but that referred only to the operating

system — which ran on a chip with no real defences against microprobing. I’ll

discuss the failings of our evaluation systems at greater length in Part III.

Third, many ﬁrms aim at using secure processors to dump risk rather than

minimise it. The banks who say ‘your chip and PIN card was used, so it’s your

fault’ are by no means alone. There are many environments, from medicine to

defense, where what’s sought is often a certiﬁcate of security rather than real

protection, and this interacts in many ways with the ﬂaws in the evaluation

system. Indeed, the main users of security evaluation are precisely those

system operators whose focus is on due diligence rather than risk reduction,

and they are also disproportionate users of tamper-resistant processors.

16.7 What Goes Wrong 517

16.7.4 Security-By-Obscurity

Until very recently, the designers of cryptoprocessors tried hard to keep their

products’ design obscure. You had to sign an NDA to get smartcard devel-

opment tools, and until the mid-1990s there was just no serious information

available on how smartcards could be attacked. Their vendors just did not

imagine that attackers might buy semiconductor lab tools second-hand and

ﬁnd ingenious ways to use them to probe data out. The security target still

used for evaluating many smartcards under the Common Criteria focuses on

maintaining obscurity of the design. Chip masks have to be secret, staff have

to be vetted, developers have to sign NDAs — there were many requirements

that pushed up industry’s costs [448]. Obscurity was also a common require-

ment for export approval, leading to a suspicion that it covers up deliberately

inserted vulnerabilities. For example, a card we tested would always produce

the same value when instructed to generate a private / public keypair and

output the public part.

In short, the industry’s security culture was inappropriate. Almost none of

the actual attacks on ﬁelded smartcard systems used inside information. Most

of them started out with a probing attack or side-channel attack on a card

bought at retail. The industry did not do hostile attacks on its own products,

so the products were weak and were eventually subjected to hostile attack by

others. The culture is now changing and some organisations, such as VISA,

specify penetration testing [1300]. However, the incentives are still wrong; a

sensible vendor will go to whatever evaluation lab gives him the easiest ride,

rather than to an ace attack team that’ll ﬁnd dozens of extra vulnerabilities

in the product and destroy its chances of launching on time. We’ll return to

this subject to discuss the underlying economics and politics of evaluation in

section 26.3.3.1.

16.7.5 Interaction with Policy

There are many other unexpected interactions with the world of policy. A

good example was the drive that started during the dotcom boom to have

smartcards adopted as the preferred device for digital signatures where

people interact with government. The European Union passed a law that gave

a strong presumption of validity to electronic signatures made using approved

smartcards. This was irrational, given the lack of a trusted user interface; for

signing documents it would be better to use something like a PDA, where at

least the customer can at least see what she’s signing and protect the device

using common sense [111]. Yet the smartcard vendors lobbied hard and got

their law. Thankfully, this was completely counterproductive: the law moved

the liability for forged signatures from the relying party to the party whose key

was apparently used. By accepting such a device, you were in effect saying, ‘I

518 Chapter 16 ■Physical Tamper Resistance

agree to be bound by any signature that appears to have been made by this

device, regardless of whether or not I actually made it’. This is unattractive

and has helped limit the use of digital signatures to niche applications.

16.7.6 Function Creep

We’ve already seen numerous examples of how function creep, and changes in

environmental conditions, have broken secure systems by undermining their

design assumptions. I mentioned, for example, how replacing paper passports

(that are unique) with electronic passports (that are easy to copy) undermines

the bail system; criminals awaiting trial could ﬂee the country using copies of

their passports. A more general problem is when multiple applications are put

on the same card. If you deploy a phone banking system, then the SIM card

that was previously just a means of identifying people to the phone company

now becomes a token that controls access to their bank accounts. It follows

that it may attract a much higher grade of attacker. Does this matter? In the

present (2007) environment of growing phishing attacks, I’d say it probably

doesn’t; using text messages to conﬁrm bank transactions gives a valuable

second authentication channel (the main risk, as described in Chapter 2, is

probably that a fraudster talks the phone company into issuing a SIM to the

wrong person).

But what will happen in ﬁve or ten years’ time, once everyone is doing it?

What if the iPhone takes off as Apple hopes, so that everyone uses an iPhone

not just as their phone, but as their web browser? All of a sudden the two

authentication channels have shrunk to one; both the SMS messages and the

IP trafﬁc to the bank web site go through the same operating system, which is

now deployed on a large enough scale to attract attention from the gentlemen

in Russia.

16.8 So What Should One Protect?

It’s common enough that the purpose for which a technology was ﬁrst invented

or marketed isn’t the one for which it takes off. The steam engine was invented

for pumping water out of coal mines, and the telephone to enable post ofﬁce

clerks to read text to telegraph operators rather than sending it through a

pneumatic tube. Similarly, the inventors of smartcards thought they would be

a hit in banking; yet they ﬁrst took off in phone cards, then pay-TV. Despite

their use in banking in Europe, they have yet to catch on in markets like

the USA where bank customers have better legal protection. The largest sales

volumes of the lowest cost crypto chips are in accessory control applications

such as printer cartridges and console games.

16.8 So What Should One Protect? 519

So what value can tamper-resistant devices actually add?

First, they can control information processing by linking it to a single physical

token. A pay-TV subscriber card can be bought and sold in a grey market,

but so long as it isn’t copied the station operator isn’t too concerned. Another

example comes from accessory control chips such as the Sony MagicGate;

any PlayStation should work with any PlayStation memory cartridge, but not

with a cheap Korean competitor. Yet another is the use of crypto to enforce

evaluation standards in government networks: if you only get key material

once your system has been inspected and accredited, then it’s inconvenient to

connect an unlicensed system of any size to the classiﬁed government network.

This enables you to link information protection with physical and operational

protection.

Second, they can be used to control value counters, as with the electricity

prepayment and postal metering discussed in Chapter 12 and in section 16.7.1

above. These typically use devices such as the iButton to hold keys and also a

credit counter. Even if the device is stolen, the total value of the service it can

vend is limited. A related application is in printers where ink cartridges are

often programmed to dispense only so much ink and then declare themselves

to be dry.

Third, they can reduce the need to trust human operators. Their main

purpose in some government systems was ‘reducing the street value of key

material to zero’. A crypto ignition key for a STU-III should allow a thief

only to masquerade as the rightful owner, and only if he has access to an

actual STU-III device, and only so long as neither the key nor the phone

have been reported stolen. The same general considerations applied in ATM

networks: no bank wanted to make its own customers’ security depend on the

trustworthiness of the staff of another bank. In effect, they not only implement

a separation of duty policy, but transfer a good portion of the trust from

people to things. If these things can be physically controlled — whether by

their classiﬁcation or their sheer mass — that reduces the exposure from both

treachery and carelessness.

This is an incomplete list. But what these applications, and their underlying

principles, have in common is that a security property can be provided

independently of the trustworthiness of the surrounding environment. In

other words, be careful when using tamper resistant devices to try to offset the

lack of a trustworthy interface. This doesn’t mean that no value at all can be

added where the interface is problematic. For example, the tamper-resistant

crypto modules used in ATM networks cannot prevent small-scale theft

using bogus ATMs; but they can prevent large-scale PIN compromise if used

properly. In general, tamper-resistant devices are often a useful component,

but only very rarely provide a fully engineered solution.

520 Chapter 16 ■Physical Tamper Resistance

16.9 Summary

Tamper resistant devices and systems have a long history, and predate the

development of electronic computing. Computers can be protected against

physical tampering in a number of ways, such as by keeping them locked

up in a guarded room. There are also several cheaper and more portable

options, from hardware security modules that cost thousands of dollars and

are certiﬁed to resist all currently known attacks, through smartcards whose

hardware can now be quite difﬁcult to penetrate, down to low-cost security

microcontrollers that can often be defeated with a few hours to days of work.

I’ve told the story of how hardware tamper-resistance developed through

the 1990s through a series of cycles of attack and improved defence, and given

many examples of applications. Almost regardless of their price point, security

processors are typically vulnerable to attacks on interfaces (human, sensor or

system) but can often deliver value in applications where we need to link

processing to physical objects and to keep track of value in the absence of a

dependable online service.

Research Problems

There are basically two strands of research in tamper resistant processor design.

The ﬁrst concerns itself with making ‘faster, better, cheaper’ processors: how

can the protection offered by a high end device be brought to products with

mid-range prices and sizes, and how mid-range protection can be brought to

smartcards. The second concerns itself with pushing forward the state of the

attack art. How can the latest chip testing technologies be used to make ‘faster,

better, cheaper’ attacks?

These are intertwined with research into emission security and into the pro-

tection of application programming interfaces, which I’ll discuss in the next

two chapters.

Further Reading

Colleagues and I wrote a survey of security processors [65] and I’d recommend

that if you’re looking for a more detailed treatment of the material in this

chapter. Beyond that, there’s a tutorial by Sergei Skorobogatov at [1186].

The best current research in the ﬁeld usually appears in the proceedings of

CHES — the workshop on Cryptographic Hardware and Embedded Systems.

FPGA security is reviewed at [400], and the other good reads include Bunnie

Huang’s book on hacking the Xbox [629].

Further Reading

There is a shortage of open literature on Emsec. The classic van Eck article [408]

is still worth a read, and the only book on computer security (until this one) to

have a chapter on the subject is Russell and Gangemi [1098]. Our work on Soft

Tempest, Teapot and related topics can be found in [753]. For power analysis,

see the papers by Kocher [728] and by Messergues et al. [877]; more papers

appearing regularly at the CHES workshop. Joel McNamara runs an unofﬁcial

Tempest Web site at [869]. For timing and power analysis, the original papers

by Paul Kocher and colleagues are the classic references [727, 728]; there’s also

a book by Stefan Mangard, Elisabeth Oswald and Thomas Popp [833].

CHAPTER

API Attacks

One is happenstance; twice is coincidence; but three times is enemy action.

— Goldﬁnger

Simplicity is the ultimate sophistication.

— Leonardo Da Vinci

18.1 Introduction

Many supposedly secure devices have some kind of application programming

interface, or API, that untrustworthy people and processes can call in order to

get some task performed.

A bank’s server will ask an attached hardware security module ‘Here’s

a customer account number and PIN, with the PIN encrypted using the

key we share with VISA. Is the PIN correct?’

If you enable javascript, then your browser exposes an application pro-

gramming interface — javascript — which the owners of websites you

visit can use to do various things.

A secure operating system may limit the calls that an application pro-

gram can make, using a reference monitor or other wrapper to enforce

a policy such as preventing information ﬂow from High to Low.

The natural question to ask is whether it’s safe to separate tasks into a

trusted component and a less trusted one, and it’s recently been realised that

the answer is very often no. Designing security APIs is a very hard problem

indeed.

547

548 Chapter 18 ■API Attacks

API security is related to some of the problems I’ve already discussed. For

example, multilevel secure systems impose ﬂow controls on static data, while

a security API is often trying to prevent some information ﬂow in the presence

of tight and dynamic interaction — a much harder problem. It’s also related

to protocol security: as I will discuss shortly, bank security modules are often

insecure because of interactions between a number of different protocols that

they support. It touches on software security: the javascript implementation in

Firefox lets calling programs scan all variables set by existing plugins — which

may compromise your privacy by leaking information about your browsing

habits and web mail usage [543]. The javascript developers just didn’t think to

have a separate sandbox for each visited website. Indeed, the most common API

failure mode is that transactions that are secure in isolation become insecure in

combination, whether because of application syntax, feature interaction, slow

information leakage or concurrency problems.

There are many other examples. In the context of embedded systems,

for example, I discussed prepayment electricity meters; the token vending

machines use tamper-resistant security modules to protect keys and value

counters. There, an attack was to change the electricity tariff to the lowest

possible value and issue tokens that entitled recipients to power for much less

than its market value. The design error was to not bind in tariff reporting into

the end-to-end protocol design.

A potentially critical example is ‘Trusted Computing’. This initiative has

put a TPM chip for secure crypto key storage on most of the PC and Mac

motherboards shipping today. The plan, according to Microsoft, is that future

applications will have a traditional ‘insecure’ part running on top of Windows

as before, and also a ‘secure’ part or NCA that will run on top of a new security

kernel known as the Nexus, which will be formally veriﬁed and thus much

more resistant to software attacks. The Nexus and the NCA will guard crypto

keys and other critical variables for applications. The question this then raises

is how the interface between the application and the NCA is to be protected.

In short, whenever a trusted computer talks to a less trusted one, the

language they use is critical. You have to expect that the less trusted device

will try out all sorts of unexpected combinations of commands in order to trick

the more trusted one. How can we analyse this systematically?

18.2 API Attacks on Security Modules

We have learned a lot about API security from failures of the hardware

security modules used by banks to protect PINs and crypto keys for ATM

networks. In 1988, Longley and Rigby identiﬁed the importance of separating

key types while doing work for security module vendor Eracom [811]. In 1993,

18.2 API Attacks on Security Modules 549

we reported a security ﬂaw that arose from a custom transaction added to

a security module [69]. However the subject really got going in 2000 when

I started to think systematically about whether there might be a series of

transactions that one could call from a security module that would break its

security [52]. I asked: ‘So how can you be sure that there isn’t some chain of

17 transactions which will leak a clear key?’ Looking through the manuals, I

discovered the following vulnerability.

18.2.1 The XOR-To-Null-Key Attack

Hardware security modules are driven by transactions sent to them by the host

computers to which they are attached. Each transaction typically consists of a

command, a serial number, and several data items. The response contains the

serial number and several other data items. The security module contains a

number of master keys that are kept in tamper-responding memory, which is

zeroized if the device is opened. However, there is often not enough storage

in the device for all the keys it might have to use, so instead keys are stored

encrypted outside the device. Furthermore, the way in which these working

keys are encrypted provides them with a type system. For example, in the

security modules provided by VISA, a key used to derive a PIN from an

account number (as described in section 10.4.1) is stored encrypted under a

particular pair of master DES keys.

Among the VISA security module’s transactions can be found support for

generating a Terminal Master Key for an ATM. You’ll recall from Chapter 10

that ATM security is based on dual control, so a way had to be found to

generate two separate keys that could be carried from the bank to the ATM,

say by the branch manager and the branch accountant, and entered into it at a

keypad. The VISA device thus had a transaction to generate a key component

and print out its clear value on an attached security printer. It also returned

its value to the calling program, encrypted under the relevant master key KM,

which was kept in the tamper-resistant hardware:

VSM −→ printer: KMTi

VSM −→ host: {KMTi}KM

It also had another transaction that will combine two such components to

produce a terminal key:

Host −→ VSM: {KMT1}KM,{KMT2}KM

VSM −→ host: {KMT1⊕KMT2}KM

The idea was that to generate a terminal key for the ﬁrst time, you’d use

the ﬁrst of these transactions twice followed by the second. Then you’d have

KMT =KMT1exclusive-or KMT2. However, there was nothing to stop the

550 Chapter 18 ■API Attacks

programmer taking any old encrypted key and supplying it twice in the

second transaction, resulting in a known terminal key (the key of all zeroes, as

the key is exclusive-or’ed with itself):

Host −→ VSM: {KMT1}KM,{KMT1}KM

VSM −→ host: {KMT1⊕KMT1}KM

So now we have managed to insert a known key into the system. What can

be done with it? Well, the module also had a transaction to encrypt any key

supplied encrypted under KM, under any other key that itself is encrypted

under KM. Sounds complicated? Well, that’s where things break. The purpose

of this odd transaction was to enable a bank to encrypt its PIN veriﬁcation

key under a terminal master key, so it could be sent out to an ATM for ofﬂine

PIN veriﬁcation. What’s more, the key type ‘terminal master key’ and the key

type ‘PIN veriﬁcation key’ were the same. The effect of this was drastic. A

programmer can retrieve the bank’s PIN veriﬁcation key — which is also kept

outside the security module, encrypted with KM, and have it encrypted under

the zero key that he now has inserted into the system:

Host −→ VSM: {0}KM,{PIN}KM

VSM −→ host: {PIN}0

The programmer can now decrypt the PIN veriﬁcation key — the bank’s

crown jewels — and can work out the PIN for any customer account. The

purpose of the security module has been completely defeated; the bank might

as well have just worked out PINs using encryption software, or kept them in

clear on its database.

The above attack went undiscovered for a long time because it’s a bit hard to

understand the implications of ‘a transaction to encrypt any key supplied encrypted

under KM, under any other key that itself is encrypted under KM’. It was just not

clear what the various types of key in the device were suppose to do, and what

security properties the type system had to have in order to achieve its goals.

In fact, there seemed to be no formal statement of the protection goals at all;

the module had simply evolved from earlier, simpler designs as banks asked

for more features.

The next attack was found using formal methods. My student Mike Bond

built a formal model of the key types used in the device and immediately

discovered another ﬂaw. The key type ‘communications key’ is used for MAC

keys, which have the property that you can input a MAC key in the clear and

get it encrypted by — you guessed it — ‘any key that itself is encrypted under

KM’. So here was another way in which you could get a known key into the

system. You could put in an account number into the system, pretending it’s a

MAC key, and get it encrypted with the PIN veriﬁcation key — this gives you

18.2 API Attacks on Security Modules 551

the customer PIN directly. (Confused? Initially everyone was — modern APIs

are just too complicated for bugs to be evident on casual inspection. Anyway,

the full details are at [65].)

18.2.2 The Attack on the 4758

We next found a number of cryptographic attacks on the API of the VISA

module and on its IBM equivalent, the 4758 [641]. For example, they both

generate ‘check values’ for keys — a key identiﬁer calculated by encrypting a

string of zeroes under the key. This opens up a birthday attack, also known

as a time-memory tradeoff attack. Suppose you want to crack a DES key of

a type that you can’t extract from the device. Now DES keys can be broken

with about 255 effort, but that’s a lot of work. However, in the security module

architecture it’s generally enough to crack any key of a given type, as data can

be converted back and forth between them. You’re also able to generate large

numbers of keys of any given type — several hundred a second.

The attack therefore goes as follows.

1. Generate a large number of terminal master keys, and collect the check

value of each.

2. Store all the check values in a hash table.

3. Perform a brute force search, by guessing a key and encrypting the ﬁxed

test pattern with it.

4. Compare the resulting check value against all the stored check values by

looking it up in the hash table (an O(1) operation).

With a 256 keyspace, an attacker who can generate 216 target keys (which

can be done over lunchtime), a target key should be hit by luck with roughly

256/216 =240 effort (which you can do in a week or so on your PC). This is also

called a ‘meet-in-the-middle’ attack, reﬂecting the meeting of effort spent by

the HSM generating keys and effort spent by the brute-force search checking

keys.

Within a short space of time, Mike Bond had come up with an actual attack

on the 4758. This really shook the industry, as the device had been certiﬁed to

FIPS 140-1 level 4; in effect the US government had said it was unbreakable.

In addition to the meet-in-the-middle attack, he used a further obscure design

error, key replication.

As DES became vulnerable to keysearch during the 1980s, ﬁnancial institu-

tions started migrating systems to two-key triple-DES: the block was encrypted

with the left key, decrypted with the right key and then encrypted with the

left key once more. This piece of cleverness gave a backward compatibility

552 Chapter 18 ■API Attacks

mode: set the left key equal to the right key, and the encryption reverts to

single-DES. The 4758 compounded the cleverness by storing left keys and right

keys separately. IBM took precautions to stop them being confused — left keys

and right keys were encrypted differently, giving them different types — but

failed to bind together the two halves.

So an attacker could use the meet-in-the-middle trick to get two single DES

keys with known values inside a 4758, then swap the left and right halves

to get a known true triple-DES key. This could then be used to export other

valuable keys. Several variants of this attack are described in [202]; the attack

was actually implemented and demonstrated on prime-time television [306].

18.2.3 Multiparty Computation, and Differential

Protocol Attacks

The next set of attacks on security module APIs was initiated by Jolyon Clulow

in 2003 [309], and they depend on manipulating the details of the application

logic so as to leak information.

His ﬁrst attack exploited error messages. One of the PIN block formats in

common use combines the PIN and the account number by exclusive-or and

then encrypts them; this is to prevent attacks such as the one I described in

section 10.4.2 where the encrypted PIN isn’t linked to the customer account

number. However (as with combining keys) it turned out that exclusive-or was

a bad way to do this. The reason was this: if the wrong account number was

sent along with the PIN block, the device would decrypt the PIN block, xor

in the account number, discover (with reasonable probability) that the result

was not a decimal number, and return an error message. The upshot was that

by sending a series of transactions to the security module that had the wrong

account number, you could quickly work out the PIN.

An even simpler attack was then found by Mike Bond and Piotr Zielinski.

Recall the method used by IBM (and most of the industry) to generate PINs,

as in Figure 18.1:

Account number PAN:8807012345691715

PIN key KP:FEFEFEFEFEFEFEFE

Result of DES {PAN}KP:A2CE126C69AEC82D

{N}KP decimalized: 0224126269042823

Natural PIN: 0224

Offset: 6565

Customer PIN: 6789

Figure 18.1: IBM method for generating bank card PINs

18.2 API Attacks on Security Modules 553

The customer account number is encrypted using the PIN veriﬁcation key,

yielding a string of 16 hex digits. The ﬁrst four are converted to decimal digits

for use as the PIN using a user-supplied function: the decimalisation table. This

gives the natural PIN; an offset is then added whose function is to enable the

customer to select a memorable PIN. The customer PIN is the natural PIN plus

the offset. The most common decimalisation table is 012345689012345,which

just takes the DES output modulo 10.

The problem is that the decimalisation table can be manipulated. If we set

the table to all zeros (i.e., 0000000000000000) then a PIN of ’0000’ will be

generated and returned in encrypted form. We then repeat the call using the

table 1000000000000000. If the encrypted result changes, we know that the

DES output contained a 0 in its ﬁrst four digits. Given a few dozen suitably-

chosen queries, the value of the PIN can be found. Since the method compares

repeated, but slightly modiﬁed, runs of the same protocol, we called this attack

differential protocol analysis.

The industry’s initial response was to impose rules on allowable decimalisa-

tion tables. One vendor decreed, for example, that a table would have to have

at least eight different values, with no value occurring more than four times.

This doesn’t actually cut it (try 0123456789012345,then1123456789012345,

and so on). The only real solution is to rip out the decimalisation table alto-

gether; you can pay your security-module vendor extra money to sell you a

machine that’s got your own bank’s decimalisation table hard-coded.

At a philosophical level, this neatly illustrates the difﬁculty of designing a

device that will perform a secure multiparty computation — where a compu-

tation has to be performed using secret information from one party, and some

inputs that can be manipulated by a hostile party [64]1. Even in this extremely

simple case, it’s so hard that you end up having to abandon the IBM method

of PIN generation, or at least nail down its parameters so hard that you might

as well not have made those parameters tweakable in the ﬁrst place.

At a practical level, it illustrates one of the main reasons APIs fail. They get

made more and more complex, to accommodate the needs of more and more

diverse customers, until suddenly there’s an attack.

18.2.4 The EMV Attack

You’d have thought that after the initial batch of API attacks were published

in 2001, security module designers would have started being careful about

adding new transactions. Not so! Again and again, the banking industry has

demanded the addition of new transactions that add to the insecurity.

1We came across this problem in Chapter 9 where we discussed active attacks on inference

control mechanisms.

554 Chapter 18 ■API Attacks

An interesting recent example is a transaction ordered by the EMV consor-

tium to support secure messaging between a smartcard and a bank security

module. The goal is that when a bank card appears in an online transaction,

the bank can order it to change some parameter, such as a new key. So far so

good. However, the speciﬁcation had a serious error, which has appeared in a

number of implementations [12].

The transaction Secure Messaging For Keys allows the server to com-

mand the security module to encrypt a text message, followed by a key, with

a key of a type for sharing with bank smartcards. The encryption can be in

CBC or ECB mode, and the text message can be of variable length. This lets an

attacker choose a message length so that just one byte of the target key crosses

the boundary of an encryption block. That byte can then be determined by

sending a series of messages that are one byte longer, where the extra byte

cycles through all possible values until the key byte is found. The attacker

can then attack each of the other key bytes, one after another. Any exportable

key can be extracted from the module in this way. (There is also a transaction

Secure Messaging For PINs, but who needs to extract PINs one by one

if you can rip off all the master keys?)

To sum up: the security modules sold to the banking industry over the

last quarter century were almost completely insecure, because the APIs were

very badly designed. At one time or another, we found an attack on at least

one version of every security module on the market. In time the vendors

stopped or mitigated most of these attacks by shipping software upgrades.

However, the customers — the banking industry — keep on thinking up cool

new things to do with payment networks, and these keep on breaking the APIs

all over again. The typical attacks involve multiple transactions, with the

technical cause being feature interaction (with particularly carelessly designed

application features) or slow information leakage. The root cause, as with

many areas of our ﬁeld, is featuritis. People make APIs so complex that they

break, and the breaks aren’t at all obvious.

18.3 API Attacks on Operating Systems

A second class of API attacks involve concurrency, and are well illustrated by

vulnerabilities found by Robert Watson in system call wrappers [1325].

System call wrappers are used to beef up operating systems security; the

reference monitors discussed in Chapter 8 are an example, and anti-virus

software is another. Wrappers intercept calls made by applications to the

operating system, parse them, audit them, and may pass them on with

some modiﬁcation: for example, a Low process that attempts to access High

data may get diverted to dummy data or given an error message. There

are various frameworks available, including Systrace, the Generic Software

18.4 Summary 555

Wrapper Toolkit (GSWTK) and CerbNG, that enable you to write wrappers

for common operating systems. They typically execute in the kernel’s address

space, inspect the enter and exit state on all system calls, and encapsulate only

security logic.

When John Anderson proposed the concept of the reference monitor in 1972,

he stipulated that it should be tamper-proof, nonbypassable, and small enough

to verify. On the face of it, today’s wrappers are so — until you start to think

of time. Wrappers generally assume that system calls are atomic, but they’re

not; modern operating system kernels are very highly concurrent. System calls

are not atomic with respect to each other, nor are they so with respect to

wrappers. There are many possibilities for two system calls to race each other

for access to shared memory, and this gives rise to time-of-check-to-time-of-use

(TOCTTOU) attacks of the kind brieﬂy mentioned in Chapter 6.

A typical attack is to race on user-process memory by getting the kernel to

sleep at an inconvenient time, for example by a page fault. In an attack on

GSWTK, Watson calls a path whose name spills over a page boundary by one

byte, causing the kernel to sleep while the page is fetched; he then replaces the

path in memory. It turns out that the race windows are large and reliable.

Processors are steadily getting more concurrent, with more processors

shipped in each CPU chip as time passes, and operating systems are optimised

to take advantage of them. This sort of attack may become more and more of

a problem; indeed, as code analysis tools make stack overﬂows rarer, it may

well be that race conditions will become the attack of choice.

What can be done to limit them? The only real solution is to rewrite the API.

In an ideal world, operating systems would move to a message-passing model,

which would eliminate (or at least greatly reduce) concurrency issues. That’s

hardly practical for operating system vendors whose business models depend

on backwards compatibility. A pragmatic compromise is to build features

into the operating system speciﬁcally to deal with concurrency attacks; Linux

Security Modules do this, as does Mac OS/X 10.5 (based on TrustedBSD, on

which Watson worked).

In short, the APIs exposed by standard operating systems have a number

of known weaknesses, but the wrapper solution, of interposing another API

in front of the vulnerable API, looks extremely fragile in a highly concurrent

environment. The wrapper would have to understand the memory layout

fairly completely to be fully effective, and this would make it as complex (and

vulnerable) as the operating system it was trying to protect.

18.4 Summary

Interfaces get richer, and dirtier, and nastier, over time. Interface design

ﬂaws are widespread, from the world of cryptoprocessors through sundry

embedded systems right through to antivirus software and the operating

556 Chapter 18 ■API Attacks

system itself. Wherever trusted software talks to less trusted software, the

interface is likely to leak more than the designer of the trusted software

intended.

Failures tend to arise from complexity. I’ve discussed two main case

histories — cryptoprocessors, which accumulated transactions beyond the

designers’ ability to understand their interactions, and system call wrap-

pers, which try to audit and ﬁlter the calls made to an operating system’s API.

At the level of pure computer science, we could see the former as instances of

the composition problem or the secure multiparty computation problem, and

the latter as concurrency failures. However, these are industrial-scale systems

rather than blackboard systems. A security module may have hundreds of

transactions, with a fresh batch being added at each upgrade. At the appli-

cation level, many of the failures can be seen as feature interactions. There

are also speciﬁc failures such as the slow leakage of information from poorly

designed cryptosystems, which would not be serious in the case of a single

transaction but which become fatal when an opponent can commandeer a

server and inject hundreds of transactions a second.

API security is already important, and becoming more so as computers

become more complex, more diverse and more concurrent. If Microsoft ships

the original mechanisms promised for Trusted Computing with a future release

of Vista, then APIs will become more critical still — application developers

will be encouraged to write code that contains a ‘more trusted’ NCA and a

‘less trusted’ normal application. Even professionals who work for security

module companies ship APIs with serious bugs; what chance is there that

NCAs will provide value, if everyone starts designing their security APIs?

What can be done? Well, one of the lessons learned is that a ‘secure’

processor isn’t, unless the API is simple enough to understand and verify.

If you’re responsible for a bank’s cryptography, the prudent thing to do is

to have a security module adapted for your needs so that it contains only

the transactions you actually need. If this is too inconvenient or expensive,

ﬁlter the transactions and throw away all but the essential ones. (Do read the

research literature carefully before deciding which transactions are essential!)

As Watson’s work shows, complex APIs for highly-concurrent systems

probably cannot be ﬁxed in this way. If you’ve got critical applications that

depend on such a platform, then maybe you should be making migration

plans.

Finally, there will probably be a whole host of API issues with interacting

applications. As the world moves to web services that start to talk to each other,

their APIs are opened up to third-party developers — as has just happened,

for example, with Facebook. Complexity alone is bad enough; I’ll discuss in

Chapter 23 how social-networking sites in particular are pushing complexity

limits in security policy as they try to capture a signiﬁcant subset of human

Further Reading

To learn more about API security, you should ﬁrst read our survey papers [63,

64, 65] as well as Robert Watson’s paper on concurrency attacks [1325]. There is

now an annual conference, the International Workshop on Analysis of Security

APIs, where you can catch up with the latest research.

CHAPTER

Electronic and Information

Warfare

All warfare is based on deception ... hold out baits to entice the enemy. Feign

disorder, and crush him.

— Sun Tzu, The Art of War, 1.18–20

Force, and Fraud, are in warre the two Cardinal Virtues.

— Thomas Hobbes

19.1 Introduction

For decades, electronic warfare has been a separate subject from computer

security, even though they use some common technologies (such as cryp-

tography). This is starting to change as elements of the two disciplines fuse

to form the new subject of information warfare. The Pentagon’s embrace of

information warfare as a slogan in the last years of the twentieth century

established its importance — even if its concepts, theory and doctrine are still

underdeveloped. The Russian denial-of-service attacks on Estonia in 2007 have

put it ﬁrmly on many policy agendas — even though it’s not clear that these

attacks were conducted by the Russian government; as far as we know, it may

have been just a bunch of Russian hackers.

There are other reasons why a knowledge of electronic warfare is important

to the security engineer. Many technologies originally developed for the war-

rior have been adapted for commercial use, and instructive parallels abound.

The struggle for control of the electromagnetic spectrum has consumed so

many clever people and so many tens of billions of dollars that we ﬁnd decep-

tion strategies and tactics of a unique depth and subtlety. It is the one area

559

560 Chapter 19 ■Electronic and Information Warfare

of electronic security to have experienced a lengthy period of coevolution of

attack and defense involving capable motivated opponents.

Electronic warfare is also our main teacher when it comes to service-denial

attacks, a topic that computer security people ignored for years. It suddenly

took center stage a few years ago thanks to denial-of-service attacks on

commercial web sites, and when blackmailers started taking down Internet

gambling sites and demanding to be paid off, it got serious.

As I develop this discussion, I’ll try to draw out the parallels between

electronic warfare and other information security problems. In general, while

people say that computer security is about conﬁdentiality, integrity and avail-

ability, electronic warfare has this reversed and back-to-front. The priorities are:

1. denial of service, which includes jamming, mimicry and physical attack;

2. deception, which may be targeted at automated systems or at people;

and

3. exploitation, which includes not just eavesdropping but obtaining any

operationally valuable information from the enemy’s use of his elec-

tronic systems.

19.2 Basics

The goal of electronic warfare is to control the electromagnetic spectrum. It is

generally considered to consist of

electronic attack, such as jamming enemy communications or radar, and

disrupting enemy equipment using high-power microwaves;

electronic protection, which ranges from designing systems resistant to

jamming, through hardening equipment to resist high-power microwave

attack, to the destruction of enemy jammers using anti-radiation mis-

siles; and

electronic support, which supplies the necessary intelligence and threat

recognition to allow effective attack and protection. It allows comman-

ders to search for, identify and locate sources of intentional and

unintentional electromagnetic energy.

These deﬁnitions are taken from Schleher [1121]. The traditional topic of

cryptography, namely communications security (Comsec), is only a small part

of electronic protection, just as it is becoming only a small part of informa-

tion protection in more general systems. Electronic support includes signals

intelligence, or Sigint, which consists of communications intelligence (Comint)

and electronic intelligence (Elint). The former collects enemy communications,

19.3 Communications Systems 561

including both message content and trafﬁc data about which units are com-

municating, while the latter concerns itself with recognizing hostile radars and

other non-communicating sources of electromagnetic energy.

Deception is central to electronic attack. The goal is to mislead the enemy

by manipulating his perceptions in order to degrade the accuracy of his

intelligence and target acquisition. Its effective use depends on clarity about

who (or what) is to be deceived, about what and how long, and — where the

targets of deception are human — the exploitation of pride, greed, laziness

and other vices. Deception can be extremely cost effective and is increasingly

relevant to commercial systems.

Physical destruction is an important part of the mix; while some enemy

sensors and communications links may be neutralized by jamming (so-called

soft kill), others will often be destroyed (hard kill). Successful electronic warfare

depends on using the available tools in a coordinated way.

Electronic weapon systems are like other weapons in that there are sensors,

such as radar, infrared and sonar; a communications links which take sensor

data to the command and control center; and output devices such as jammers,

lasers, missiles, bombs and so on. I’ll discuss the communications system

issues ﬁrst, as they are the most self-contained, then the sensors and associated

jammers, and ﬁnally other devices such as electromagnetic pulse generators.

Once we’re done with e-war, we’ll look at the lessons we might take over to

i-war.

19.3 Communications Systems

Military communications were dominated by physical dispatch until about

1860, then by the telegraph until 1915, and then by the telephone until

recently [923]. Nowadays, a typical command and control structure is made

up of various tactical and strategic radio networks supporting data, voice and

images, operating over point-to-point links and broadcast. Without situational

awareness and the means to direct forces, the commander is likely to be

ineffective. But the need to secure communications is much more pervasive

than one might at ﬁrst realize, and the threats are much more diverse.

One obvious type of trafﬁc is the communications between ﬁxed sites

such as army headquarters and the political leadership. A signiﬁcant his-

torical threat here was that the cipher security might be penetrated and

the orders, situation reports and so on compromised, whether as a result

of cryptanalysis or — more likely — equipment sabotage, subversion of

personnel or theft of key material. The insertion of deceptive messages

may also be a threat in some circumstances. But cipher security will often

include protection against trafﬁc analysis (such as by link encryption) as

562 Chapter 19 ■Electronic and Information Warfare

well as of the transmitted message conﬁdentiality and authenticity. The

secondary threat is that the link might be disrupted, such as by destruc-

tion of cables or relay stations.

There are more stringent requirements for communications with covert

assets such as agents in the ﬁeld. Here, in addition to cipher security

issues, location security is important. The agent will have to take steps to

minimize the risk of being caught as a result of communications mon-

itoring. If he sends messages using a medium which the enemy can

monitor, such as the public telephone network or radio, then much of his

effort may go into frustrating trafﬁc analysis and radio direction ﬁnding.

Tactical communications, such as between HQ and a platoon in the ﬁeld,

also have more stringent (but slightly different) needs. Radio direction

ﬁnding is still an issue, but jamming may be at least as important, and

deliberately deceptive messages may also be a problem. For example,

there is equipment that enables an enemy air controller’s voice com-

mands to be captured, cut into phonemes and spliced back together into

deceptive commands, in order to gain a tactical advantage in air com-

bat [506]. As voice morphing techniques are developed for commercial

use, the risk of spooﬁng attacks on unprotected communications will

increase. So cipher security may include authenticity as well as conﬁden-

tiality and covertness.

Control and telemetry communications, such as signals sent from an

aircraft to a missile it has just launched, must be protected against jam-

ming and modiﬁcation. It would also be desirable if they could be covert

(so as not to trigger a target’s warning receiver) but that is in tension

with the power levels needed to defeat defensive jamming systems.

One solution is to make the communications adaptive — to start off in

a low-probability-of-intercept mode and ramp up the power if needed in

response to jamming.

So the protection of communications will require some mix, depending on

the circumstances, of content secrecy, authenticity, resistance to trafﬁc analysis

and radio direction ﬁnding, and resistance to various kinds of jamming. These

interact in some rather unobvious ways. For example, one radio designed for

use by dissident organizations in Eastern Europe in the early 1980s operated

in the radio bands normally occupied by the Voice of America and the BBC

World Service — which were routinely jammed by the Russians. The idea was

that unless the Russians were prepared to turn off their jammers, they would

have difﬁculty doing direction ﬁnding.

Attack also generally requires a combination of techniques — even where

the objective is not analysis or direction ﬁnding but simply denial of service.

19.3 Communications Systems 563

Owen Lewis sums it up succinctly: according to Soviet doctrine, a comprehen-

sive and successful attack on a military communications infrastructure would

involve destroying one third of it physically, denying effective use of a second

third through techniques such as jamming, trojans or deception, and then

allowing the adversary to disable the remaining third by attempting to pass

all his trafﬁc over a third of his installed capacity [789]. This applies even in

guerilla wars; in Malaya, Kenya and Cyprus the rebels managed to degrade

the telephone system enough to force the police to set up radio nets [923].

NATO developed a comparable doctrine, called Counter-Command, Control

and Communications operations (C-C3, pronounced C C cubed), in the 80s. It

achieved its ﬁrst ﬂowering in the Gulf War. Of course, attacking an army’s

command structures is much older than that; it’s a basic principle to shoot at

an ofﬁcer before shooting at his men.

19.3.1 Signals Intelligence Techniques

Before communications can be attacked, the enemy’s network must be mapped.

The most expensive and critical task in signals intelligence is identifying and

extracting the interesting material from the cacophony of radio signals and the

huge mass of trafﬁc on systems such as the telephone network and the Internet.

The technologies in use are extensive and largely classiﬁed, but some aspects

are public.

In the case of radio signals, communications intelligence agencies use

receiving equipment, that can recognize a huge variety of signal types, to

maintain extensive databases of signals — which stations or services use which

frequencies. In many cases, it is possible to identify individual equipment by

signal analysis. The components can include any unintentional frequency

modulation, the shape of the transmitter turn-on transient, the precise center

frequency and the ﬁnal-stage ampliﬁer harmonics. This RF ﬁngerprinting,or

RFID, technology was declassiﬁed in the mid-1990s for use in identifying

cloned cellular telephones, where its makers claim a 95% success rate [534,

1121]. It is the direct descendant of the World War 2 technique of recognizing

a wireless operator by his ﬁst — the way he used Morse Code [836].

Radio Direction Finding (RDF) is also critical. In the old days, this involved

triangulating the signal of interest using directional antennas at two monitoring

stations. So spies might have several minutes to send a message home before

having to move. Modern monitoring stations use time difference of arrival

(TDOA) to locate a suspect signal rapidly, accurately and automatically by

comparing the phase of the signals received at two sites; anything more than

a second or so of transmission can be a giveaway.

Trafﬁc analysis — looking at the number of messages by source and desti-

nation — can also give very valuable information, not just about imminent

attacks (which were signalled in World War 1 by a greatly increased volume of

564 Chapter 19 ■Electronic and Information Warfare

radio messages) but also about unit movements and other more routine mat-

ters. However, trafﬁc analysis really comes into its own when sifting through

trafﬁc on public networks, where its importance (both for national intelligence

and police purposes) is difﬁcult to overstate. Until a few years ago, trafﬁc

analysis was the domain of intelligence agencies — when NSA men referred

to themselves as ‘hunter-gatherers’, trafﬁc analysis was much of the ‘hunting’.

In the last few years, however, trafﬁc analysis has come out of the shadows

and become a major subject of study.

One of the basic techniques is the snowball search. If you suspect Alice

of espionage (or drug dealing, or whatever), you note everyone she calls,

and everyone who calls her. This gives you a list of dozens of suspects.

You eliminate the likes of banks and doctors, who receive calls from too

many people to analyze (your whitelist), and repeat the procedure on each

remaining number. Having done this procedure recursively several times,

you have a mass of thousands of contacts — they accumulate like a snowball

rolling downhill. You now sift the snowball you’ve collected — for example,

for people already on one of your blacklists, and for telephone numbers that

appear more than once. So if Bob, Camilla and Donald are Alice’s contacts,

with Bob and Camilla in contact with Eve and Donald and Eve in touch

with Farquhar, then all of these people may be considered suspects. You now

draw a friendship tree which gives a ﬁrst approximation to Alice’s network,

and reﬁne it by collating it with other intelligence sources. Covert community

detection has become a very hot topic since 9/11, and researchers have tried all

sorts of hierarchical clustering and graph partitioning methods to the problem.

As of 2007, the leading algorithm is by Mark Newman [966]; it uses spectral

methods to partition a network into its natural communities so as to maximise

modularity.

But even given good mathematical tools for analysing abstract networks,

reality is messier. People can have several numbers, and people share numbers.

When conspirators take active countermeasures, it gets harder still; Bob might

get a call from Alice at his work number and then call Eve from a phone

box. (If you’re running a terrorist cell, your signals ofﬁcer should get a job at

a dentist’s or a doctor’s or some other place that’s likely to be whitelisted.)

Also, you will need some means of correlating telephone numbers to people.

Even if you have access to the phone company’s database of unlisted numbers,

prepaid mobile phones can be a serious headache, as can cloned phones and

hacked PBXs. Tying IP addresses to people is even harder; ISPs don’t always

keep the Radius logs for long. I’ll discuss all these issues in more detail in

later chapters; for now, I’ll just remark that anonymous communications aren’t

new. There have been letter boxes and public phone booths for generations.

But they are not a universal answer for the crook as the discipline needed

to use anonymous communications properly is beyond most criminals. It’s

reported, for example, that the 9/11 mastermind Khalid Sheikh Mohammed

19.3 Communications Systems 565

was caught after he used in his mobile phone in Pakistan a prepaid SIM card

that had been bought in Switzerland in the same batch as a SIM that had been

used in another Al-Qaida operation.

Signals collection is not restricted to getting phone companies to give access

to the content of phone calls and the itemised billing records. It also involves a

wide range of specialized facilities ranging from expensive ﬁxed installations

that copy international satellite links, down to temporary tactical arrange-

ments. A book by Nicky Hagar [576] describes the main ﬁxed collection

network operated by the USA, Canada, the UK, Australia and New Zealand.

Known as Echelon, this consists of a number of ﬁxed collection stations that

monitor international phone, fax and data trafﬁc with computers called dic-

tionaries which search passing trafﬁc for interesting phone numbers, network

addresses and machine-readable content; this is driven by search strings

entered by intelligence analysts. One can think of this as a kind of Google for

the world’s phone system (though given the data volumes nowadays, content

generally has to be selected in real time; not even the NSA can afford to store

all the data on the Internet and the phone networks).

This ﬁxed network is supplemented by tactical collection facilities as needed;

Hagar describes, for example, the dispatch of Australian and New Zealand

navy frigates to monitor domestic communications in Fiji during military

coups in the 1980s. Koch and Sperber discuss U.S. and German installations

in Germany in [725]; Fulghum describes airborne signals collection in [506];

satellites are also used to collect signals, and there are covert collection facilities

too that are not known to the host country.

But despite all this huge capital investment, the most difﬁcult and expensive

part of the whole operation is trafﬁc selection rather than collection [770]. Thus,

contrary to one’s initial expectations, cryptography can make communications

more vulnerable rather than less (if used incompetently, as it usually is). If

you just encipher all the trafﬁc you consider to be important, you have thereby

marked it for collection by the enemy. And if your cryptosecurity were perfect,

you’ve just helped the enemy map your network, which means he can collect

all the unencrypted trafﬁc that you share with third parties.

Now if everyone encrypted all their trafﬁc, then hiding trafﬁc could be

much easier (hence the push by signals intelligence agencies to prevent the

widespread use of cryptography, even if it’s freely available to individuals).

This brings us to the topic of attacks.

19.3.2 Attacks on Communications

Once you have mapped the enemy network, you may wish to attack it. People

often talk in terms of ‘codebreaking’ but this is a gross oversimpliﬁcation.

First, although some systems have been broken by pure cryptanalysis, this

is fairly rare. Most production attacks have involved theft of key material, as

566 Chapter 19 ■Electronic and Information Warfare

when the State Department code book was stolen during World War 2 by the

valet of the American ambassador to Rome, or errors in the manufacture and

distribution of key material, as in the ‘Venona’ attacks on Soviet diplomatic

trafﬁc [676]. Even where attacks based on cryptanalysis have been possible,

they have often been made much easier by operational errors, an example

being the attacks on the German Enigma trafﬁc during World War 2 [677].

The pattern continues to this day. The history of Soviet intelligence during the

Cold War reveals that the USA’s technological advantage was largely nulliﬁed

by Soviet skills in ‘using Humint in Sigint support’ — which largely consisted

of recruiting traitors who sold key material, such as the Walker family [77].

Second, access to content is often not the desired result. In tactical situations,

the goal is often to detect and destroy nodes, or to jam the trafﬁc. Jamming can

involve not just noise insertion but active deception. In World War 2, the Allies

used German speakers as bogus controllers to send German nightﬁghters

confusing instructions, and there was a battle of wits as authentication tech-

niques were invented and defeated. More recently, as I noted in the chapter on

biometrics, the U.S. Air Force has deployed more sophisticated systems based

on voice morphing. I mentioned in an earlier chapter the tension between

intelligence and operational units: the former want to listen to the other side’s

trafﬁc, and the latter to deny them its use [103]. Compromises between these

goals can be hard to ﬁnd. It’s not enough to jam the trafﬁc you can’t read as

that tells the enemy what you can read!

Matters can be simpliﬁed if the opponent uses cryptography — especially

if they’re competent and you can’t read their encrypted trafﬁc. This removes

the ops/intel tension, and you switch to RDF or the destruction of protected

links as appropriate. This can involve the hard-kill approach of digging up

cables or bombing telephone exchanges (both of which the Allies did during

the Gulf War), the soft-kill approach of jamming, or whatever combination

of the two is economic. Jamming is useful where a link is to be disrupted for

a short period, but is often expensive; not only does it tie up facilities, but the

jammer itself becomes a target. Cases where it is more effective than physical

attack include satellite links, where the uplink can often be jammed using a

tight beam from a hidden location using only a modest amount of power.

The increasing use of civilian infrastructure, and in particular the Internet,

raises the question of whether systematic denial-of-service attacks might be

used to jam trafﬁc. (There were anecdotes during the Bosnian war of Serbian

information warfare cells attempting to DDoS NATO web sites.) This threat

is still considered real enough that many Western countries have separate

intranets for government and military use.

19.3 Communications Systems 567

19.3.3 Protection Techniques

As should be clear from the above, communications security techniques

involve not just protecting the authenticity and conﬁdentiality of the con-

tent — which can be achieved in a relatively straightforward way by encryp-

tion and authentication protocols — but also preventing trafﬁc analysis,

direction ﬁnding, jamming and physical destruction. Encryption can stretch

to the ﬁrst of these if applied at the link layer, so that all links appear to

have a constant-rate pseudorandom bitstream on them at all times, regardless

of whether there is any message trafﬁc. But link layer encryption alone is

not always enough, as enemy capture of a single node might put the whole

network at risk.

Encryption alone cannot protect against RDF, jamming, and the destruction

of links or nodes. For this, different technologies are needed. The obvious

solutions are:

redundant dedicated lines or optical ﬁbers;

highly directional transmission links, such as optical links using infrared

lasers or microwave links using highly directional antennas and ex-

tremely high frequencies;

low-probability-of-intercept (LPI), low-probability-of-position-ﬁx (LPPF) and

anti-jam radio techniques.

The ﬁrst two of these options are fairly straightforward to understand, and

where they are feasible they are usually the best. Cabled networks are very

hard to destroy completely, unless the enemy knows where the cables are and

has physical access to cut them. Even with massive artillery bombardment, the

telephone network in Stalingrad remained in use (by both sides) all through

the siege.

The third option is a substantial subject in itself, which I will now describe

(albeit only brieﬂy).

A number of LPI/LPPF/antijam techniques go under the generic name of

spread spectrum communications. They include frequency hoppers,direct sequence

spread spectrum (DSSS) and burst transmission. From beginnings around World

War 2, spread spectrum has spawned a substantial industry and the technology

(especially DSSS) has been applied to numerous other problems, ranging from

high resolution ranging (in the GPS system) through copyright marks in digital

images (which I’ll discuss later). I’ll look at each of these three approaches in

turn.

568 Chapter 19 ■Electronic and Information Warfare

19.3.3.1 Frequency Hopping

Frequency hoppers are the simplest spread spectrum systems to understand

and to implement. They do exactly as their name suggests — they hop rapidly

from one frequency to another, with the sequence of frequencies deter-

mined by a pseudorandom sequence known to the authorized principals.

They were invented, famously, over dinner in 1940 by actress Hedy Lamarr

and screenwriter George Antheil, who devised the technique as a means

of controlling torpedos without the enemy detecting them or jamming their

transmissions [763]. A frequency hopping radar was independently developed

at about the same time by the Germans [1138].

Hoppers are resistant to jamming by an opponent who doesn’t know the hop

sequence. If the hopping is slow and a nearby opponent has capable equipment,

then an option might be follower jamming — observing the signal and following

it around the band, typically jamming each successive frequency with a single

tone. However if the hopping is fast enough, or propagation delays are exces-

sive, the opponent may have to jam much of the band, which requires much

more power. The ratio of the input signal’s bandwidth to that of the trans-

mitted signal is called the process gain of the system; thus a 100 bit/sec signal

spread over 10MHz has a process gain of 107/102=105= 50dB. The jamming

margin, which is deﬁned as the maximum tolerable ratio of jamming power to

signal power, is essentially the process gain modulo implementation and other

losses (strictly speaking, process gain divided by the minimum bit energy-to-

noise density ratio). The optimal jamming strategy, for an opponent who can’t

predict or effectively follow the hop sequence, is partial band jamming —tojam

enough of the band to introduce an unacceptable error rate in the signal.

Frequency hopping is used in some civilian applications, such as Bluetooth,

where it gives a decent level of interference robustness at low cost. On the

military side of things, although hoppers can give a large jamming margin,

they give little protection against direction ﬁnding. A signal analysis receiver

that sweeps across the frequency band of interest will usually intercept them

(and depending on the relevant bandwidths, sweep rate and dwell time, it

might intercept a hopping signal several times).

Since frequency hoppers are simple to implement and give a useful level

of jam-resistance, they are often used in combat networks, such as man pack

radios, with hop rates of 50–500 per second. To disrupt these communications,

the enemy will need a fast or powerful jammer, which is inconvenient for the

battleﬁeld. Fast hoppers (deﬁned in theory as having hop rates exceed-

ing the bit rate; in practice, with hop rates of 10,000 per second or more) can

pass the limit of even large jammers. Hoppers are less ‘LPI’ than the techniques

I’ll describe next, as an opponent with a sweep receiver can detect the presence

of a signal; and slow hoppers have some vulnerability to eavesdropping and

direction ﬁnding, as an opponent with suitable wideband receiving equipment

can often follow the signal.

19.3 Communications Systems 569

19.3.3.2 DSSS

In direct sequence spread spectrum, we multiply the information-bearing

sequence by a much higher rate pseudorandom sequence, usually generated

by some kind of stream cipher (see Figures 19.1 and 19.2). This spreads the

spectrum by increasing the bandwidth. The technique was ﬁrst described by

a Swiss engineer, Gustav Guanella, in a 1938 patent application [1138], and

developed extensively in the USA in the 1950s. Its ﬁrst deployment in anger

was in Berlin in 1959.

Like hopping, DSSS can give substantial jamming margin (the two systems

have the same theoretical performance). But it can also make the signal

signiﬁcantly harder to intercept. The trick is to arrange things so that at the

intercept location, the signal strength is so low that it is lost in the noise ﬂoor

unless the opponent knows the spreading sequence with which to recover it.

Of course, it’s harder to do both at the same time, since an antijam signal

should be high power and an LPI/LPPF signal low power; the usual tactic is

to work in LPI mode until detected by the enemy (for example, when coming

within radar range) and then boost transmitter power into antijam mode.

Nbits

N*R bits

Narrow band original signal

Over sampled original signal

Spread signal

Wide band pseudonoise

XOR

Figure 19.1: Spreading in DSSS (courtesy of Roche and Dugelay)

Wide band pseudonoise

Spread signal

Demodulated signal

Restored signal

XOR

Figure 19.2: Unspreading in DSSS (courtesy of Roche and Dugelay)

570 Chapter 19 ■Electronic and Information Warfare

There is a large literature on DSSS, and the techniques have now been

taken up by the commercial world as code division multiple access (CDMA) in

various mobile radio and phone systems. Third-generation mobile phones in

particular rely on CDMA for their performance.

DSSS is sometimes referred to as ‘encrypting the RF’ and it comes in a

number of variants. For example, when the underlying modulation scheme is

FM rather than AM it’s called chirp. The classic introduction to the underlying

mathematics and technology is [1026]; the engineering complexity is higher

than with frequency hop for various reasons. For example, synchronization is

particularly critical. One strategy is to have your users take turns at providing

a reference signal. If your users have access to a reference time signal (such

as GPS, or an atomic clock) you might rely on this; but if you don’t control

GPS, you may be open to synchronization attacks, and even if you do the GPS

signal might be jammed. It was reported in 2000 that the French jammed

GPS in Greece in an attempt to sabotage a British bid to sell 250 tanks to the

Greek government, a deal for which France was a competitor. This caused

the British tanks to get lost during trials. When the ruse was discovered, the

Greeks found it all rather amusing [1269]. Now GPS jammers are commodity

items, and I’ll discuss them in more detail below.

19.3.3.3 Burst Communications

Burst communications, as their name suggests, involve compressing the data

and transmitting it in short bursts at times unpredictable by the enemy. They

are also known as time-hop. They are usually not so jam-resistant (except insofar

as the higher data rate spreads the spectrum) but can be even more difﬁcult

to detect than DSSS; if the duty cycle is low, a sweep receiver can easily miss

them. They are often used in radios for special forces and intelligence agents.

Really high-grade room bugs often use burst.

An interesting variant is meteor burst transmission (also known as meteor

scatter). This relies on the billions of micrometeorites that strike the Earth’s

atmosphere each day, each leaving a long ionization trail that persists for

typically a third of a second and provides a temporary transmission path

between a mother station and an area of maybe a hundred miles long and a

few miles wide. The mother station transmits continuously; whenever one of

the daughters is within such an area, it hears mother and starts to send packets

of data at high speed, to which mother replies. With the low power levels used

in covert operations one can achieve an average data rate of about 50 bps, with

an average latency of about 5 minutes and a range of 500–1500 miles. With

higher power levels, and in higher latitudes, average data rates can rise into

the tens of kilobits per second.

As well as special forces, the USAF in Alaska uses meteor scatter as backup

communications for early warning radars. It’s also used in civilian applications

19.3 Communications Systems 571

such as monitoring rainfall in remote parts of the third world. In niche markets

where low bit rates and high latency can be tolerated, but where equipment

size and cost are important, meteor scatter can be hard to beat. The technology

is described in [1120].

19.3.3.4 Combining Covertness and Jam Resistance

There are some rather complex tradeoffs between different LPI, LPPF and

jam resistance features, and other aspects of performance such as resistance

to fading and multipath, and the number of users that can be accommodated

simultaneously. They also behave differently in the face of specialized jam-

ming techniques such as swept-frequency jamming (where the jammer sweeps

repeatedly through the target frequency band) and follower. Some types of

jamming translate between different modes: for example, an opponent with

insufﬁcient power to block a signal completely can do partial time jamming

on DSSS by emitting pulses that cover a part of its utilized spectrum, and on

frequency hop by partial band jamming.

There are also engineering tradeoffs. For example, DSSS tends to be about

twice as efﬁcient as frequency hop in power terms, but frequency hop gives

much more jamming margin for a given complexity of equipment. On the

other hand, DSSS signals are much harder to locate using direction ﬁnding

techniques [461].

System survivability requirements can impose further constraints. It may be

essential to prevent an opponent who has captured one radio and extracted its

current key material from using this to jam a whole network.

So a typical modern military system will use some combination of tight

beams, DSSS, hopping and burst.

The Jaguar tactical radio used by UK armed forces hops over one of nine

6.4 MHz bands, and also has an antenna with a steerable null which can

be pointed at a jammer or at a hostile intercept station.

Both DSSS and hopping are used with TDMA in Joint Tactical Information

Distribution System (JTIDS) — a U.S. data link system used by AWACS to

communicate with ﬁghters [1121]. TDMA separates transmission from

reception and lets users know when to expect their slot. It has a DSSS sig-

nal with a 57.6 KHz data rate and a 10 MHz chip rate (and so a jamming

margin of 36.5 dB), which hops around in a 255 MHz band with mini-

mum jump of 30 MHz. The hopping code is available to all users, while

the spreading code is limited to individual circuits. The rationale is that

if an equipment capture leads to the compromise of the spreading code,

this would allow jamming of only a single 10MHz band, not the full

255 MHz.

572 Chapter 19 ■Electronic and Information Warfare

MILSTAR is a U.S. satellite communications system with 1 degree beams

from a geostationary orbit (20 GHz down, 44 GHz up). The effect of the

narrow beam is that users can operate within three miles of the enemy

without being detected. Jam protection is from hopping: its channels hop

several thousand times a second in bands of 2 GHz.

A system designed to control MX missiles is described in [530] and gives

an example of extreme survivability engineering. To be able to withstand

a nuclear ﬁrst strike, the system had to withstand signiﬁcant levels of

node destruction, jamming and atmospheric noise. The design adopted

was a frequency hopper at 450 KHz with a dynamically reconﬁgurable

network. It was not in the end deployed.

French tactical radios have remote controls. The soldier can use the

handset a hundred yards from the radio. This means that attacks on the

high-power emitter don’t have to endanger the troops so much [348].

There are also some system level tricks, such as interference cancellation —

here the idea is to communicate in a band which you are jamming and whose

jamming waveform is known to your own radios, so they can cancel it out or

hop around it. This can make jamming harder for the enemy by forcing him

to spread his available power over a larger bandwidth, and can make signals

intelligence harder too [1074].

19.3.4 Interaction Between Civil and Military Uses

Civil and military uses of communications are increasingly intertwined. Oper-

ation Desert Storm (the First Gulf War against Iraq) made extensive use of the

Gulf States’ civilian infrastructure: a huge tactical communications network

was created in a short space of time using satellites, radio links and leased

lines, and experts from various U.S. armed services claim that the effect of

communications capability on the war was absolutely decisive [634]. It can

be expected that both military and substate groups will attack civilian infras-

tructure to deny it to their opponents. Already, as I noted, satellite links are

vulnerable to uplink jamming.

Another example of growing interdependency is given by the Global Posi-

tioning System, GPS. This started off as a U.S. military navigation system and

had a selective availability feature that limited the accuracy to about a hundred

yards unless the user had the relevant cryptographic key. This had to be turned

off during Desert Storm as there weren’t enough military GPS sets to go round

and civilian equipment had to be used instead. As time went on, GPS turned

out to be so useful, particularly in civil aviation, that the FAA helped ﬁnd ways

to defeat selective availability that give an accuracy of about 3 yards compared

19.3 Communications Systems 573

with a claimed 8 yards for the standard military receiver [431]. Finally, in May

2000, President Clinton announced the end of selective availability. Various

people have experimented with jamming GPS, which turns out to be not that

difﬁcult, and there has been some discussion of the systemic vulnerabilities

that result from overreliance on it [490].

The U.S. government still reserves the right to switch off GPS, or to introduce

errors into it, for example if terrorists were thought to be using it. But many

diverse systems now depend on GPS, and many of them have motivated

opponents; some countries are starting to use GPS to do road pricing, or

to enforce parole terms on released prisoners via electronic ankle bracelets.

As a result, GPS jammers appeared in car magazines in 2007 for $700; the

price is bound to come down as truck drivers try to cheat road toll systems

and car drivers try to beat pay-as-you-drive insurance schemes. Once their

use becomes widespread, the consequences could be startling for other GPS

users. Perhaps the solution lies in diversity: Russia has a separate navigation

satellite system, and Europe’s thinking of building one. Anyway, the security

of navigation signals is starting to become a topic of research [751].

The civilian infrastructure also provides some defensive systems that gov-

ernment organizations (especially in the intelligence ﬁeld) use. I mentioned

the prepaid mobile phone, which provides a fair degree of anonymity; secure

web servers offer some possibilities; and another example is the anonymous

remailer — a device that accepts encrypted email, decrypts it, and sends it on to

a destination contained within the outer encrypted envelope. The Tor network,

pioneered by the U.S. Navy, does much the same for web pages, providing

a low-latency way to browse the web via a network of proxies. I’ll discuss

this technology in more detail in section 23.4.2; the Navy makes it available to

everyone on the Internet so as to generate lots of cover trafﬁc to hide its own

communications [1062]. Indeed, many future military applications are likely

to use the Internet, and this will raise many interesting questions — ranging

from the ethics of attacking the information infrastructure of hostile or neutral

countries, to the details of how military trafﬁc of various kinds can be hidden

among civilian packets and bistreams.

There may indeed be some convergence. Although communications security

on the net has until now been interpreted largely in terms of message conﬁ-

dentiality and authentication, the future may become much more like military

communications in that jamming, service denial, anonymity, and deception

will become increasingly important. I’ll return to this theme later.

Next, let’s look at the aspects of electronic warfare that have to do with target

acquisition and weapon guidance, as these are where the arts of jamming and

deception have been most highly developed. (In fact, although there is much

more in the open literature on the application of electronic attack and defense

to radar than to communications, much of the same material applies to both.)

574 Chapter 19 ■Electronic and Information Warfare

19.4 Surveillance and Target Acquisition

Although some sensor systems use passive direction ﬁnding, the main methods

used to detect hostile targets and guide weapons to them are sonar, radar and

infrared. The ﬁrst of these to be developed was sonar, which was invented

and deployed in World War 1 (under the name of ‘Asdic’) [574]. Except

in submarine warfare, the key sensor is radar. Although radar was invented in

1904 as a maritime anti-collision device, its serious development only occurred

in the 1930s and it was used by all major participants in World War 2 [578, 670].

The electronic attack and protection techniques developed for it tend to be

better developed than, and often go over to, systems using other sensors. In

the context of radar, ‘electronic attack’ usually means jamming (though in

theory it also includes stealth technology), and ‘electronic protection’ refers to

the techniques used to preserve at least some radar capability.

19.4.1 Types of Radar

A wide range of systems is in use, including search radars, ﬁre-control radars,

terrain-following radars, counter-bombardment radars and weather radars.

They have a wide variety of signal characteristics. For example, radars with

alowRFandalowpulse repetition frequency (PRF) are better for search while

high frequency, high PRF devices are better for tracking. A good textbook on

the technology is by Schleher [1121].

Simple radar designs for search applications may have a rotating antenna

that emits a sequence of pulses and detects echos. This was an easy way to

implement radar in the days before digital electronics; the sweep in the display

tube could be mechanically rotated in synch with the antenna. Fire control

radars often used conical scan: the beam would be tracked in a circle around

the target’s position, and the amplitude of the returns could drive positioning

servos (and weapon controls) directly. Now the beams are often generated

electronically using multiple antenna elements, but tracking loops remain

central. Many radars have a range gate, circuitry which focuses on targets

within a certain range of distances from the antenna; if the radar had to track

all objects between (say) zero and 100 miles, then its pulse repetition frequency

would be limited by the time it takes radio waves to travel 200 miles. This

would have consequences for angular resolution and tracking performance

generally.

Doppler radar measures the velocity of the target by the change in frequency

in the return signal. It is very important in distinguishing moving targets from

clutter, the returns reﬂected from the ground. Doppler radars may have velocity

gates that restrict attention to targets whose radial speed with respect to the

antenna is within certain limits.

19.4 Surveillance and Target Acquisition 575

19.4.2 Jamming Techniques

Electronic attack techniques can be passive or active.

The earliest countermeasure to be widely used was chaff — thin strips of

conducting foil that are cut to a half the wavelength of the target signal and

then dispersed to provide a false return. Toward the end of World War 2,

allied aircraft were dropping 2000 tons of chaff a day to degrade German air

defenses. Chaff can be dropped directly by the aircraft attempting to penetrate

the defenses (which isn’t ideal as they will then be at the apex of an elongated

signal), or by support aircraft, or ﬁred forward into a suitable pattern using

rockets or shells. The main counter-countermeasure against chaff is the use of

Doppler radars; as the chaff is very light it comes to rest almost at once and

can be distinguished fairly easily from moving targets.

Other decoy techniques include small decoys with active repeaters that

retransmit radar signals and larger decoys that simply reﬂect them; sometimes

one vehicle (such as a helicopter) acts as a decoy for another more valuable

one (such as an aircraft carrier). These principles are quite general. Weapons

that home in on their targets using RDF are decoyed by special drones that

emit seduction RF signals, while infrared guided missiles are diverted using

ﬂares.

The passive countermeasure in which the most money has been invested

is stealth — reducing the radar cross-section (RCS) of a vehicle so that it can

be detected only at very much shorter range. This means, for example, that

the enemy has to place his air defense radars closer together, so he has to

buy a lot more of them. Stealth includes a wide range of techniques and a

proper discussion is well beyond the scope of this book. Some people think

of it as ‘extremely expensive black paint’ but there’s more to it than that; as

an aircraft’s RCS is typically a function of its aspect, it may have a ﬂy-by-wire

system that continually exhibits an aspect with a low RCS to identiﬁed hostile

emitters.

Active countermeasures are much more diverse. Early jammers simply

generated a lot of noise in the range of frequencies used by the target radar;

this technique is known as noise jamming or barrage jamming. Some systems used

systematic frequency patterns, such as pulse jammers, or swept jammers that

traversed the frequency range of interest (also known as squidging oscillators).

But such a signal is fairly easy to block — one trick is to use a guard band

receiver, a receiver on a frequency adjacent to the one in use, and to blank

the signal when this receiver shows a jamming signal. It should also be noted

that jamming isn’t restricted to one side; as well as being used by the radar’s

opponent, the radar itself can also send suitable spurious signals from an

auxiliary antenna to mask the real signal or simply overload the defenses.

At the other end of the scale lie hard-kill techniques such as anti-radiation

missiles (ARMs), often ﬁred by support aircraft, which home in on the sources

576 Chapter 19 ■Electronic and Information Warfare

of hostile signals. Defenses against such weapons include the use of decoy

transmitters, and blinking transmitters on and off.

In the middle lies a large toolkit of deception jamming techniques. Most

jammers used for self-protection are deception jammers of one kind or another;

barrage and ARM techniques tend to be more suited to use by support vehicles.

The usual goal with a self-protection jammer is to deny range and bearing

information to attackers. The basic trick is inverse gain jamming or inverse gain

amplitude modulation. This is based on the observation that the directionality

of the attacker’s antenna is usually not perfect; as well as the main beam it

has sidelobes through which energy is also transmitted and received, albeit

much less efﬁciently. The sidelobe response can be mapped by observing the

transmitted signal, and a jamming signal can be generated so that the net

emission is the inverse of the antenna’s directional response. The effect, as

far as the attacker’s radar is concerned, is that the signal seems to come from

everywhere; instead of a ‘blip’ on the radar screen you see a circle centered

on your own antenna. Inverse gain jamming is very effective against the older

conical-scan ﬁre-control systems.

More generally, the technique is to retransmit the radar signal with a

systematic change in delay and/or frequency. This can be non-coherent, in

which case the jammer’s called a transponder, or coherent — that is, with the

right waveform — when it’s a repeater. (It is now common to store received

waveforms in digital radio frequency memory (DRFM) and manipulate them

using signal processing chips.)

An elementary countermeasure is burn-through.Byloweringthepulse

repetition frequency, the dwell time is increased and so the return signal is

stronger — at the cost of less precision. A more sophisticated countermeasure

is range gate pull-off (RGPO). Here, the jammer transmits a number of fake

pulses that are stronger than the real ones, thus capturing the receiver, and

then moving them out of phase so that the target is no longer in the receiver’s

range gate. Similarly, with Doppler radars the basic trick is velocity gate pull-off

(VGPO). With older radars, successful RGPO would cause the radar to break

lock and the target to disappear from the screen. Modern radars can reacquire

lock very quickly, and so RGPO must either be performed repeatedly or

combined with another technique — commonly, with inverse gain jamming

to break angle tracking at the same time.

An elementary counter-countermeasure is to jitter the pulse repetition

frequency. Each outgoing pulse is either delayed or not depending on a lag

sequence generated by a stream cipher or random number generator. This

means that the jammer cannot anticipate when the next pulse will arrive and

has to follow it. Such follower jamming can only make false targets that appear

to be further away. So the counter-counter-countermeasure, or (counter)3-

measure, is for the radar to have a leading edge tracker, which responds only

to the ﬁrst return pulse; and the (counter)4-measures can include jamming at

19.4 Surveillance and Target Acquisition 577

such a high power that the receiver’s automatic gain control circuit is captured.

An alternative is cover jamming in which the jamming pulse is long enough to

cover the maximum jitter period.

The next twist of the screw may involve tactics. Chaff is often used to force a

radar into Doppler mode, which makes PRF jitter difﬁcult (as continuous wave-

forms are better than pulsed for Doppler), while leading edge trackers may be

combined with frequency agility and smart signal processing. For example,

true target returns ﬂuctuate, and have realistic accelerations, while simple

transponders and repeaters give out a more or less steady signal. Of course,

it’s always possible for designers to be too clever; the Mig-29 could decelerate

more rapidly in level ﬂight by a rapid pull-up than some radar designers

had anticipated, so pilots could use this manoeuvre to break radar lock.

And now of course, CPUs are powerful enough to manufacture realistic false

returns.

19.4.3 Advanced Radars and Countermeasures

A number of advanced techniques are used to give an edge on the jammer.

Pulse compression was ﬁrst developed in Germany in World War 2, and uses a

kind of direct sequence spread spectrum pulse, ﬁltered on return by a matched

ﬁlter to compress it again. This can give processing gains of 10–1000. Pulse

compression radars are resistant to transponder jammers, but are vulnerable

to repeater jammers, especially those with digital radio frequency memory.

However, the use of LPI waveforms is important if you do not wish the target

to detect you long before you detect him.

Pulsed Doppler is much the same as Doppler, and sends a series of phase

stable pulses. It has come to dominate many high end markets, and is widely

used, for example, in look-down shoot-down systems for air defense against

low-ﬂying intruders. As with elementary pulsed tracking radars, different

RF and pulse repetition frequencies give different characteristics: we want

low frequency/PRF for unambiguous range/velocity and also to reduce

clutter — but this can leave many blind spots. Airborne radars that have to

deal with many threats use high PRF and look only for velocities above some

threshold, say 100 knots — but are weak in tail chases. The usual compromise

is medium PRF — but this suffers from severe range ambiguities in airborne

operations. Also, search radar requires long, diverse bursts but tracking needs

only short, tuned ones. An advantage is that pulsed Doppler can discriminate

some very speciﬁc signals, such as modulation provided by turbine blades

in jet engines. The main deception strategy used against pulsed Doppler is

velocity gate pull-off, although a new variant is to excite multiple velocity

gates with deceptive returns.

Monopulse is becoming one of the most popular techniques. It is used, for

example, in the Exocet missiles that proved so difﬁcult to jam in the Falklands

578 Chapter 19 ■Electronic and Information Warfare

war. The idea is to have four linked antennas so that azimuth and elevation

data can be computed from each return pulse using interferometric techniques.

Monopulse radars are difﬁcult and expensive to jam, unless a design defect can

be exploited; the usual techniques involve tricks such as formation jamming

and terrain bounce. Often the preferred defensive strategy is just to use towed

decoys.

One of the more recent tricks is passive coherent location. Lockheed’s ‘Silent

Sentry’ system has no emitters at all, but rather utilizes reﬂections of com-

mercial radio and television broadcast signals to detect and track airborne

objects [807], and the UK ‘Celldar’ project aims to use the signals from mobile-

phone masts for the same purpose [246]. The receivers, being passive, are

hard to locate and attack; knocking out the system entails destroying major

civilian infrastructure, which opponents will often prefer not to do for legal

and propaganda reasons. Passive coherent location is effective against some

kinds of stealth technology, particularly those that entail steering the aircraft

so that it presents the nulls in its radar cross-section to visible emitters.

Attack and defence could become much more complex given the arrival

of digital radio frequency memory and other software radio techniques.

Both radar and jammer waveforms may be adapted to the tactical situation

with much greater ﬂexibility than before. But fancy combinations of spectral,

temporal and spatial characteristics will not be the whole story. Effective

electronic attack is likely to continue to require the effective coordination of

different passive and active tools with weapons and tactics. The importance

of intelligence, and of careful deception planning, is likely to increase.

19.4.4 Other Sensors and Multisensor Issues

Much of what I’ve said about radar applies to sonar as well, and a fair amount

to infrared. Passive decoys — ﬂares — worked very well against early heat-

seeking missiles which used a mechanically spun detector, but are less effective

against modern detectors that incorporate signal processing. Flares are like

chaff in that they decelerate rapidly with respect to the target, so the attacker

can ﬁlter on velocity or acceleration. They are also like repeater jammers in

that their signals are relatively stable and strong compared with real targets.

Active infrared jamming is harder and thus less widespread than radar

jamming; it tends to exploit features of the hostile sensor by pulsing at a rate

or in a pattern which causes confusion. Some infrared defense systems are

starting to employ lasers to disable the sensors of incoming weapons; and it’s

been admitted that a number of ‘UFO’ sightings were actually due to various

kinds of jamming (both radar and infrared) [119].

One growth area is multisensor data fusion whereby inputs from radars,

infrared sensors, video cameras and even humans are combined to give better

target identiﬁcation and tracking than any could individually. The Rapier air

19.5 IFF Systems 579

defense missile, for example, uses radar to acquire azimuth while tracking

is carried out optically in visual conditions. Data fusion can be harder than

it seems. As I discussed in section 15.9, combining two alarm systems will

generally result in improving either the false alarm or the missed alarm rate,

while making the other worse. If you scramble your ﬁghters when you see a

blip on either the radar or the infrared, there will be more false alarms; but if

you scramble only when you see both then it will be easier for the enemy to

jam you or sneak through.

System issues become more complex where the attacker himself is on a

platform that’s vulnerable to counter-attack, such as a ﬁghter bomber. He will

have systems for threat recognition, direction ﬁnding and missile approach

warning, and the receivers in these will be deafened by his jammer. The usual

trick is to turn the jammer off for a short ‘look-through’ period at random

times.

With multiple friendly and hostile platforms, things get more complex still.

Each side might have specialist support vehicles with high power dedicated

equipment, which makes it to some extent an energy battle — ‘he with the most

watts wins’. A SAM belt may have multiple radars at different frequencies to

make jamming harder. The overall effect of jamming (as of stealth) is to reduce

the effective range of radar. But jamming margin also matters, and who has

the most vehicles, and the tactics employed.

With multiple vehicles engaged, it’s also necessary to have a reliable way of

distinguishing friend from foe.

19.5 IFF Systems

Identify-Friend-or-Foe (IFF) systems are both critical and controversial, with a

signiﬁcant number of ‘blue-on-blue’ incidents in Iraq being due to equipment

incompatibility between U.S. and allied forces. Incidents in which U.S. aircraft

bombed British soldiers have contributed signiﬁcantly to loss of UK public

support for the war, especially after the authorities in both countries tried

and failed to cover up such incidents out of a wish to both preserve technical

security and also to minimise political embarrassment.

IFF goes back in its non-technical forms to antiquity; see for example the

quote from Judges 12:5–6 at the head of Chapter 15 on identifying soldiers by

whether they could pronounce ‘Shibboleth’. World War 2 demonstrated the

need for systems that could cope with radar; the Japanese aircraft heading

toward Pearl Harbour were seen by a radar operator at Diamond Head

but assumed to be an incoming ﬂight of U.S. planes. Initial measures were

procedural; returning bombers would be expected to arrive at particular times

and cross the coast at particular places, while stragglers would announce their

lack of hostile intent by some pre-arranged manoeuvre such as ﬂying in an

580 Chapter 19 ■Electronic and Information Warfare

equilateral triangle before crossing the coast. (German planes would roll over

when the radio operator challenged them, so as to create a ‘blip’ in their radar

cross-section.) There were also some early attempts at automation, with the

‘Mark 1’ system being mechanically tuned and not very usable. There were

also early attempts at spooﬁng.

The Korean war saw the arrival on both sides of jet aircraft and missiles,

which made it impractical to identify targets visually and imperative to have

automatic IFF. Early systems simply used a vehicle serial number or ‘code of

the day’, but this was wide open to spooﬁng, and the world’s air forces started

work on cryptographic authentication.

Since the 1960s, U.S. and other NATO aircraft have used the Mark XII

system. This uses a crypto unit with a block cipher that is a DES precursor, and

is available for export to non-NATO customers with alternative block ciphers.

However, it isn’t the cryptography that’s the hard part, but rather the protocol

problems discussed in Chapter 3. The Mark XII has four modes of which the

secure mode uses a 32-bit challenge and a 4-bit response. This is a precedent

set by its predecessor, the Mark X; if challenges or responses were too long,

then the radar’s pulse repetition frequency (and thus it accuracy) would be

degraded. So it’s necessary to use short challenge-response pairs for radar

security reasons, and many of them for cryptosecurity reasons. The Mark 12

sends 12–20 challenges in a series, and in the original implementation the

responses were displayed on a screen at a position offset by the arithmetic

difference between the actual response and the expected one. The effect

was that while a foe had a null or random response, a ‘friend’ would have

responses at or near the center screen, which would light up. Reﬂection attacks

are prevented, and MIG-in-the-middle attacks made much harder, because

the challenge uses a focussed antenna, while the receiver is omnidirectional.

(In fact, the antenna used for the challenge is typically the ﬁre control radar,

which in older systems was conically scanned.)

This mechanism still doesn’t completely stop ‘ack wars’ when two squadrons

(or naval ﬂotillas) meet each other. Meanwhile systems are becoming ever

more complex. There’s a program to create a NATO Mark XIIA that will

be backwards-compatible with the existing Mark X/XII systems, and a U.S.

Mark XV, both of which use spread-spectrum waveforms. The systems used in

military aircraft also have compatibility modes with the civil systems used by

aircraft to ‘squawk’ their ID to secondary surveillance radar. However, that’s

only for air-to-air IFF, and the real problems are now air-to-ground. NATO’s

IFF systems evolved for a Cold War scenario of thousands of tactical aircraft

on each side of the Iron Curtain; how do they fare in a modern conﬂict like

Iraq or Afghanistan?

Historically, about 10–15% of casualties were due to ‘friendly ﬁre’ but

in the First Gulf War this rose to 25%. Such casualties are more likely

at the interfaces between air and land battle, and between sea and land,

19.5 IFF Systems 581

because of the different services’ way of doing things; joint operations are

thus particularly risky. Coalition operations also increase the risk because of

different national systems. Following this experience, several experimental

systems were developed to extend IFF to ground troops. One U.S. system

combines laser and RF components. Shooters have lasers, and soldiers have

transponders; when the soldier is illuminated with a suitable challenge his

equipment broadcasts a ‘don’t shoot me’ message using frequency-hopping

radio [1372]. An extension allows aircraft to broadcast targeting intentions on

millimeter wave radio. The UK started developing a cheaper system in which

friendly vehicles carry an LPI millimeter-wave transmitter, and shooters carry

a directional receiver [599]. (Dismounted British foot soldiers, unlike their

American counterparts, were not deemed worthy of protection.) A prototype

system was ready in 2001 but not put into production. Other countries started

developing yet other systems.

But when Gulf War 2 came along, nothing decent had been deployed.

A report from Britain’s National Audit Ofﬁce from 2002 describes what

went wrong [930]. In a world where defence is purchased not just by nation

states, and not just by services, but by factions within these services, and

where legislators try to signal their ‘patriotism’ to less-educated voters by

blocking technical collaboration with allies (‘to stop them stealing our jobs

and our secrets’), it’s hard. The institutional and political structures just aren’t

conducive to providing defense ‘public goods’ such as a decent IFF system that

would work across NATO. And NATO is a broad alliance; as one insider told

me, ‘‘Trying to evolve a solution that met the aspirations of both the U.S. at

one extreme and Greece (for example) at the other was a near hopeless task.’’

Project complexity is one issue: it’s not too hard to stop your air force

planes shooting each other, it’s a lot more complex to stop them shooting

at your ships or tanks, and it’s much harder still when a dozen nations are

involved. Technical ﬁxes are still being sought; for example, the latest U.S.

software radio project, the Joint Tactial Radio System (JTRS, or ‘jitters’), may

eventually equip all services with radios that interoperate and do at least two

IFF modes. However, it’s late, over budget, and fragmented into subprojects

managed by the different services. There are also some sexy systems used by

a small number of units in Iraq that let all soldiers see each others’ positions

superimposed in real time on a map display on a helmet-mounted monocle.

They greatly increase force capability in mobile warfare, allowing units to

execute perilous manoevres like driving through each others’ kill zones, but

are not a panacea in complex warfare such as Iraq in 2007: there, the key

networks are social, not electronic, and it’s hard to automate networks with

nodes of unknown trustworthiness [1116].

In any case, experience so far has taught us that even with ‘hard-core’ IFF,

such as where ships and planes identify each other, the hardest issues weren’t

technical but to do with economics, politics and doctrine. Over more than a

582 Chapter 19 ■Electronic and Information Warfare

decade of wrangling within NATO, America wanted an expensive high-tech

system, for which its defense industry was lobbying hard, while European

countries wanted something simpler and cheaper that they could also build

themselves, for example by tracking units through the normal command-and-

control system and having decent interfaces between nations. But the USA

refused to release the location of its units to anyone else for ‘security’ reasons.

America spends more on defense than its allies combined and believed it

should lead; the allies didn’t want their own capability further marginalised

by yet more dependence on U.S. suppliers.

Underlying doctrinal tensions added to this. U.S. doctrine, the so-called

‘Revolution in Military Affairs’ (RMA) promoted by Donald Rumsfeld and

based on an electronic system-of-systems, was not only beyond the allies’

budget but was distrusted, based as it is on minimising one’s own casu-

alties through vast material and technological supremacy. The Europeans

argued that one shouldn’t automatically react to sniper ﬁre from a vil-

lage by bombing the village; as well as killing ten insurgents, you kill

a hundred civilians and recruit several hundred of their relatives to the

other side. The American retort to this was that Europe was too weak and

divided to even deal with genocide in Bosnia. The result was deadlock;

countries decided to pursue national solutions, and no real progress has

been made on interoperability in twenty years. Allied forces in Iraq and

Afghanistan were reduced to painting large color patches on the roofs of

their vehicles and hoping the air strikes would pass them by. U.S. aircraft

duly bombed and killed a number of allied servicemen, which weakened

the alliance. Perhaps we’ll have convergence in the long run, as European

countries try to catch up with U.S. military systems, and U.S. troops revert

to a more traditional combat mode as they discover the virtues of win-

ning local tribal allies in the ﬁght against Al-Qaida in Iraq. However, for

a converged solution to be stable, we may well need some institutional

redesign.

19.6 Improvised Explosive Devices

A signiﬁcant effort has been invested in 2004–7 in electronic-warfare measures

to counter the improvised explosive devices (IEDs) that are the weapon of

choice of insurgents in Iraq and, increasingly, Afghanistan. Since the ﬁrst IED

attack on U.S. forces in March 2003, there have been 81,000 attacks, with 25,000

in 2007 alone. These bombs have become the ‘signature weapon’ of the Iraq

war, as the machine-gun was of World War 1 and the laser-guided bomb of

Gulf War I. (And now that unmanned aerial vehicles are built by hobbyists for

about $1000, using model-aircraft parts, a GPS receiver and a Lego Mindstorms

robotics kit, we might even see improvised cruise missiles.)

19.6 Improvised Explosive Devices 583

Anyway, over 33,000 jammers have been made and shipped to coalition

forces. The Department of Defense spent over $1bn on them in 2006, in

an operation that, according to insiders, ‘proved the largest technological

challenge for DOD in the war, on a scale last experienced in World War

2’ [94]. The overall budget for the Pentagon’s Joint IED Defeat Organization

was claimed to almost $4bn by the end of 2006. Between early 2006 and late

2007, the proportion of radio-controlled IEDs dropped from as much as 70%

to 10%; the proportion triggered by command wires increased to 40%.

Rebels have been building bombs since at least Guy Fawkes, who tried to

blow up Britain’s Houses of Parliament in 1605. Many other nationalist and

insurgent groups have used IEDs, from anarchists through the Russian resis-

tance in World War 2, the Irgun, ETA and the Viet Cong to Irish nationalists.

The IRA got so expert at hiding IEDs in drains and culverts that the British

Army had to use helicopters instead of road vehicles in the ‘bandit country’

near the Irish border. They also ran bombing campaigns against the UK on

a number of occasions in the twentieth century. In the last of these, from

1970–94, they blew up the Grand Hotel in Brighton when Margaret Thatcher

was staying there for a party conference, killing several of her colleagues;

later, London suffered two incidents in which the IRA set off truckloads of

home-made explosive causing widespread devastation. The ﬁght against the

IRA involved 7,000 IEDs, and gave UK defense scientists much experience in

jamming: barrage jammers were ﬁtted in VIP cars that would cause IEDs to

go off either too early or too late. These were made available to allies; such a

jammer saved the life of President Musharraf of Pakistan when Al-Qaida tried

to blow up his convoy in 2005.

The electronic environment in Iraq turned out to be much more difﬁcult

than either Belfast or the North-West Frontier. Bombers can use any device

that will ﬂip a switch at a distance, and employed everything from key fobs

to cellphones. Meanwhile the RF environment in Iraq had become complex

and chaotic. Millions of Iraqis used unregulated cellphones, walkie-talkies and

satellite phones, as most of the optical-ﬁbre and copper infrastructure had been

destroyed in the 2003 war or looted afterwards. 150,000 coalition troops also

sent out a huge variety of radio emissions, which changed all the time as units

rotated. Over 80,000 radio frequencies were in use, and monitored using 300

databases — many of them not interoperable. Allied forces only started to get

on top of the problem when hundreds of Navy electronic warfare specialists

were deployed in Baghdad; after that, coalition jamming efforts were better

coordinated and started to cut the proportion of IEDs detonated by radio.

But the ‘success’ in electronic warfare hasn’t translated into a reduction in

allied casualties. The IED makers have simply switched from radio-controlled

bombs to devices detonated by pressure plates, command wires, passive

infrared or volunteers. The focus is now shifting to a mix of tactics: ‘right of

boom’ measures such as better vehicle armor, and ‘left of boom’ measures

584 Chapter 19 ■Electronic and Information Warfare

such as disrupting the bomb-making networks (Britain and Israel had for years

targeted bombmakers in Ireland and Lebanon respectively). Better armor at

least is having some effect: while in 2003 almost every IED caused a coalition

casualty, now it takes four devices on average [94]. Armored vehicles were also

a key tactic in other insurgencies. Network disruption, though, is a longer-term

play as it depends largely on building up good sources of human intelligence.

19.7 Directed Energy Weapons

In the late 1930s, there was panic in Britain and America on rumors that the

Nazis had developed a high-power radio beam that would burn out vehicle

ignition systems. British scientists studied the problem and concluded that this

was infeasible [670]. They were correct — given the relatively low-powered

radio transmitters, and the simple but robust vehicle electronics, of the 1930s.

Things started to change with the arrival of the atomic bomb. The detonation

of a nuclear device creates a large pulse of gamma-ray photons, which in

turn displace electrons from air molecules by Compton scattering. The large

induced currents give rise to an electromagnetic pulse (EMP), which may be

thought of as a very high amplitude pulse of radio waves with a very short

rise time.

Where a nuclear explosion occurs within the earth’s atmosphere, the

EMP energy is predominantly in the VHF and UHF bands, though there is

enough energy at lower frequencies for a radio ﬂash to be observable thou-

sands of miles away. Within a few tens of miles of the explosion, the radio

frequency energy may induce currents large enough to damage most electronic

equipment that has not been hardened. The effects of a blast outside the earth’s

atmosphere are believed to be much worse (although there has never been a

test). The gamma photons can travel thousands of miles before they strike the

earth’s atmosphere, which could ionize to form an antenna on a continental

scale. It is reckoned that most electronic equipment in Northern Europe could

be burned out by a one megaton blast at a height of 250 miles above the North

Sea. For this reason, critical military systems are carefully shielded.

Western concern about EMP grew after the Soviet Union started a research

program on non-nuclear EMP weapons in the mid-80s. At the time, the

United States was deploying ‘neutron bombs’ in Europe — enhanced radiation

weapons that could kill people without demolishing buildings. The Soviets

portrayed this as a ‘capitalist bomb’ which would destroy people while leaving

property intact, and responded by threatening a ‘socialist bomb’ to destroy

property (in the form of electronics) while leaving the surrounding people

intact.

By the end of World War 2, the invention of the cavity magnetron had

made it possible to build radars powerful enough to damage unprotected

19.7 Directed Energy Weapons 585

electronic circuitry at a range of several hundred yards. The move from valves

to transistors and integrated circuits has increased the vulnerability of most

commercial electronic equipment. A terrorist group could in theory mount a

radar in a truck and drive around a city’s ﬁnancial sector wiping out the banks.

In fact, the banks’ underground server farms would likely be unaffected; the

real damage would be to everyday electronic devices. For example, some

electronic car keys are so susceptible to RF that they can be destroyed if left

next to a cell phone [1073]. Replacing the millions of gadgets on which a city’s

life depends would be extremely tiresome.

For battleﬁeld use, it’s useful if the weapon can be built into a standard bomb

or shell casing rather than having to be truck-mounted. The Soviets are said

to have built high-energy RF (HERF) devices, and the U.S. responded with its

own arsenal: a device called Blow Torch was tried in Iraq as a means of frying

the electronics in IEDs, but it didn’t work well [94]. There’s a survey of usable

technologies at [737] that describes how power pulses in the Terawatt range

can be generated using explosively-pumped ﬂux compression generators and

magnetohydrodynamic devices, as well as by more conventional high-power

microwave devices.

By the mid 1990s, the concern that terrorists might get hold of these

weapons from the former Soviet Union led the agencies to try to sell commerce

and industry on the idea of electromagnetic shielding. These efforts were

dismissed as hype. Personally, I tend to agree. Physics suggests that EMP is

limited by the dielectric strength of air and the cross-section of the antenna.

In nuclear EMP, the effective antenna size could be a few hundred meters

for an endoatmospheric blast, up to several thousand kilometers for an

exoatmospheric one. But in ‘ordinary’ EMP/HERF, the antenna will usually

just be a few meters. According to the cited paper, EMP bombs need to be

dropped from aircraft and deploy antennas before detonation in order to get

decent coupling, and even so are lethal to ordinary electronic equipment for a

radius of only a few hundred meters. NATO planners concluded that military

command and control systems that were already hardened for nuclear EMP

should be unaffected.

And as far as terrorists are concerned, I wrote here in the ﬁrst edition of

this book: ‘As for the civilian infrastructure, I suspect that a terrorist can do

a lot more damage with an old-fashioned truck bomb made with a ton of

fertilizer and fuel oil, and he doesn’t need a PhD in physics to design one!’

That was published a few months before 9/11. Of course, a Boeing 767 will do

more damage than a truck bomb, but a truck bomb still does plenty, as we see

regularly in Iraq, and even small IEDs of the kind used by Al-Qaida in London

in 2005 can kill enough people to have a serious political effect. In addition,

studies of the psychology of terror support the view that lethal attacks are

much more terrifying than nonlethal ones almost regardless of the economic

586 Chapter 19 ■Electronic and Information Warfare

damage they do (I’ll come back to this in Part III). So I expect that terrorists

will continue to prefer a truckload of fertiliser to a truckload of magnetrons.

There remains one serious concern: that the EMP from a single nuclear

explosion at an altitude of 250 miles would do colossal economic damage,

while killing few people directly [80]. This gives a blackmail weapon to

countries such as Iran and North Korea with nuclear ambitions but primitive

technology otherwise. North Korea recently ﬁred a missile into the sea near

Japan, which together with their nuclear test sent a clear signal: ‘We can

switch off your economy any time we like, and without directly killing a single

Japanese civilian either’. And how would Japan respond? (They’re hurriedly

testing anti-missile defences.) What, for that matter, would the USA do if

Kim Jong-Il mounted a missile on a ship, sailed it towards the Panama Canal,

and ﬁred a nuke 250 miles above the central United States? That could knock

out computers and communications from coast to coast. A massive attack on

electronic communications is more of a threat to countries such as the USA

and Japan that depend on them, than on countries such as North Korea (or

Iran) that don’t.

This observation goes across to attacks on the Internet as well, so let’s now

turn to ‘Information Warfare’.

19.8 Information Warfare

From about 1995, the phrase Information warfare came into wide use. Its

popularity was boosted by operational experience in Desert Storm. There, air

power was used to degrade the Iraqi defenses before the land attack was

launched, and one goal of NSA personnel supporting the allies was to enable

the initial attack to be made without casualties — even though the Iraqi air

defenses were at that time intact and alert. The attack involved a mixture of

standard e-war techniques such as jammers and antiradiation missiles; cruise

missile attacks on command centers; attacks by special forces who sneaked

into Iraq and dug up lengths of communications cabling from the desert;

and, allegedly, the use of hacking tricks to disable computers and telephone

exchanges. (By 1990, the U.S. Army was already calling for bids for virus

production [825].) The operation successfully achieved its mission of ensuring

zero allied casualties on the ﬁrst night of the aerial bombardment. Military

planners and think tanks started to consider how the success could be built on.

After 9/11, information warfare was somewhat eclipsed as the security-

industrial complex focussed on topics from airport screening to the detection

of improvised explosive devices. But in April 2007, it was thrust back on the

agenda by events in Estonia. There, the government had angered Russia by

moving an old Soviet war memorial, and shortly afterwards the country was

subjected to a number of distributed denial-of-service attacks that appeared

19.8 Information Warfare 587

to originate from Russia [359]. Estonia’s computer emergency response team

tackled the problem with cool professionalism, but their national leadership

didn’t. Their panicky reaction got world headlines [413]; they even thought of

invoking the NATO treaty and calling for U.S. military help against Russia.

Fortunately common sense prevailed. It seems that the packet storms were

simply launched by Russian botnet herders, reacting to the news from Esto-

nia and egging each other on via chat rooms, rather than being an act of

state aggression; the one man convicted of the attacks was an ethnic Russian

teenager in Estonia itself. There have been similar tussles between Israeli and

Palestinian hackers, and between Indians and Pakistanis. Estonia also had

some minor street disturbances caused by rowdy ethnic Russians objecting

to the statue’s removal; ‘Web War 1’ seems to have been the digital equiv-

alent. Since then, however, there have been press reports alleging Chinese

attacks on government systems in both the USA and the UK, including

service-denial attacks and attempted intrusions, causing ‘minor administra-

tive disruptions’ [973]. Defense insiders leak reports saying that China has a

massive capability to attack the West [1063]. Is this serious, or is it just the

agencies shaking the tin for more money?

But what’s information warfare anyway? There is little agreement on deﬁ-

nitions. The conventional view, arising out of Desert Storm, was expressed by

Whitehead [1314]:

The strategist ... should employ (the information weapon) as a

precursor weapon to blind the enemy prior to conventional attacks

and operations.

Meanwhile, the more aggressive view is that properly conducted infor-

mation operations should encompass everything from signals intelligence to

propaganda, and given the reliance that modern societies place on information,

it should sufﬁce to break the enemy’s will without ﬁghting.

19.8.1 Deﬁnitions

In fact, there are roughly three views on what information warfare means:

that it is just ‘a remarketing of the stuff that the agencies have been doing

for decades anyway’, in an attempt to maintain the agencies’ budgets

post-Cold-War;

that it consists of the use of ‘hacking’ in a broad sense — network attack

tools, computer viruses and so on — in conﬂict between states or sub-

state groups, in order to deny critical military and other services whether

for operational or propaganda purposes. It is observed, for example, that

the Internet was designed to withstand thermonuclear bombardment,

but was knocked out by the Morris worm;

588 Chapter 19 ■Electronic and Information Warfare

that it extends the electronic warfare doctrine of controlling the elec-

tromagnetic spectrum to control all information relevant to the conﬂict.

It thus extends traditional e-war techniques such as radar jammers by

adding assorted hacking techniques, but also incorporates propaganda

and news management.

The ﬁrst of these views was the one taken by some cynical defense insiders.

The second is the popular view found in newspaper articles, and also White-

head’s. It’s the one I’ll use as a guide in this section, but without taking a

position on whether it actually contains anything really new either technically

or doctrinally.

The third ﬁnds expression by Dorothy Denning [370] whose deﬁnition of

information warfare is ‘operations that target or exploit information media in

order to win some advantage over an adversary’. Its interpretation is so broad

that it includes not just hacking but all of electronic warfare and all existing

intelligence gathering techniques (from Sigint through satellite imagery to

spies), but propaganda too. In a later article she discussed the role of the net

in the propaganda and activism surrounding the Kosovo war [371]. However

the bulk of her book is given over to computer security and related topics.

A similar view of information warfare, and from a writer whose back-

ground is defense planning rather than computer security, is given by Edward

Waltz [1314]. He deﬁnes information superiority as ‘the capability to collect, pro-

cess and disseminate an uninterrupted ﬂow of information while exploiting

or denying an adversary’s ability to do the same’. The theory is that such

superiority will allow the conduct of operations without effective opposition.

The book has less technical detail on computer security matters than Denning

but set forth a ﬁrst attempt to formulate a military doctrine of information

operations.

19.8.2 Doctrine

When writers such as Denning and Waltz include propaganda operations in

information warfare, the cynical defense insider will remark that nothing has

changed. From Roman and Mongol efforts to promote a myth of invincibility,

through the use of propaganda radio stations by both sides in World War 2 and

the Cold War, to the bombing of Serbian TV during the Kosovo campaign and

denial-of-service attacks on Chechen web sites by Russian agencies [320] — the

tools may change but the game remains the same.

But there is a twist, perhaps thanks to government and military leaders’ lack

of familiarity with the Internet. When teenage kids deface a U.S. government

department web site, an experienced computer security professional is likely to

see it as the equivalent of grafﬁti scrawled on the wall of a public building. After

all, it’s easy enough to do, and easy enough to remove. But the information

19.8 Information Warfare 589

warfare community can paint it as undermining the posture of information

dominance that a country must project in order to deter aggression.

So there is a fair amount of debunking to be done before the political and

military leadership can start to think clearly about the issues. For example,

it’s often stated that information warfare provides a casualty-free way to win

wars: ‘just hack the Iranian power grid and watch them sue for peace’. The

three obvious comments are as follows.

The denial-of-service attacks that have so far been conducted on infor-

mation systems without the use of physical force have mostly had a tran-

sient effect. A computer comes down; the operators ﬁnd out what hap-

pened; they restore the system from backup and restart it. An outage of a

few hours may be enough to let a bomber aircraft get through unscathed,

but is unlikely to bring a country to its knees. In this context, the failure

of the Millennium Bug to cause the expected damage may be a useful

warning.

Insofar as there is a vulnerability, more developed countries are more

exposed. The power grid in the USA or the UK is likely to be much more

computerized than that in a developing country.

Finally, if such an attack causes the deaths of several dozen people in

hospitals, the Iranians aren’t likely to see the matter as being much dif-

ferent from a conventional military attack that killed the same number

of people. Indeed, if information war targets civilians to an even greater

extent than the alternatives, then the attackers’ leaders are likely to be

portrayed as war criminals. The Pinochet case, in which a former head

of government only escaped extradition on health grounds, should give

pause for thought.

Having made these points, I will restrict discussion in the rest of this section

to technical matters.

19.8.3 Potentially Useful Lessons from Electronic

Warfare

Perhaps the most important policy lesson from the world of electronic warfare

is that conducting operations that involve more than one service is very much

harder than it looks. Things are bad enough when army, navy and air force

units have to be coordinated — during the U.S. invasion of Grenada, a ground

commander had to go to a pay phone and call home using his credit card

in order to call down an air strike, as the different services’ radios were

incompatible. (Indeed, this was the spur for the development of software

radios [761].) Things are even worse when intelligence services are involved,

as they don’t train with warﬁghters in peacetime and thus take a long time

590 Chapter 19 ■Electronic and Information Warfare

to become productive once the ﬁghting starts. Turf ﬁghts also get in the

way: under current U.S. rules, the air force can decide to bomb an enemy

telephone exchange but has to get permission from the NSA and/or CIA to

hack it [103]. The U.S. Army’s communications strategy is now taking account

of the need to communicate across the traditional command hierarchy, and to

make extensive use of the existing civilian infrastructure [1115].

At the technical level, there are many concepts which may go across from

electronic warfare to information protection in general.

The electronic warfare community uses guard band receivers to detect

jamming, so it can be ﬁltered out (for example, by blanking receivers at

the precise time a sweep jammer passes through their frequency). The

use of bait addresses to detect spam is essentially the same concept.

There is also an analogy between virus recognition and radar signal

recognition. Virus writers may make their code polymorphic,inthatit

changes its form as it propagates, in order to make life harder for the

virus scanner vendors; similarly, radar designers use very diverse wave-

forms in order to make it harder to store enough of the waveform in

digital radio frequency memory to do coherent jamming effectively.

Our old friends, the false accept and false reject rate, continue to dom-

inate tactics and strategy. As with burglar alarms or radar jamming,

the ability to cause many false alarms (however crudely) will always

be worth something: as soon as the false alarm rate exceeds about 15%,

operator performance is degraded. As for ﬁltering, it can usually be

cheated.

The limiting economic factor in both attack and defense will increasingly

be the software cost, and the speed with which new tools can be created

and deployed.

It is useful, when subjected to jamming, not to let the jammer know

whether, or how, his attack is succeeding. In military communications,

it’s usually better to respond to jamming by dropping the bit rate rather

than boosting power; similarly, when a non-existent credit card number

is presented at your web site, you might say ‘Sorry, bad card number, try

again’, but the second time it happens you want a different line (or the

attacker will keep on trying). Something like ‘Sorry, the items you have

requested are temporarily out of stock and should be dispatched within

ﬁve working days’ may do the trick.

Although defense in depth is in general a good idea, you have to be

careful of interactions between the different defenses. The classic case

in e-war is when chaff dispensed to defend against an incoming cruise

missile knocks out the anti-aircraft gun. The side-effects of defenses can

also be exploited. The most common case on the net is the mail bomb in

19.8 Information Warfare 591

which an attacker forges offensive newsgroup messages that appear to

come from the victim, who then gets subjected to a barrage of abuse and

attacks.

Finally, some perspective can be drawn from the differing roles of hard

kill and soft kill in electronic warfare. Jamming and other soft-kill attacks

are cheaper, can be used against multiple threats, and have reduced

political consequences. But damage assessment is hard, and you may just

divert the weapon to another target. As most information war is soft-kill,

these comments can be expected to go across too.

19.8.4 Differences Between E-war and I-war

As well as similarities, there are differences between traditional electronic

warfare and the kinds of attack that can potentially be run over the net.

There are roughly two kinds of war — open war and guerilla war. Elec-

tronic warfare comes into its own in the ﬁrst of these: in air combat,

most naval engagements, and the desert. In forests, mountains and cities,

the man with the AK47 can still get a result against mechanized forces.

Guerilla war has largely been ignored by the e-war community, except

insofar as they make and sell radars to detect snipers and concealed mor-

tar batteries.

In cyberspace, the ‘forests, mountains and cities’ are the large numbers

of insecure hosts belonging to friendly or neutral civilians and organi-

zations. The distributed denial of service attack, in which millions of

innocent machines are subverted and used to bombard a target website

with trafﬁc, has no real analogue in the world of electronic warfare: yet it

is the likely platform for launching attacks even on ‘open’ targets such as

large commercial web sites. So it’s unclear where the open countryside in

cyberspace actually is.

Another possible source of asymmetric advantage for the guerilla is

complexity. Large countries have many incompatible systems, which

makes little difference when ﬁghting another large country with simi-

larly incompatible systems, but can leave them at a disadvantage to a

small group with simple coherent systems.

Anyone trying to attack the USA in future is unlikely to repeat Saddam

Hussein’s mistake of taking on the West in a tank battle. Asymmetric

conﬂict is now the norm, and although cyberspace has some potential

here, physical attacks have so far got much more traction — whether at

the Al-Qaida level of murderous attacks, or at the lower level of (say)

animal rights activists, who set out to harass people rather than mur-

der them and thus stay just below the threshold at which a drastic state

592 Chapter 19 ■Electronic and Information Warfare

response would be invoked. A group that wants to stay at this level — so

that its operatives risk short prison sentences rather than execution —

can have more impact if it uses physical as well as electronic harassment.

As a member of Cambridge University’s governing body, the Council,

I was subjected for some months to this kind of hassle, as animal rights

fanatics protested at our psychology department’s plans to construct

a new building to house its monkeys. I also watched the harassment’s

effects on colleagues. Spam ﬂoods were easily enough dealt with; peo-

ple got much more upset when protesters woke them and their families

in the small hours, by throwing rocks on their house roofs and screaming

abuse. I’ll discuss this later in Part III.

There is no electronic-warfare analogue of script kiddies — people who

download attack scripts and launch them without really understand-

ing how they work. That such tools are available universally, and for

free, has few analogues in meatspace. You might draw a comparison

with the lawless areas of countries such as Afghanistan where all men go

about armed. But the damage done by Russian script kiddies to Estonia

was nothing like the damage done to allied troops by Afghan tribes-

men — whether in the present Afghan war or in its nineteenth century

predecessors.

19.9 Summary

Electronic warfare is much more developed than most other areas of informa-

tion security. There are many lessons to be learned, from the technical level up

through the tactical level to matters of planning and strategy. We can expect

that if information warfare takes off, and turns from a fashionable concept

into established doctrine and practice, these lessons will become important for

engineers.

Research Problems

An interesting research problem is how to port techniques and experience

from the world of electronic warfare to the Internet. This chapter is only a

sketchy ﬁrst attempt at setting down the possible parallels and differences.

Further Reading

A good (although non-technical) introduction to radar is by P. S. Hall [578].

The best all-round reference for the technical aspects of electronic warfare,

from radar through stealth to EMP weapons, is by Curtis Schleher [1121]; a

good summary was written by Doug Richardson [1074]. The classic intro-

duction to the anti-jam properties of spread spectrum sequences is by

Andrew Viterbi [1301]; the history of spread spectrum is ably told by Robert

Scholtz [1138]; the classic introduction to the mathematics of spread spectrum

is by Raymond Pickholtz, Donald Schilling and Lawrence Milstein [1026];

while the standard textbook is by Robert Dixon [393]. The most thorough

reference on communications jamming is by Richard Poisel [1029]. An overall

history of British electronic warfare and scientiﬁc intelligence, which was

written by a true insider and gives a lot of insight not just into how the

technology developed but also into strategic and tactical deception, is by R. V.

Jones [670, 671].

CHAPTER

Telecom System Security

I rarely had to resort to a technical attack. Companies can spend

millions of dollars toward technological protections and that’s

wasted if somebody can basically call someone on the telephone and

either convince them to do something on the computer that lowers

the computer’s defenses or reveals the information they were seeking.

— Kevin Mitnick

There are two kinds of fools. One says, ‘‘This is old, therefore it is

good.’’ The other one says, ‘‘This is new, therefore it is better’’.

— Dean William Inge

20.1 Introduction

The protection of telecommunications systems is an important case study for a

number of reasons. First, many distributed systems rely on the ﬁxed or mobile

phone network in ways that are often not obvious, and the dependability of

these networks is declining. For example, POTS — the ‘plain old telephone

system’ — typically required exchanges to have backup generators with

enough diesel to survive a six-week outage in the electricity supply, while

cellular systems typically use batteries that will last at most 48 hours. What’s

worse, the electricity companies rely on mobile phones to direct their engi-

neers when repairing faults. When people realised that this could cause serious

problems where outages lasted more than two days, the electricity companies

started buying satellite phones as a backup.

Second, the history of telecomms security failures is very instructive.

Early attacks were carried out on phone companies by enthusiasts (‘phone

phreaks’) to get free calls; then the phone system’s vulnerabilities started to be

595

596 Chapter 20 ■Telecom System Security

exploited by crooks to evade police wiretapping; then premium rate calls were

introduced, which created the motive for large-scale fraud; then when

telecomms markets were liberalized, some phone companies started con-

ducting attacks on each other’s customers; and some phone companies have

even attacked each other. At each stage the defensive measures undertaken

were not only very expensive but also tended to be inadequate for various rea-

sons. The same pattern is repeating with the Internet — only with history much

speeded up. A number of the policy issues that arose with wireline phones,

such as wiretapping, have played themselves out again on the Internet.

Finally, the latest developments in telecomms, from VOIP at the consumer

level to the adoption of IP networks as the underlying technology by telecomms

providers, create further interactions. Skype’s two-day failure in August 2007

following Microsoft’s Patch Tuesday is just one case in point. Systems are

becoming much more complex and interdependent, and are generally not

being engineered to the old standards.

20.2 Phone Phreaking

The abuse of communication services goes back centuries. Before Sir Rowland

Hill invented the postage stamp, postage was paid by the recipient. Unsolicited

mail became a huge problem — especially for famous people — so recipients

were allowed to inspect a letter and reject it rather than paying for it. People

soon worked out schemes to send short messages on the covers of letters

which their correspondents rejected. Regulations were brought in to stop this,

but were never really effective [979].

A second set of abuses developed with the telegraph. The early optical

telegraphs, which worked using semaphores or heliographs, were abused by

people to place foreknowledge bets on races; if you could learn which horse

had won before the bookmaker did, you were well away. People would bribe

operators, or ‘hack the local loop’ by observing the last heliograph station

through a telescope. Here too, attempts to legislate the problem away were

a failure [1215]. The problems got even more widespread when the electric

telegraph brought costs down; the greater volumes of communication, and the

greater ﬂexibility that got built into and on top of the service, led to greater

complexity and volume of abuse.

The telephone was to be no different.

20.2.1 Attacks on Metering

Early metering systems were wide open to abuse.

In the 1950’s, the operator in some systems had to listen for the sound of

coins dropping on a metal plate to tell that a callbox customer had paid,

20.2 Phone Phreaking 597

so some people acquired the knack of hitting the coinbox with a piece of

metal that struck the right note.

Initially, the operator had no way of knowing which phone a call had

come from, so she had to ask the caller his number. He could give the

number of someone else — who would be charged. This was risky to

do from your own phone, so people did it from call boxes. Operators

started calling back to verify the number for international calls, so peo-

ple worked out social engineering attacks (‘This is IBM here, we’d like to

book a call to San Francisco and because of the time difference can our

Managing Director take it at home tonight? His number’s xxx-yyyy’). So

call box lines had a feature added to alert the operator. But in

the UK implementation, there was a bug: a customer who had called

the operator from a callbox could depress the rest for a quarter second

or so, whereupon he’d be disconnected and reconnected (often to a dif-

ferent operator), with no signal this time that the call was from a callbox.

He could then place a call to anywhere and bill it to any local number.

Early systems also signalled the entry of a coin by one or more pulses,

each of which consisted of the insertion of a resistance in the line fol-

lowed by a brief open circuit. At a number of colleges, enterprising stu-

dents installed ‘magic buttons’ which could simulate this in a callbox in

the student union so people could phone for free. (The bill in this case

went to the student union, for which the magic button was not quite so

amusing.)

Attacks on metering mechanisms continue. Many countries have moved

their payphones to chip cards in order to cut the costs of coin collection and

vandalism. Some of the implementations have been poor (as I remarked in the

chapter on tamper resistance) and villains have manufactured large quantities

of bogus phone cards. Other attacks involve what’s called clip-on: physically

attaching a phone to someone else’s line to steal their service.

In the 1970’s, when international phone calls were very expensive, foreign

students would clip their own phone on to a residential line in order to call

home; an unsuspecting home owner could get a huge bill. Despite the fact

that in most countries the cable was the phone company’s legal responsibility

up to the service socket in the house, phone companies were mostly adamant

that householders should pay and could threaten to blacklist them if they

didn’t. Now that long distance calls are cheap, the ﬁnancial incentive for

clip-on fraud has largely disappeared. But it’s still enough of a problem that

the Norwegian phone company designed a system whereby a challenge and

response are exchanged between a wall-socket mounted authentication device

and the exchange software before a dial tone is given [673].

Clip-on fraud had a catastrophic effect on a family in Cramlington, a town

in the North East of England. The ﬁrst sign they had of trouble was hearing

598 Chapter 20 ■Telecom System Security

a conversation on their line. The next was a visit from the police who said

there’d been complaints of nuisance phone calls. The complainants were three

ladies, all of whom had a number one digit different from a number to which

this family had supposedly made a huge number of calls. When the family’s

bill was examined, there were also calls to clusters of numbers that turned

out to be payphones; these had started quite suddenly at the same time as the

nuisance calls. When the family had complained later to the phone company

about a fault, their connection was rerouted and this had solved the problem.

But the phone company denied the possibility of a tap, despite the report

from their maintenance person which noted that the family’s line had been

tampered with at the distribution cabinet. (The phone company later claimed

this report was in error.) It turned out that a drug dealer had lived close by,

and it seemed a reasonable inference that he’d tapped their line in order to

call his couriers at the payphones. By using an innocent family’s phone line

instead of his own, he not only saved on the phone bill, but also had a better

chance of evading police surveillance. But both the police and the local phone

company refused to go into the house where the dealer had lived, claiming it

was too dangerous — even though the dealer had by now got six years in jail.

The Norwegian phone company declined an invitation to testify about clip-on

for the defence. The upshot was that the subscriber was convicted of making

harassing phone calls, in a case widely believed to have been a miscarriage of

justice. There was discussion at the time about whether the closing of ranks

between the phone company and the police was a bureaucratic reﬂex — or

something more sinister. Since 9/11, it’s emerged that many phone companies

have been giving the police easy access to systems for years, often without

warrants, in return for favours. The logical consequence was a policy of

covering up anything that could stray into this territory — even if the coverup

caused collateral damage. I’ll discuss all this later in the third part of this book.

Stealing dial tone from cordless phones is another variant on the theme.

In the 1990s, this became so widespread in Paris that France Telecom broke

with phone company tradition and announced that it was happening, claim-

ing that the victims were using illegally imported cordless phones which

were easy to spoof [745]. Yet to this day I am unaware of any cordless

phones — authorised or not — with decent air link authentication. The new

digital cordless phones use the DECT standard which allows for challenge-

response mechanisms [1283] but the equipment sold so far seems to simply

send a handset serial number to the base station.

Social engineering is also widespread. A crook calls you pretending to be

from AT&T security and asks whether you made a large number of calls

to Peru on your calling card. When you deny this, he says that they were

obviously fake and, in order to reverse out the charges, can he conﬁrm that

your card number is 123-456-7890-6543? No, you say (if you’re not really alert),

20.2 Phone Phreaking 599

it’s 123-456-7890-5678. Now 123-456-7890 is your phone number and 5678 your

password, so you’ve just given that caller the ability to bill calls to you.

The growth of premium rate phone services during the 1990s also led to

scamsters developing all sorts of tricks to get people to call them: pager

messages, job ads, fake emergency messages about relatives, ‘low cost’ calling

cards with 0900 access numbers, you name it. (In fact, the whole business of

tricking people into calling expensive premium numbers enabled crooks to

develop a lot of the techniques we now see used in email as part of phishing

attacks.) The 809 area code for the Caribbean used to be a favourite cover for

crooks targeting U.S. subscribers; many people weren’t aware that ‘domestic’

numbers (numbers within the USA’s +1 international direct dialling code)

extend outside the relatively cheap USA (and Canada). Even though many

people have now learned that +1 809 is ‘foreign’ and more expensive, the

introduction of still more Caribbean area codes, such as +1 345 for the Cayman

Islands, has made it even harder to spot premium rate numbers.

Phone companies advised their customers ‘Do not return calls to unfamiliar

telephone numbers’ and ‘Beware of faxes, e-mail, voice mail and pages

requesting a return call to an unfamiliar number’ [22] — but how practical

is that? Just as banks now train their customers to click on links in marketing

emails and thus make them vulnerable to phishing attacks, so I’ve had junk

marketing calls from my phone company — even though I’m on the do-not-

call list. And as for governments, they have tended to set up weak regulators

to oversee phone system abuses at home, and avoid anything that might get

them involved in trying to regulate premium rate scams overseas. For example,

they let phone companies harass their customers into paying bills for overseas

services even when they knew that the overseas trafﬁc was fraudulent.

Indeed, by no means all premium-rate scams involved obviously dodgy

companies running sex lines; as I write in 2007, the British press are full of

stores about how TV companies rip off their customers by getting them to call

premium lines in order to compete, and vote, in all sorts of shows. It’s turned

out that many of these are recorded, so the calls are totally futile; and even

the live ones are managed so that people who live in the wrong part of the

country or speak with the wrong accent have no chance. The authorities tried

to leave this to ‘self-regulation’ and on-air apologies from TV bosses, until a

public outcry (whipped up by their competitors) led to the ﬁle being sent to

the police in October 2007. It’s a recurring pattern that the biggest scams are

often run by ‘respectable’ companies rather than by Russian gangsters.

20.2.2 Attacks on Signaling

The term ‘phone phreaking’ refers to attacks on signaling as well as pure toll

fraud. Until the 1980s, phone companies used signalling systems that worked

in-band by sending tone pulses in the same circuit that carried the speech. The

600 Chapter 20 ■Telecom System Security

ﬁrst attack I’ve heard of dates back to 1952, and by the mid-to-late 1960s many

enthusiasts in both America and Britain had worked out ways of rerouting

calls. One of the pioneers, Joe Engresia, had perfect pitch and discovered as

a child that he could make free phone calls by whistling a tone he’d heard

in the background of a long-distance call. His less gifted colleagues typically

used home-made tone generators, of which the most common were called

blue boxes. The trick was to call an 0800 number and then send a 2600Hz tone

that would clear down the line at the far end — that is, disconnect the called

party while leaving the caller with a trunk line connected to the exchange.

The caller could now enter the number he really wanted and be connected

without paying. Phone phreaking was one of the roots of the computer hacker

culture that took root in the Bay Area and was formative in the development

and evolution of personal computers [835]. For example, Steve Jobs and Steve

Wozniak ﬁrst built blue boxes before they diversiﬁed into computers [502].

Phone phreaking started out with a strong ideological element. In those

days most phone companies had monopolies. They were large, faceless and

unresponsive. In America, AT&T was such an abusive monopoly that the

courts eventually broke it up; most phone companies in Europe were govern-

ment departments. People whose domestic phone lines had been involved in

a service theft found they were stuck with the charges. If the young man who

had courted your daughter was (unknown to you) a phone phreak who hadn’t

paid for the calls he made to her, you would suddenly ﬁnd the company trying

to extort either the young man’s name or a payment. Phone companies were

also aligned with state security. Phone phreaks in many countries discovered

signalling codes or switch features that would enable the police or the spooks

to tap your phone from the comfort of their desks, without having to send out a

lineman to install a wiretap. Back in the days of Vietnam and student protests,

this was inﬂammatory stuff. Phone phreaks were counterculture heroes, while

phone companies were hand-in-hand with the forces of darkness.

As there was no way to stop blue-box type attacks so long as telephone

signalling was carried in-band, the phone companies spent years and many

billions of dollars upgrading exchanges so that the signaling was moved out-

of-band, in separate channels to which the subscribers had no easy access.

Gradually, region by region, the world was closed off to blue box attacks. There

are still a few places left. For example, the ﬁrst time that USAF operations

were disrupted by an ‘information warfare’ attack by noncombatants was in

1994 when two British hackers broke into the Rome Air Force Base via an

analog link through an ancient phone system in Argentina which they used to

hold up investigators [1202]. There’s also an interesting legacy vulnerability

in wiretapping systems: common phone-tapping equipment was designed to

be backwards compatible with in-band signalling, with the result that you can

evade surveillance by using a blue box to convince the police equipment that

20.2 Phone Phreaking 601

you’ve hung up. The telephone exchange ignores this signal, so you remain

on the phone but with the police recording stopped [1151].

But to defeat a modern telephone network — as opposed to its law-

enforcement add-ons — different techniques are needed.

20.2.3 Attacks on Switching and Conﬁguration

The second wave of attacks targeted the computers that did the switching.

Typically these were Unix machines on a LAN in the exchange, which also

had machines with administrative functions such as scheduling maintenance.

By hacking one of these less well guarded machines, a phreak could go across

the LAN and break into the switching equipment — or in to other secondary

systems such as subscriber databases. For a survey of PacBell’s experience of

this, see [271]; for Bellcore’s, see [722].

Using these techniques, unlisted phone numbers could be found, calls could

be forwarded without a subscriber’s knowledge, and all sorts of mischief

became possible. A Californian phone phreak called Kevin Poulsen got root

access to many of PacBel’s switches and other systems in 1985–88: this

apparently involved burglary as much as hacking (he was eventually convicted

of conspiring to possess ﬁfteen or more counterfeit, unauthorized and stolen

access devices.) He did petty things like obtaining unlisted phone numbers

for celebrities and winning a Porsche from Los Angeles radio station KIIS-FM.

Each week KIIS would give a Porsche to the 102nd caller, so Poulsen and his

accomplices blocked out all calls to the radio station’s 25 phone lines save

their own, made the 102nd call and collected the Porsche. He was also accused

of unlawful wiretapping and espionage; these charges were dismissed. In

fact, the FBI came down on him so heavily that there were allegations of an

improper relationship between the agency and the phone companies, along

the lines of ‘you scratch our backs with wiretaps when needed, and we’ll

investigate your hacker problems’ [472].

Although the unauthorized wiretapping charges against Poulsen were

dismissed, the FBI’s sensitivity does highlight the possibility that attacks on

phone company computers can be used by foreign intelligence agencies to

conduct remote wiretaps. Some of the attacks mentioned in [271] were from

overseas, and the possibility that such tricks might be used to crash the whole

phone system in the context of an information warfare attack has for some years

worried the NSA [495, 754]. Countries that import their telephone exchanges

rather than building their own are in an even worse position; a prudent

nations will assume that its telephone switchgear has vulnerabilities known to

the government of the country from which they bought it. (It was notable that

during the invasion of Afghanistan in 2001, Kabul had two exchanges: an old

electromechanical one and a new electronic one. The USAF bombed only the

ﬁrst of these.)

602 Chapter 20 ■Telecom System Security

But although high-tech attacks do happen, and newspaper articles on phone

phreaking tend to play up the ‘evil hacker’ aspects, most real attacks are

much simpler. Many involve insiders, who deliberately misconﬁgure systems

to provide free calls from (or through) favored numbers. This didn’t matter

all that much when the phone company’s marginal cost of servicing an

extra phone call was near zero, but with the modern proliferation of value-

added services, people with access to the systems can be tempted to place

(or forge) large numbers of calls to accomplices’ sex lines. Deregulation,

and the advent of mobile phones, have also made fraud serious as they

give rise to cash payments between phone companies [317]. Insiders also

get up to mischief with services that depend on the security of the phone

network. In a hack reminiscent of Poulsen, two staff at British Telecom were

dismissed after they each won ten tickets for Concorde from a phone-in offer

at which only one randomly selected call in a thousand was supposed to get

through [1266].

As for outsiders, the other ‘arch-hacker’ apart from Poulsen was Kevin

Mitnick, who got arrested and convicted following a series of break-ins, many

of which involved phone systems and which made him the target of an

FBI manhunt. They initially thought that he was a foreign agent who was

abusing the U.S. phone system in order to wiretap sensitive U.S. targets. As I

mentioned in Chapter 2, he testiﬁed after his release from prison that almost

all of his exploits had involved social engineering. He came out with the

quote at the head of this chapter: ’Companies can spend millions of dollars

toward technological protections and that’s wasted if somebody can basically

call someone on the telephone and either convince them to do something on

the computer that lowers the computer’s defenses or reveals the information

they were seeking’ [895]. So phone company systems are vulnerable to careless

insiders as well as malicious insiders — just like hospital systems and many

others we’ve discussed.

A worrying recent development is the emergence of switching exploits

by organisations. The protocols used between phone companies to switch

calls — notably 5ESS — aren’t particularly secure, as the move from in-band

to out-of-band signaling was supposed to restrict access to trusted parties. But

once again, changing environments undermine security assumptions. Now

that there are many entrepreneurial phone companies rather than a handful

of large ones, all sorts of people have access to the switching. An example

is location service. This is provided for a fee by mobile networks; you can

through a website. One entrepreneur undercut this service in the UK by

using the switching interface exported by a local telco. While such issues can

generally be resolved by contracts, litigation and regulation, there remains a

lingering worry that attackers might bring down a telco by exploiting access to

its switching and network management. This worry increases as telcos migrate

20.2 Phone Phreaking 603

their networks to IP, and they start to converge with VOIP services that give

users access to the IP layer. I’ll return to VOIP later.

20.2.4 Insecure End Systems

After direct attacks on the systems kept on phone company premises, the

next major vulnerabilities of modern phone systems are insecure terminal

equipment and feature interaction.

There have been a number of cases where villains exploited people’s

answering machines. The same technique can be used for at least two different

purposes: the relatively innocuous one of tricking someone into dialling a

premium rate number, or the somewhat more sinister one of using their

answering machine as a covert remailer for a voicemail message. The problem

arises from phone company switches that give you dial tone twelve seconds

after the other party hangs up. So a terrorist who wants to send an untraceable

instruction to a colleague can record on your answering machine thirteen

blank seconds, followed by the tones needed to dial his colleague’s number

and the secret message. He then calls again, gets the machine to play back its

messages, and hangs up on it.

But the really big frauds using insecure end systems are directed against

companies and government departments. Attacks on corporate private branch

exchange systems (PBXes) had become big business by the mid-1990’s and

cost business billions of dollars a year [322]. PBXes are usually supplied with

facilities for reﬁling calls, also known as direct inward system access (DISA). The

typical application is that the company’s sales force can call in to an 0800

number, enter a PIN or password, and then call out again taking advantage

of the low rates a large company can get for long distance calls. As you’d

expect, these PINs become known and get traded by villains [911]. The result

is known as dial-through fraud.

In many cases, the PINs are set to a default by the manufacturer, and

never changed by the customer. In other cases, PINs are captured by crooks

who monitor telephone trafﬁc in hotels anyway in order to steal credit card

numbers; phone card numbers and PBX PINs are a useful sideline. Many PBX

designs have ﬁxed engineering passwords that allow remote maintenance

access, and prudent people reckon that any PBX will have at least one back

door installed by the manufacturer to give easy access to law enforcement and

intelligence agencies (it’s said, as a condition of export licensing). Of course

such features get discovered and abused. In one case, the PBX at Scotland

Yard was compromised and used by villains to reﬁle calls, costing the Yard a

million pounds, for which they sued their telephone installer. The crooks were

never caught [1244]. This was particularly poignant, as one of the criminals’

motivations in such cases is to get access to communications that will not

be tapped. Businesses who’re the victims of such crimes nevertheless ﬁnd

604 Chapter 20 ■Telecom System Security

the police reluctant to investigate, and one reason for this is that the phone

companies aren’t particularly helpful — presumably as they don’t like having

their bills disputed [1088].

In another case, Chinese gangsters involved in labor market racketeer-

ing — smuggling illegal immigrants from Fujian, China, into Britain where

they were put to work in sweatshops, on farms and so on — hacked the PBX

of an English district council and used it to reﬁle over a million pounds’

worth of calls to China. The gang was tackled by the police after a number

of its labourers died; they were picking shellﬁsh in Morecambe Bay when

the tide came in and drowned them. The council had by now discovered the

discrepancy in its phone bills and sued the phone company for its money

back. The phone company argued that it wasn’t to blame, even although it had

supplied the insecure PBX. Here, too, the gangsters were interested not just in

saving money but in evading surveillance. (Indeed, they routed their calls to

China via a compromised PBX in Albania, so that the cross-border segment of

the call, which is most likely to be monitored by the agencies, was between

whitelisted numbers; the same trick seems to have been used in the Scotland

Yard case, where the crooks made their calls via the USA.)

Such cases apart, dial-through fraud is mostly driven by premium rate

services: the main culprits are crooks who are in cahoots with premium line

owners. Most companies don’t understand the need to guard their ‘dial tone’

and don’t know how to even if they wanted to. PBXes are typically run

by company telecomms managers who know little about security, while the

security manager often knows little about phones. This is changing, albeit

slowly, as VOIP technologies take over and the company phone network

merges with the data network.

Exploits of insecure end-systems sometimes affect domestic subscribers too.

A notorious case was the Moldova scam. In 1997, customers of a porn site

were told to download a ‘viewer’ program that dropped their phone line

and connected them to a phone number in Moldova (having turned off their

modem speakers so they wouldn’t notice). The new connection stayed up until

they turned off their computers; thousands of subscribers incurred hundreds

of thousands of dollars in international long distance charges at over $2 per

minute. Their phone companies tried to collect this money but there was an

outcry. Eventually the subscribers got their money back, and the Federal Trade

Commission enjoined and prosecuted the perpetrators [456]. Since then there

have been a number of copycat scams [870]; but as more and more people move

to cable modems or broadband, and their PCs are no longer able to dial out on

the plain old telephone system, this kind of abuse is getting less common. The

latest twist is premium-rate mobile malware: in 2006, for example, the Red

Browser worm cashed out by sending $5 SMSs to Russia [633].

20.2 Phone Phreaking 605

Premium rate scams and anonymous calling are not the only motives. Now

that phones are used more and more for tasks such as voting, securing entry

into apartment buildings, checking that offenders are observing their parole

terms, and authenticating ﬁnancial transactions, more motives are created for

ever more creative kinds of mischief, and especially for hacks that defeat

caller line ID. For example, caller-line ID hacks make middleperson attacks on

payment systems easier; SMS spooﬁng and attacks on the SS7 signaling in the

underlying network can have similar effects [897].

And sometimes attacks are conducted by upstanding citizens for per-

fectly honorable motives. A neat example, due to Udi Manber, is as follows.

Suppose you have bought something which breaks, and the manufacturer’s

helpline only has an answering machine. To get service, you have to take

the answering machine out of service. This can often be done by recording

its message, and playing it back so that it appears as the customer message.

With luck the machine’s owner will think it’s broken and it’ll be sent off for

maintenance.

20.2.5 Feature Interaction

More and more cases of telephone manipulation involve feature interaction.

Inmates at the Clallam Bay Correctional Center in Washington state,

who were only allowed to make collect calls, found an interesting exploit

of a system which the phone company (‘Fone America’) introduced to

handle collect calls automatically. The system would call the dialled

number and a synthesised voice would say: ‘If you will accept a col-

lect call from ...(name of caller) ...please press the number 3 on your

telephone twice’. Prisoners were supposed to state their name for the

machine to record and insert. The system had, as an additional feature,

the ability to have the greeting delivered in Spanish. Inmates did so, and

when asked to identify themselves, said ‘If you want to hear this mes-

sage in English, press 33’. This worked often enough that they could get

through to corporate PBXes and talk the operator into giving them an

outside line. The University of Washington was hit several times by this

scam [476].

A number of directory-enquiry services will connect you to the num-

ber they’ve just given you, as a service to motorists who can’t dial while

driving. But this can often be used to defeat mechanisms that depend on

endpoint identiﬁcation. Adulterers use it to prevent their spouses seeing

lovers’ numbers on the family phone bill, and naughty children use it to

call sex lines despite call barring [977].

606 Chapter 20 ■Telecom System Security

Call forwarding is a source of many scams. In the old days, it was used

for pranks, such as kids social-engineering a phone company operator

to forward calls for someone they didn’t like to a sex line. Nowadays,

it’s quite often both professional and nasty. For example, a fraudster

may tell a victim to conﬁrm her phone number with the bank by dialing

a sequence of digits — which forwards her incoming calls to a num-

ber controlled by the attacker. So the bank’s callback machanisms are

defeated when the customer isn’t aware that a certain sequence of dialed

numbers can alter the behavior of her phone.

British Telecom launched a feature called ‘Ringback’. If you dial an

engaged number, you can then enter a short code and as soon as the

called number is free, both your phone and theirs will ring. The result-

ing call is billed to you. However, when you used ringback used from a

pay phone, it was the phone’s owner who ended up with the bill. Peo-

ple with private pay phones, such as pub landlords and shopkeepers,

lost a lot of money, which the phone company was eventually obliged to

refund [652].

Conference calls also cause a lot of trouble. For example, football hooli-

gans in some countries are placed under a curfew that requires them

to be at home during a match, and to prove this by calling the proba-

tion service, which veriﬁes their number using caller ID. The trick is to

get one of your kids to set up a conference call with the probation ser-

vice and the mobile you’ve taken to the match. If the probation ofﬁcer

asks about the crowd noise, you tell him it’s the TV and you can’t turn it

down or your mates will kill you. (And if he wants to call you back, you

get your kids to forward the call.)

This brings us to the many problems with mobile phones.

20.3 Mobile Phones

Since their beginnings as an expensive luxury in the early 1980s, mobile phones

have become one of the big technological success stories. By 2007, we now

have over a billion subscribers; it’s said that over a billion phones will be sold

this year and the total subscriber base may rise to two billion. In developed

countries, most people have at least one mobile, and many new electronic

services are being built on top of them. Scandinavia has led here: you get a

ferry ticket in Helsinki by sending a text message to the vending machine, and

you get a can of Coke the same way. You can also scan a bar code at a bus stop

with your phone camera, and get sent a text message 90 seconds before the

next bus arrives; that way you don’t have to stand out in the snow.

20.3 Mobile Phones 607

Growth is rapid in developing countries too, where the wireline network

is often dilapidated and people used to wait years for phone service to be

installed. In some places it’s the arrival of mobile phone service that’s connected

villages to the world. Criminals also make heavy use of mobiles, and not just for

communications: and in large tracts of the third world, mobile phone units have

become a de facto currency. If you get kidnapped in Katanga, the kidnappers

will tell your relatives in Kinshasa to buy mobile phone units and text them

the magic numbers. In developed countries, the criminal interest is largely in

communications, and most police wiretaps are now on mobile numbers.

So mobile phones are very important to the security engineer, both as part

of the underlying infrastructure and as a channel for service delivery. They

can also teach us a lot about fraud techniques and countermeasures.

20.3.1 Mobile Phone Cloning

The ﬁrst generation of mobile phones used analog signals with no real

authentication. The handset simply sent its serial numbers in clear over the

air link. (In the U.S. system, there were two of them: one for the equipment,

and one for the subscriber.) So villains built devices to capture these numbers

from calls in the neighborhood. (I’ve even seen a phone that a student

had reprogrammed to do this by a simple software hack.) One of the main

customers was the call-sell operation that would steal phone service and resell

it cheaply, often to immigrants or students who wanted to call home. The

call-sell operators would hang out at known pitches with cloned mobiles, and

their customers would queue up to phone home for a few dollars.

So a black market developed in phone serial numbers. The call-sell mar-

ket was complemented by the market for anonymous communications for

criminals: enterprising engineers built mobile phones which used a different

identity for each call. Known as tumblers, these were particularly hard for

the police to track [636]. The demand for serial numbers grew rapidly and

satisfying it was increasingly difﬁcult, even by snooping at places like airports

where lots of mobiles were turned on. So prices rose, and as well as passive

listening, active methods started to get used.

Modern mobile phones are cellular, in that the operator divides the service

area up into cells, each covered by a base station. The mobile uses whichever

base station has the strongest signal, and there are protocols for handing off

calls from one cell to another as the customer roams. (For a survey of mobile

phone technology, see [1061].) The active attack consists of a fake base station,

typically at a place with a lot of passing trafﬁc such as a freeway bridge. As

phones pass by, they hear a stronger base station signal and attempt to register

by sending their serial numbers.

A number of mechanisms were tried to cut the volume of fraud. Most

operators developed or bought intrusion detection systems, which watch out

608 Chapter 20 ■Telecom System Security

for suspicious patterns of activity. A number of heuristics were developed.

For example, genuine mobiles which roam and call home regularly, but then

stop calling home, have usually been stolen; other indicators include too-rapid

movement (such as calls being made from New York and LA within an hour

of each other) and even just a rapid increase in call volume or duration.

In the chapter on electronic warfare, I mentioned RF ﬁngerprinting — a

formerly classiﬁed military technology in which signal characteristics that arise

from manufacturing variability in the handset’s radio transmitter are used to

identify individual devices and tie them to the claimed serial numbers [534].

Although this technique works — it was used by Vodafone in the UK to almost

eliminate cloning fraud from analogue mobiles — it is expensive as it involves

modifying the base stations. (Vodafone also used an intrusion detection

system that tracked customer call patterns and mobility, described in [1283];

their competitor Cellnet simply blocked international calls from analogue

mobiles, which helped move its high value customers to its more modern

digital network.) Another proposed solution was to adopt a cryptographic

authentication protocol, but there are limits on how much can be done without

changing the whole network. For example, one can use a challenge-response

protocol to modify the serial number [485]. But many of the mechanisms

people proposed to fortify the security of analog cellular phones have turned

out to be weak [1305].

Eventually the industry decided to upgrade to a new digital system. Revenue

protection was an issue, but far from the only one; digital systems offered more

efﬁcient use of bandwidth, and a whole host of new features — including

easier international roaming (important in Europe with lots of small countries

jammed close together), and the ability to send and receive short text messages.

(Text messages were almost an afterthought; the designers didn’t realise they’d

be hugely popular.) From the operators’ viewpoint, the move to standard

digital equipment cut costs and enabled rapid, wide-scale deployment.

20.3.2 GSM Security Mechanisms

The second generation of mobile phones adopted digital technology. Most

handsets worldwide use the Global System for Mobile Communications,orGSM,

which was designed from the start to facilitate international roaming; it was

founded when 15 companies signed up to the GSM Association in 1987, and

service was launched in 1992. As of 2007, the GSM system extends to over two

billion handsets in over 200 countries; a typical developed country has more

handsets in service than it has people [133]. The USA, Japan, Korea and Israel

had different second-generation digital standards (although the USA has GSM

service too). Since about 2001, most countries also have a third-generation

service, which I’ll describe in the next section.

20.3 Mobile Phones 609

The designers of GSM set out to secure the system against cloning and other

attacks: their goal was that GSM should be at least as secure as the wireline

system. What they did, how they succeeded and where they failed, make an

interesting case history.

The authentication protocols are described in a number of places, such

as [232] (which also describes the mechanisms in an incompatible U.S. sys-

tem). The industry initially tried to keep secret the cryptographic and other

protection mechanisms which form the core of the GSM protocols. This

didn’t work: some eventually leaked and the rest were discovered by reverse

engineering. I’ll describe them brieﬂy here.

Each network has two databases, a home location register (HLR) that contains

the location of its own mobiles, and a visitor location register (VLR) for the loca-

tion of mobiles which have roamed in from other networks. These databases

enable incoming calls to be forwarded to the correct cell.

The handsets are commodity items. They are personalised using a subscriber

identity module (SIM) — a smartcard you get when you sign up for a network

service, and which you load into your handset. The SIM can be thought of as

containing three numbers:

1. there may be a personal identiﬁcation number that you use to unlock the

card. In theory, this stops stolen mobiles being used. In practice, many

networks set an initial PIN of 0000, and most users never change it or

even use it;

2. there’s an international mobile subscriber identiﬁcation (IMSI), a unique

number that maps on to your mobile phone number;

3. ﬁnally there is a subscriber authentication key Ki, a 128-bit number that

serves to authenticate that IMSI and is known to your home network.

Unlike the banks, which used master keys to generate PINs, the phone com-

panies decided that master keys were too dangerous. So instead of diversifying

amasterkeyKM to manufacture the authentication keys as Ki={IMSI}KM,the

keys are generated randomly and kept in an authentication database attached

to the HLR.

The protocol used to authenticate the handset to the network runs as follows

(see Figure 20.1). On power-up, the SIM may request the customer’s PIN; if this

isn’t conﬁgured, or once it’s entered correctly, the SIM emits the IMSI, which

the handset sends to the nearest base station. It’s relayed to the subscriber’s

HLR, which generates ﬁve triplets. Each triplet consists of:

RAND, a random challenge;

SRES, a response; and

Kc, a ciphering key.

610 Chapter 20 ■Telecom System Security

The relationship between these values is that RAND, encrypted under the

SIM’s authentication key Ki, gives an output which is SRES concatenated

with Kc:

{RAND}Ki=(SRES|Kc)

The standard way to do this encryption is using a one-way function called

Comp128, or A3/A8. (A3 refers to the SRES output and A8 to the Kcoutput).

Comp128 is a hash function with 40 rounds, described in detail in [226], and like

most proprietary algorithms that were designed in the shadows and ﬁelded

quietly in the 1980s and 90s, it turns out to be vulnerable to cryptanalysis. The

basic design of the function is much like in Figure 5.10 — each round consists

of table lookups followed by mixing. There are ﬁve tables, with 512, 256, 128,

64 and 32 byte entries each, and the hash function uses them successively in

each block of ﬁve rounds; there are eight of these blocks. This may seem very

complex, but once its design became public, a vulnerability was soon noticed.

Four of the bytes at the output of the second round depend only on the value

of the same bytes of the input. This four-byte to four-byte channel is called a

narrow pipe and it’s possible to probe it by tweaking input bytes until you detect

a collision. Once all the details have been worked out, it turns out that you

need about 150,000 suitably chosen challenges to extract the key [1306, 1307].

The effect is that given access to a SIM issued by a network that uses Comp128,

the authentication key can be extracted in several hours using software that is

now freely available.

This attack is yet another example of the dangers of using a secret crypto

primitive that has been evaluated by only a few friends; the cryptanalytic

techniques necessary to ﬁnd the ﬂaw were well known [1287] and if Comp128

had been open to hostile public scrutiny, the ﬂaw would most probably have

been found. Thankfully, a phone company can replace Comp128 with a proper

hash function such as SHA-256 without affecting anyone else; the hash function

is present only in the SIM cards it issues to its customers and the software at

its HLR. In any case, there don’t seem to be any industrial-scale attacks based

on a vulnerable hash function; the normal user doesn’t have any incentive to

crack his Kiout from his SIM as it doesn’t let him bill calls to (or steal calls

from) anyone else.

HLRVLRBSC

SIM Mobile

Figure 20.1: GSM authentication system components

20.3 Mobile Phones 611

Anyway, the triplets are sent to the base station, which now presents

the ﬁrst RAND to the mobile. It passes this to the SIM, which computes SRES.

The mobile returns this to the base station and if it’s correct the mobile and the

base station can now communicate using the ciphering key Kc.Sothewhole

authentication protocol runs as in Figure 20.2.

SIM →HLR IMSI

HLR →BSC (RAND, SRES, Kc), ...

BSC →SIM RAND

SIM →BSC SRES

BSC →mobile {trafﬁc}Kc

Figure 20.2: GSM authentication protocol

There are several vulnerabilities in this protocol. In most countries the

communications between base stations and the VLR pass unencrypted on

microwave links1. So an attacker could send out an IMSI of his choice and

then intercept the triplet on the microwave link back to the local base station.

A German mobile operator, which offered a reward of 100,000 Deutschmarks

to anyone who could bill a call to a mobile number whose SIM card was held

in their lawyer’s ofﬁce, backed down when asked for the IMSI [44].

Second, triples can be replayed. An unscrupulous foreign network can get

ﬁve triples while you are roaming on it and then keep on reusing them to

allow you to phone as much as you want. (Home networks could stop this by

contract but they don’t.) So the visited network doesn’t have to refer back to

your home network for further authorisation — and even if they do, it doesn’t

protect you as the visited network might not bill you for a week or more. So

your home network can’t reliably shut you down while you roam and it may

still be liable to pay the roamed network the money. Dishonest networks also

defraud roaming customers by cramming — by creating false billing records, a

practice I’ll describe in more detail later. So even if you thought you’d limited

your liability by using a pre-paid SIM, you might still end up with your

network trying to collect money from you. This is why, to enable roaming

with a pre-paid SIM, you’re normally asked for a credit card number. You can

end up being billed for more than you expected.

1The equipment can encrypt trafﬁc, but the average phone company has no incentive to switch

the cryptography on. Indeed, as intelligence agencies often monitor the backhaul near major

switching nodes as an efﬁcient means of getting warrantless access to trafﬁc, a phone company

that did switch on the crypto might ﬁnd that persons unknown started jamming the link to make

them stop.

612 Chapter 20 ■Telecom System Security

The introduction of GSM caused signiﬁcant shifts in patterns of crime

generally. The authentication mechanisms made phone cloning difﬁcult, so

the villains switched their modus operandi to buying phones using stolen

credit cards, using stolen identities or bribing insiders [1352]. Robbery was

another issue. We’ve had a spate of media stories in Britain about kids being

mugged for their phones. Mobile phone crime did indeed increase 190%

between 1995 and 2002, but to keep this in context, the number of subscribers

went up 600% in the same period [583]. Some of the theft is bullying — kids

taking smaller kids’ phones; some is insurance fraud by subscribers who’ve

dropped their phones in the toilet and report them as stolen as their insurance

doesn’t cover accidental damage; but there is a hard core of theft where

muggers take phones and sell them to fences. Many of the fences either work

at mobile phone shops that have authorised access to tools for reprogramming

the International Mobile Equipment Identiﬁer (IMEI), the serial number in

the handset, or else have links to organised criminals who ship the handsets

abroad. Things are worse in Brazil, where kidnapping is endemic: there are

now fake kidnappings in which a child’s phone is snatched by a criminal who

phones its parents to demand a ransom2.

From about 1997, prepaid mobile phones were introduced. This kicked off

a period of rapid growth in the industry as the technology became available

to people without credit ratings. For example, prepaids make up 90% of the

market in Mexico but 15% in the USA. Worldwide, they’re over half, and

growing. They also made anonymous communication much more practical,

and many criminals have started using them. The issues include not just

evading police wiretapping but stalking, extortion, bullying and other kinds

of harassment. Prepaids also facilitate simple frauds; if your identity isn’t

checked when you buy a phone, there’s little risk to you if you recharge it with

a stolen credit card [343].

It must be said though that most people who use prepaid phones for crime

are only getting lightweight anonymity, and remain undetected only because

the police don’t put serious effort into catching petty crooks. If a really serious

crime is committed, trafﬁc analysis will be used, and most criminals don’t

have any clue of the level of operational discipline needed to stop this. As I

already remarked, the alleged 9/11 mastermind Khalid Shaikh Mohammed

was caught when he used a prepaid SIM from the same batch as one that had

been used by another Al-Qaida member; and after the failed 21/7 London

bombings, the would-be bomber Husein Osman ﬂed to Rome, where he was

promptly caught. He had changed the SIM in his mobile phone en route; but

2There are also completely fake kidnappings in which the bad guys just lie: they say they’ve

snatched the child, and if they call its phone it will be killed. The perpetrators of these crimes

are often in prison and have little to lose. 20% of parents still pay up rather than take the

risk.

20.3 Mobile Phones 613

call records show not just the IMSI from the SIM, but also the IMEI from the

handset. If you’ve got all the world’s police after you, just changing the SIM

isn’t anything like enough. Operational security requires a detailed technical

understanding of how networks operate, and levels of training and discipline

that are most unusual outside national intelligence agencies.

Finally, prepaid mobiles were a gift to crooked call-sell operators. As the

billing infrastructure is only invoked when a phone goes on-hook, it’s possible

for a bad guy to make a call last all day: the numbers called by each of his

clients are simply added to a long conference call one after the other. At the

end of the day, the phone goes on-hook, a bill for thousands of dollars is

generated, the alarm goes off, and the crook tosses the phone in the river. The

following day he buys another. That’s why network operators typically tear

down any call that lasts more than a few hours.

In addition to authentication, the GSM system is supposed to provide two

further kinds of protection — location security and call content conﬁdentiality.

The location security mechanism is that once a mobile is registered to a

network, it is issued with a temporary mobile subscriber identiﬁcation (TMSI),

which acts as its address as it roams through the network. The attack on

this mechanism uses a device called an IMSI-catcher, which is sold to police

forces [488]. The IMSI-catcher, which is typically operated in a police car tailing

a suspect, pretends to be a GSM base station. Being closer than the genuine

article, its signal is stronger and the mobile tries to register with it. The IMSI

catcher claims not to understand the TMSI, so the handset helpfully sends it

the cleartext IMSI. This feature is needed if mobiles are to be able to roam

from one network to another without the call being dropped, and to recover

from failures at the VLR [1283]. The police can now get a warrant to intercept

the trafﬁc to that mobile or — if they’re in a hurry — just do a middleperson

attack in which they pretend to be the network to the mobile and the mobile

to the network.

The GSM system is supposed to provide call content conﬁdentiality by

encrypting the trafﬁc between the handset and the base station once the

authentication and registration are completed. The speech is digitized, com-

pressed and chopped into packets; each packet is encrypted by xor-ing it with

a pseudorandom sequence generated from the ciphering key Kcand the packet

number. The algorithm commonly used in Europe is A5/1.

A5/1, like Comp128, was originally secret; like Comp128, it was leaked

and attacks were quickly found on it. The algorithm is shown in Figure 20.3.

There are three linear feedback shift registers of lengths 19, 22 and 23 and their

outputs are combined using exclusive-or to form the output keystream. The

nonlinearity in this generator comes from a majority-clocking arrangement

whereby the middle bits ciof the three shift registers are compared and the

two or three shift registers whose middle bits agree are clocked.

614 Chapter 20 ■Telecom System Security

m = Majority (C1, C2, C3 )

Figure 20.3: A5 (courtesy of Alex Biryukov and Adi Shamir)

The obvious attack on this arrangement is to guess the two shorter registers

and then work out the value of the third. As there are 41 bits to guess, one

might think that about 240 computations would be needed on average. It’s

slightly more complex than this, as the generator loses state; many states have

more than one possible precursor, so more guesses are needed. This led to an

attack using a lot of FPGAs that is sold for about $1m3. Then Biryukov and

Shamir found some optimizations and tradeoffs that let A5 be broken with

much less effort. Their basic idea is to compute a large ﬁle of special points

to which the state of the algorithm converges, and then look for a match with

the observed trafﬁc. Given this precomputed ﬁle, the attack can use several

seconds of trafﬁc and several minutes’ work on a PC, or several minutes of

trafﬁc and several seconds’ work [173]. Reverse engineering actual systems

also showed that the keying of A5 was deliberately weakened. Although in

theory A5 has a 64-bit key (the initial loads for the shift registers) the actual

implementations set the ten least signiﬁcant key bits to zero. Anyway, the

response of the GSM vendors to these disclosures was to introduce a third

cipher, A5/3, which is based on a strong block cipher known as Kasumi that’s

also used in third-generation mobile phones.

3A system of 15 boxes, each with 20 cards, each with 18 ICs, each with 32 cores, each running at

150MHz, checking one key per Hz, takes 7.5 sec per key and burns 15KW.

20.3 Mobile Phones 615

However, this attempt to ﬁx the content-conﬁdentiality problem has been

undermined by an attack that exploits the underlying authentication protocol

and that is now coming into widespread use by police and surveillance folks.

It relies on the fact that phone companies in many countries use a weakened

version of A5/1, called A5/2. Mobile phones are generally able to use either

algorithm, so that they can roam freely. The attack was invented by Elad

Barkan, Eli Biham and Nathan Keller [117], and runs as follows. If you’re

following a suspect who uses his mobile, you record the call, including the

initial protocol exchange of challenge and response. Once he’s ﬁnished, you

switch on your IMSI-catcher and cause him to register with your bogus base

station. The IMSI-catcher tells his phone to use A5/2 rather than A5/1, and

a key is duly set up — with the IMSI-catcher sending the challenge that was

used before. So the mobile phone generates the same key Kcas before. As this

is now being used in a weak cipher, it can be cracked quickly, giving access to

the target conversation recorded previously. In fact it doesn’t matter whether

that was protected using the medium-security A5/1 or the high-security A5/3.

The facts that there’s a low-security algorithm, that keys are determined by one

of the two principals, and that keys are shared between algorithms, together

make the whole system weak.

The A5/1 vulnerability was introduced following pressure from Europe’s

intelligence agencies, and the ostensible justiﬁcation was that they didn’t want

even halfways decent security exported to potentially hostile countries, as it

would be harder to spy on them. The conspiracy theorists had a ﬁeld day with

all this — for example, there was a political row in Australia when it turned

out that A5/2 was being used there, as it was seen as implying that European

governments saw Australia as an intelligence target rather than an ally.

The truth is, as always, more subtle. Weak ciphers can deﬁnitely help in some

tactical situations. Consider the case I mentioned in the chapter on electronic

warfare, where the New Zealand navy sent a frigate to monitor a coup in Fiji.

If the Fijian phone company had been allowed to use A5/1 rather than A5/2,

this would not have frustrated the mission: the sigint ofﬁcers could snatch the

triplets off the microwave links, hack the location register, and if all else fails,

brute force a key. But being able to break trafﬁc quickly is very convenient.

On the other hand, imposing weak cipher security on less developed

countries can also have operational costs. In 2007, the Taleban started tapping

mobile phone calls made by British soldiers to friends and relatives back home,

whom they then called. The wife of one RAF ofﬁcer was told: ‘You’ll never see

your husband alive — we have just killed him’. It was some hours before she

could conﬁrm that he was still safe and well [603]. British troops have now

been banned from using mobile phones — which doesn’t exactly help morale.

In domestic or friendly-country operations, it’s even more complex. The

agencies can get lawful access to cleartext from phone companies, but some-

times the warrantry procedures are (or are felt to be) too cumbersome. Hence

616 Chapter 20 ■Telecom System Security

the sweetheart deals with some phone companies to get access without war-

rants; this has led to a number of scandals recently. For example, it emerged

that AT&T had been giving the US authorities access to itemised billing data for

years. Sweetheart deals were common with companies that used to be monop-

olies, but are becoming less common as most countries have now deregulated

telecomms and have hundreds of phone companies, not just one. Some phone

companies are distrusted by the authorities because they’re owned by foreign-

ers or even by gangsters. Others are tiny, have never been tapped before, the

police just don’t know who to talk to. Other companies are respectable and

willing to comply with warrants, but unable to do so. One of Britain’s most

closely guarded law-enforcement and intelligence secrets for several years

was that one of the mobile phone networks couldn’t be conveniently tapped.

Engineers tried to hook up their complex, messy systems to the intelligence

community’s complex, messy systems but the project went off the rails and

they just could not get the thing to work. The police were really scared that

crooks might ﬁnd this out, and migrate en masse to that network.

How are we to make sense of this mess? It’s as well to admit that there are

many reasons, some honourable and some less so, why government employees

may want to get access to trafﬁc by technical means rather than by using the

lawful interfaces provided for the purpose. But that doesn’t mean that such

access should be allowed. The huge improvements in liberty, prosperity and

quality of life that the West has enjoyed since the Enlightenment and the

Industrial Revolution are due in large part to our replacing the divine right

of kings with freedom under the law in a democracy. We elect legislators to

make the rules, and if we decide through them that the police can’t torture

suspects any more, then they’d better not try. Similarly, if our legislators

decide that police and intelligence agencies should get warrants to wiretap,

then the agencies must obey the law like anyone else. The GSM security story

provides a good example of how the deliberate insertion of vulnerabilities can

have wide-ranging and unforeseen engineering consequences. It must also be

said that the facilities that phone companies and ISPs are now being compelled

to provide for properly warranted law-enforcement access are often so poorly

engineered that they can be abused [1151]. In 2004–5, persons unknown

tapped the mobile phones of the Greek Prime Minister and about a hundred of

that country’s political, law enforcement and military elite, by subverting the

wiretapping facilities built into Vodafone’s Greek network. Both Vodafone,

and their equipment supplier Ericsson, were heavily ﬁned [1042]. Colleagues

and I warned about this problem years ago [4] and I expect it to get worse. I’ll

discuss it at greater length in Part III.

There are further policy issues with location privacy. There was a storm

in Switzerland in 1997 when the press found that the phone company was

routinely giving location data to the police [1030], while in the USA, the FCC

ordered mobile phone companies to be able to locate people ‘so that 911 calls

20.3 Mobile Phones 617

could be dispatched to the right place’. This was imposed on every user of

mobile phone service, rather than letting users decide whether to buy mobile

location services or not. Privacy activists were not happy with this.

Anyway, the net effect is that the initial GSM security mechanisms provided

slightly better protection than the wireline network in countries allowed to

use A5/1, and slightly worse protection elsewhere, until the protocol attacks

were discovered and exploited. Now privacy is slightly worse everywhere,

as people with the right equipment can get fairly straightforward access to

trafﬁc. But it’s probably not a big deal. Relatively few people ever get followed

around by an investigator with professional snooping equipment — and if

you’re such a person, then ways to detect and prevent semi-active attacks on

your mobile phone are just a small part of the tradecraft you need to know. If

you’re an average subscriber, the privacy threat comes from possible abuse of

data collected by the phone company, such as itemized billing data. From that

viewpoint, the vulnerabilities in the communications security mechanisms

neither expose you to additional wiretapping, nor prevent the frauds that are

likely to cause you the most grief.

20.3.3 Third Generation Mobiles – 3gpp

The third generation of digital mobile phones was initially known as the

Universal Mobile Telecommunications System (UMTS) and now as the Third

Generation Partnership Project (3gpp, or just 3g). These systems are now available

almost everywhere, the exception being in China which is working on its own

proprietary variant. Elsewhere, the security is much the same as GSM, but

upgraded to deal with a number of GSM’s known vulnerabilities. Third

generation systems entered service in 2003–2004; the main advantage of 3g

over GSM is higher data rates; instead of the 9.6kb/s of GSM and the tens of

kilobits per second of GPRS, third-generation data rates are in the hundreds

of thousands to millions of bits per second. The vision is that 3g will enable

all sorts of mobile services, from mobile TV to laptops that just go online

anywhere.

The overall security strategy is described in [1310], and the security archi-

tecture is at [1298]. The crypto algorithms A5/1, A5/2 and Comp128 are

replaced by various modes of operation of a block cipher called Kasumi [696].

Kasumi is public and is based on a design by Mitsuru Matsui called Misty,

which was properly peer-reviewed and has now withstood public scrutiny

for a decade [844]. All keys are now 128 bits. Cryptography is used to protect

the integrity and conﬁdentiality of both message content and signalling data,

rather than just content conﬁdentiality, and the protection at least runs from

the handset to a fairly central node, rather than simply to the local base station.

This means the picking up the triples, or the plaintext, from the microwave

backhaul is no longer an attack. The authentication is now two-way rather

618 Chapter 20 ■Telecom System Security

than one-way, ending the vulnerability to rogue base stations; so IMSI-catchers

don’t work against third generation mobiles. Instead, there is a properly engi-

neered interface for lawful interception [1299]. This can supply key material

as well as plaintext, so that if the police follow a suspect, record a call and

identify the mobile in the process, they can decrypt that call later, rather than

being limited to the plaintext of calls recorded by the phone company after

they get their warrant approved.

The protocol mechanics work as follows (see Figure 20.4). In the basic 3gpp

protocol, the authentication is pushed back from the base station controller to

the visitor location register. The home location register is now known as the

home environment (HE) and the SIM as the UMTS SIM (USIM). The home envi-

ronment chooses a random challenge RAND as before and enciphers it with

the USIM authentication key Kto generate a response RES, a conﬁdentiality

key CK, and integrity key IK, and an anonymity key AK.

{RAND}K=(RES|CK|IK|AK)

There is also a sequence number SEQ known to the HE and the USIM.

A MAC is computed on RAND and SEQ, and then the sequence number

is masked by exclusive-or’ing it with the anonymity key. The challenge, the

expected response, the conﬁdentiality key, the integrity key, and the masked

sequence number made up into an authentication vector AV which is sent from

the HE to the VLR. The VLR then sends the USIM the challenge, the masked

sequence number and the MAC; the USIM computes the response and the

keys, unmasks the sequence number, veriﬁes the MAC, and if it’s correct

returns the response to the VLR.

USIM →HE IMSI (this can optionally be encrypted)

HE →VLR RAND, XRES, CK,IK,SEQ ⊕AK,MAC

VLR →USIM RAND, SEQ ⊕AK,MAC

USIM →VLR RES

Figure 20.4: 3gpp authentication protocol

The UMTS standards set out are many other features, including details

of sequence number generation, identity and location privacy mechanisms,

backwards compatibility with GSM, mechanisms for public-key encryption of

authentication vectors in transit from HEs to VLRs, and negotiation of various

optional cryptographic mechanisms.

The net effect is that conﬁdentiality will be improved over GSM: eaves-

dropping on the air link is prevented by higher-quality mechanisms, and

the current attacks on the backbone network, or by bogus base stations, are

excluded. Police wiretaps are done at the VLR. In a number of countries,

third-generation mobiles were hard for the police to tap in the ﬁrst few years,

20.3 Mobile Phones 619

as they had to learn to operate through formal channels and also to integrate

their systems with those of the network operators. In the second phase, it’s

proposed to have end-to-end encryption, so that the call content and some of

the associated signaling will be protected from one handset to another. This

led to government demands for a key escrow protocol — a protocol to make

keys available to police and intelligence services on demand. The catch is that

if a mobile phone call takes place from a British phone company’s subscriber

using a U.S. handset, roaming in France, to a German company’s subscriber

roaming in Switzerland using a Finnish handset, and the call goes via a long

distance service based in Canada and using Swedish exchange equipment,

then which of these countries’ intelligence agencies will have access to the

keys? [1299] (Traditionally, most of them would have had access to the call

content one way or another.)

One solution pushed by the agencies in Britain and France is the so-called

Royal Holloway protocol [663], designed largely by Vodafone, which gives

access to the countries where the subscribers are based (so in this case, Britain

and Germany). This is achieved by using a variant of Difﬁe-Hellman key

exchange in which the users’ private keys are obtained by encrypting their

names under a super-secret master key known to the local phone company

and/or intelligence agency. Although this protocol has been adopted in the

British civil service and the French health service, it is at odds with the phone

company security philosophy that master keys are a bad thing. The protocol

is also clunky and inefﬁcient [76].

So 3gpp won’t provide a revolution in conﬁdentiality, merely a mod-

est improvement. As with GSM, its design goal is that security should be

comparable with that of the wired network [621] and this looks like being

achieved.

20.3.4 Platform Security

The ﬁnal point I need to make here is that as mobile phones become more

widespread and more programmable, they may suffer from the malware

problems that have plagued the PC. They have followed the pattern predicted

by security economics. At ﬁrst, the platform vendors — the ﬁrms selling

operating systems, such as Symbian, Microsoft and Linux — didn’t incorporate

much security, as it would have got in application developers’ way and

appealing to complementers is vital when building share in a new market with

network externalities. Then, as one platform pulled ahead of the others, the

malware writers targeted it. In 2007, viruses and worms are being detected at

the rate of about 300 per annum for Symbian phones, and one or two a year for

the others. Symbian has started a program of hardening their platform, with

progressively more sophisticated access controls, code signing, and so on.

620 Chapter 20 ■Telecom System Security

Mobile phone platforms have also acquired DRM mechanisms. The Open

Mobile Alliance DRM version 1 supports the download of ringtones, while

version 2 supports music and video download with more general mechanisms.

Version 1 is widely used, but version 2 has been held up by an interesting dis-

pute. The main OMA promoter, Nokia, would like to become the distribution

channel of choice for mobile music, just as Apple has become for PCs with

iTunes. However the network operators are extremely reluctant to let Nokia

have a business relationship with their customers directly, and this has held

up deployment.

In general, security is made more difﬁcult by the long and complex supply

chain that delivers mobile phone service to customers. IP companies like

ARM own the chip designs; foundries such as Inﬁneon make the chips;

handset designers like Samsung manufacture the actual mobiles; Symbian

provides the operating system; Nokia may provide some more software for

a download interface; a network operator such as Vodafone provides the

national infrastructure; and a local operating company then bills the customer.

There has been a tendency for everyone in this chain to see security as a

problem to be tossed over the fence to the companies on either side. On top of

this there might be apps provided by games vendors, and corporate apps such

as ﬂeet management that are provided by third-party software houses. Add

the next generation of location-based services and offerings from the likes of

Google and eBay, and the whole thing becomes complex beyond belief.

One ﬁnal aspect of mobile phone platforms is locking. In many countries,

handsets are subsidised out of future call revenue: you get a cheap phone

in return for a year’s contract with a network. The downside is that your

phone is locked to the network. Even some prepaid phones are locked, and

carry an implicit subsidy from expected future token sales. However, in some

countries, this business model isn’t permitted. There has thus arisen a brisk

trade in unlocking tools and services, some of which are devised by ﬁnding

exploits in the phone software, and others using the kind of hardware reverse-

engineering techniques I described in Chapter 16. Legal skirmishing between

the phone companies and the unlocking services came to a head after Apple

launched the iPhone, which had a twist on this business model: the networks

for which you could buy it were those that had paid Apple the most for

the privilege. This annoyed iPhone purchasers as the U.S. network of choice,

AT&T, was fairly slow. The iPhone was duly hacked, and AT&T sent its

lawyers after the unlockers. It also shipped a software upgrade that disabled

unlocked phones.

Laws on locking and unlocking vary widely. In the USA, the Digital

Millennium Copyright Act (DMCA), which prohibits interference with any

technical mechanism that enforces copyright, has a speciﬁc exemption for

cellphone unlocking, in order that copyright law shouldn’t be abused to

stiﬂe competition in the mobile phone market. It’s now being argued that

20.3 Mobile Phones 621

this exemption covers only people who unlock their own phones, but doesn’t

extend to the sale of unlocking tools or services [821]. So in America, unlocking

is legal if you’re a customer (though the provider may brick your phone), and

may be open to challenge if you do it commercially. At the other end of the

scale, courts in France and Germany have held mobile phone locking to be

illegal: Apple will have to offer unlocked models for sale there if it offers

the product at all. Such variations in law and business practice have led to the

development of a thriving grey market whereby phones are shipped from one

country to another; this in turn has driven secondary markets for unlocking

tools and even for assorted frauds.

20.3.5 So Was Mobile Security a Success or a Failure?

Whether mobile-phone security has been a success or a failure depends on

whom you ask.

From the point of view of cryptography, it was a failure. Both the Comp128

hash function and the A5 encryption algorithm were broken once they

became public. In fact, GSM is often cited as an object lesson in Kerckhoffs’

Principle — that cryptographic security should reside in the choice of the key,

rather than in the obscurity of the mechanism. The mechanism will leak sooner

or later and it’s better to subject it to public review before, rather than after, a

hundred million units have been manufactured. (GSM security wasn’t a dis-

aster for most cryptographers, of course, as it provided plenty opportunities

to write research papers.)

From the phone companies’ point of view, GSM was a success. The share-

holders of GSM operators such as Vodafone have made vast amounts of

money, and a (small) part of this is due to the challenge-response mechanism

in GSM stopping cloning. The crypto weaknesses were irrelevant as they were

never exploited (at least not in ways that did signiﬁcant harm to call revenue).

There are one or two frauds that persist, such as the long conference call trick;

but on balance the GSM design has been good to the phone companies.

From the criminals’ point of view, GSM was also ﬁne. It did not stop them

stealing phone service: the modus operandi merely changed, with the cost

falling on credit card companies or on individual victims of ‘identity theft’ or

street robbery. It did not stop calls from anonymous phones; the rise of the

prepaid phone industry made them even easier. (The phone companies were

happy with both of these changes.) And of course GSM did nothing about

dial-through fraud.

From the point of view of the large-country intelligence agencies, GSM was

ﬁne. They have access to local and international trafﬁc in the clear anyway,

and the weakened version of A5 facilitates tactical signint against developing

countries. And the second wave of GSM equipment is bringing some juicy

features, such as remote control of handsets by the operator [1061]. If you can

622 Chapter 20 ■Telecom System Security

subvert (or masquerade as) the operator, then there seems to be nothing to

stop you quietly turning on a target’s mobile phone without his knowledge

and listening to the conversation in the room.

From the point of view of the police and low-resource intelligence agencies,

things are not quite so bright. The problem isn’t the added technical complexity

of GSM networks: court-ordered wiretaps can be left to the phone company

(although ﬁnding the number to tap can be a hassle if the suspect is mobile).

The problem is the introduction of prepaid mobile phones. This not only

decreases the signal to noise ratio of trafﬁc analysis algorithms and makes it

harder to target wiretaps, but also encourages crimes such as extortion and

stalking.

From the customer’s point of view, GSM was originally sold as being com-

pletely secure. Was this accurate? The encryption of the air link certainly did

stop casual eavesdropping, which was an occasional nuisance with analog

phones. (There had been some high-proﬁle cases of celebrities being embar-

rassed, including one case in Britain where Prince Charles was overheard

talking to his mistress Camilla Parker-Bowles before his divorce from Princess

Diana, and one in the USA involving Newt Gingrich.) But almost all the phone

tapping in the world is done by large intelligence agencies, to whom the

encryption doesn’t make much difference.

Things are even less positive for the subscriber when we look at billing.

Cryptographic authentication of handsets can’t stop the many frauds perpe-

trated by premium rate operators and phone companies. If anything it makes

it harder to wriggle out of bogus charges, as the phone company can say in

court that your smartcard and your PIN must have been used in the handset

that made the call. The same will apply to 3rd generation phones. The one

minor compensation is that GSM facilitated the spread of prepaid phones,

which can limit the exposure.

So the security features designed into GSM don’t help the subscriber much.

They were designed to provide ‘security’ from the phone company’s point of

view: they dump much of the toll fraud risk, while not interrupting the ﬂow

of premium rate business — whether genuine or fraudulent.

In the medium term, the one ray of comfort for the poor subscriber is that

the increasing complexity of both handsets and services may create regulatory

pressure for transparent mechanisms that enable the customer to control the

amount she’s billed. There are a number of factors pushing for this, such as

the growing vulnerability of platforms to malware; and a number of factors

pushing in the other direction, such as the phone companies’ desire to keep

pricing opaque so that they can rip customers off. I mean this not just in the

strict sense that phone companies often defraud their customers, but also in

the colloquial sense that confusion pricing is a mainstay of phone company

economics. I’ll go into this in more detail in the next section, where I’ll look at

the economics of telecomms and how they relate to fraud and abuse.

20.3 Mobile Phones 623

20.3.6 VOIP

The latest development in telephone is voice over IP (VOIP), in which voice

trafﬁc is digitised, compressed and routed over the Internet. This had experi-

mental beginnings in the 1970s; products started appearing in the 1990s; but

decent call quality requires bandwidth of about 80 kbit/sec, more than can be

got from a dial-up modem. As a result, VOIP only really took off once a crit-

ical mass of households had broadband connections. Since about 2005, it has

become big business, with eBay purchasing Skype in 2006 for $2.6bn. In fact,

most normal phone calls are digitized and sent over IP networks belonging to

the phone companies, so in a technical sense almost all phone calls are ‘VOIP’,

but in line with common usage I’ll use the term only for those calls made by

customers over an IP service that they access directly, via their ISP.

The VOIP market is still fragmented, and suffers many of the problems

you’d expect. Most products were shipped quickly in a race to market, with

security an afterthought at best. There was the usual crop of stack overﬂows

and other vulnerabilities at the implementation level. The most popular VOIP

protocol, the Session Initiation Protocol (SIP), turns out to have vulnerabilities

that enable third parties to wrongly bill calls to subscribers [1375]. These could

give rise to difﬁcult disputes between VOIP services and their customers.

There are many issues with VOIP as the market shakes down. The lead-

ing system, Skype, is a closed system using proprietary protocols while its

many competitors are fragmented. One business opportunity is ‘click-to-call’

whereby an advertisement could contain a VOIP link enabling a prospective

customer to click to speak to a representative; early efforts have run up against

compatibility problems. Technical problems range from differing protocols

through how to deal with the jitter (time delay variation) caused by the vari-

able quality of service on the Internet, and how to deal with network address

translation at ﬁrewalls.

The interaction with security is complex. Corporate security policies can

result in ﬁrewalls refusing to pass VOIP trafﬁc. Phone calls can be more secure

if made over VOIP, as encryption is easy to add and with some services

(such as Skype) it comes turned on by default. For some time at least, it

may be more difﬁcult for police and intelligence services to get access to call

contents and to records of who called whom; many services have not yet got

round to engineering a law-enforcement interface, and in the case of Skype, its

peer-to-peer nature might make that difﬁcult. The FBI is currently pushing for

the CALEA regulations to be applied rigorously to VOIP providers, while the

industry and civil liberties groups are resisting. (I’ll have more to say about

this in Part III.) Wiretaps are not the only issue; click-to-call clearly will have

other security, privacy and safety issues in the context of social networking

sites, especially those used by minors.

624 Chapter 20 ■Telecom System Security

A more high-proﬁle regulatory issue is that governments want emergency

calls made through VOIP services to work reliably, and provide information

about the location of the caller. This is hard; an IP packet stream can be coming

from anywhere, and no-one owns enough of the Internet to guarantee quality

of service. At a deeper level than that, the dispute over dependability is just the

latest tussle in the forty years’ war between computer companies and phone

companies, which I’ve alluded to from time to time. Computer companies are

innovative and entrepreneurial, racing to market with products that are just

about good enough; phone companies are slow-moving and heavily regulated,

with a ﬁfteen-year product cycle and services engineered for 99.999% availabil-

ity. Ever since people started using modems on a large scale in the 1960s, there

has been one battle after another — which the computer companies pretty well

always win. A glimpse into the future may have been provided by a two-day

Skype outage in August 2007, caused by poorly-engineered protocols. After

a large number of computers rebooted following Microsoft’s ‘patch Tuesday’

security update, tens of millions of Skype clients simultaneously tried to con-

tact the network. The congestion caused the network to fail, yet the clients

kept on trying. We can probably expect a lot more failures. Although a VOIP

handset looks like a phone and works like a phone, it’s more like email in

terms of reliability. If the power goes off, so does your service.

The main problems beyond that have to do with the incompatibility of

the phone companies’ business model and the nature of the Internet. Phone

companies make their money by charging you vastly different rates for

different types of content: their typical rates work out at a hundredth of a

cent per megabyte for cable TV, eight cents per megabyte for wireline phone,

three dollars a megabyte for mobile phone, and a whopping three thousand

dollars a megabyte for text messages. The opportunity exploited by VOIP is to

arbitrage these, and it’s hardly surprising that the phone companies do what

they can to get in the way. ISPs who are also phone companies deliberately

cause problems for VOIP to stop it competing with their phone service; this is

particularly pronounced with mobile IP service. For these reasons, we’d better

look at the economics of phone companies and the interaction with security.

20.4 Security Economics of Telecomms

Phone companies are classic examples of a business with extremely high

ﬁxed costs and very low marginal costs. Building a nationwide network costs

billions and yet the cost of handling an additional phone call is essentially zero.

As I discussed in Chapter 7 on Economics, this has a couple of implications.

First, there’s a tendency towards dominant-ﬁrm markets in which the

winner takes all. Indeed for many years telephone service was considered in

20.4 Security Economics of Telecomms 625

most countries to be a ‘natural monopoly’ and operated by the government;

the main exception was the USA where the old AT&T system was heavily

regulated. After the breakup of AT&T following an antitrust case, and Margaret

Thatcher’s privatisation of BT, the world moved to a different model, of

regulated competition. The details vary from one country to another but, in

general, some sectors (such as mobile phones) had a ﬁxed number of allowed

competitors; others (such as long-distance provision) were free for companies

to compete in; and others (such as local loop provision) remained de facto

monopolies but were regulated.

Second, because the marginal cost of service provision is zero, the compet-

itive sectors (such as long-distance calling) saw prices drop quickly to a very

low level — in many cases below the level needed to recoup the investment.

(The large investments made during the dotcom bubble end up being treated

as sunk costs.)

In such a market, ﬁrms will try to maintain price discrimination by whatever

means they can. In many telecomms markets, the outcome is confusion pric-

ing — products are continually churned, with new offerings giving generous

introductory discounts to compete with the low-cost providers, but with rates

sneakily raised afterwards. The effect is to discriminate between ‘people who

will spend money to save time, and people who will spend time to save

money’. If you can be bothered to continually check prices, you can get really

good deals, but often at the cost of indifferent service. If you don’t have the

time to keep scrutinising your phone bills, and the latest emails you get from

your ISP advising you of the latest service changes, you can ﬁnd that the

call home you made from your mobile while on business in France just ate

up all that month’s savings. In the end, you can end up paying about the

same amount per month that you did in the past. Andrew Odlyzko, a scholar

of phone-company economics, suggests the eventual way forward will be

ﬁxed-price contracts: for a certain amount per month you’ll get a home phone,

ISP service, a couple of mobiles and a decent allowance of air minutes and

texts [982]. In the meantime, telecomms pricing remains murky, contentious

and far from transparent. This leads directly to abuse.

20.4.1 Frauds by Phone Companies

One of the steadily growing scams is the unscrupulous phone company that

bills lots of small sums to unwitting users. It collects phone numbers in various

ways. (For example, if you call an 800 number, then your own number will

be passed to the far end regardless of whether you tried to block caller line

ID.) The wicked phone company then bills you a few dollars. Your own phone

company passes on this charge and you ﬁnd there’s no effective way to dispute

it. Sometimes the scam uses a legal loophole: if you call an 800 number in the

USA, the company may say ‘Can we call you right back?’ and if you agree

626 Chapter 20 ■Telecom System Security

then you’re deemed to have accepted the charges, which are likely to be at a

high premium rate. The same can happen if you respond to voice prompts as

the call progresses. These practices are known as cramming.

I was myself the victim on an attempt at cramming. On holiday in Barcelona,

my wife’s bag was snatched, so we called up and cancelled the phone that

she’d had in it. Several months later, we got a demand from our mobile

provider to pay a few tens of dollars roaming charges recently incurred by that

SIM card in Spain. In all probability, the Spanish phone company was simply

putting through a few charges to a number that they’d seen previously, in the

knowledge that they’d usually get away with it. My mobile service provider

initially insisted that even though I’d cancelled the number, I was still liable for

calls billed to it months afterwards and had to pay up. I got out of the charges

only because I’d met the company’s CEO at an academic seminar and was

able to get his private ofﬁce to ﬁx the problem. Customers without such access

usually get the short end of the stick. Indeed, UK phone companies’ response

to complaints has been to offer its customers ‘insurance’ against fraudulent

charges. That they can get away with this is a clear regulatory (and indeed

policing) failure.

Another problem is slamming — the unauthorized change of a subscriber’s

long distance telephone service provider without their consent. The slammers

tell your local phone company that you have opted for their service; your

phone company routes your long distance calls through them; they hope you

don’t notice the change and dispute the bill; and the telephone charges can

then be jacked up. Some local phone companies, such as Bell Atlantic, allow

their customers to freeze their chosen long distance carrier [22].

It would be a mistake to assume that cramming and slamming are just done

by small ﬂy-by-night operators. AT&T is one of the worst offenders, having

been ﬁned $300,000 not only for slamming, but for actually using forged

signatures of subscribers to make it look as if they had agreed to switch to

their service. They got caught when they forged a signature of the deceased

spouse of a subscriber in Texas [390]. As for the UK, slamming wasn’t even

made illegal until 2005.

Another problem is the ﬂy-by-night phone company. As anyone in the USA

is legally entitled to set up a phone company, it is straightforward to set one

up, collect some cash from subscribers, and then vanish once the invoices

for interconnect fees come in. Companies also advertise sex lines with normal

phone numbers to trap the unwary, then send huge bills to the subscriber at his

residential addresses and try to intimidate him into paying. The regulation of

premium-rate providers and their business practices is a widespread problem.

And it’s not just the small operators that indulge in sharp practice. An

example that affects even some large phone companies is the short termination

of international calls.

20.4 Security Economics of Telecomms 627

Although premium-rate numbers are used for a number of more or less

legitimate purposes such as software support, many of them exploit minors

or people with compulsive behaviour disorders. So regulators have forced

phone companies in many countries to offer premium-rate number blocking

to subscribers. Phone companies get round this by disguising premium rate

numbers as international ones. I mentioned scams with Caribbean numbers in

section 20.2.1 above. Now many other phone companies from small countries

with lax regulators have got into the act, and offer sex line operators a range

of numbers on which they share the revenue.

Often a call made to a small-country phone company doesn’t go anywhere

near its ostensible destination. One of the hacks used to do this is called short

termination, and here’s an example that surfaced a few years ago. Normally

calls for the small Paciﬁc country of Tuvalu went via Telstra in Perth, Australia,

where they were forwarded by satellite. However, the sex line numbers were

marked as invalid in Telstra’s system, so they were automatically sent via the

second-choice operator — a company in New Zealand. (The girls — or to be

more precise, the elderly retired hookers who pretend to be young girls — were

actually in Manchester, England.) Technically, this is an interesting case of

a fallback mechanism being used as an attack vehicle. Legally, it is hard to

challenge as there is an international agreement (the Nairobi Convention) that

stops phone companies selectively blocking international destinations. So if

you want to stop your kids phoning the sex line in Tuvalu, you have to block

all international calls, which makes it harder for you to phone that important

client in Germany.

Problems like these are ultimately regulatory failures, and they are increas-

ingly common. (For example, in the Moldova scam I mentioned earlier, the

calls didn’t go to Moldova but to Canada [251].) They are continuing to get

worse as technology makes new, complex services possible and the regulators

fail to keep up.

20.4.2 Billing Mechanisms

Billing mechanisms are a growing source of problems, as the economic forces

discussed above lead to ever-more-complex rate cards. Even the phone com-

panies themselves sometimes fall foul of the growing complexity. Sometimes

their rates get so complex that people can arbitrage against them; there has

been more than one case in which an entrepreneur found he could set up an

international premium-rate service and be paid more per minute for calling it,

than it cost him to call it using the best discount rate. Phone companies have

tried to recover such trading proﬁts through the courts, claiming fraud — with

mixed success.

The security of the billing mechanisms covers a much wider range of issues.

Present arrangements are inadequate for a number of reasons.

628 Chapter 20 ■Telecom System Security

Acall detail record (CDR) is only generated once the calling phone goes

on-hook. This was a long-established feature of wireline networks, but

once the environment changed to mobile it became a serious problem.

As I mentioned above, someone running a call-sell operation can set up

a long conference call using a stolen or prepaid mobile, which clients join

and leave one after the other. The phone stays off-hook continuously for

hours. As soon as it goes on-hook, a CDR for several thousand dollars

is generated, and the alarm goes off. The operator throws the phone in

the river and starts using the next one. By 1996, this had become so seri-

ous that Vodafone introduced a six hour limit on all mobile calls. But

it won’t be acceptable to just drop all 3gpp calls after six hours. Many

users are expected to have always-on internet connections (such as from

their laptops) with relatively light packet trafﬁc most of the time.

More seriously, the back-end accounting system was designed in the

days when phone companies were sleepy government departments or

national monopolies, and there were no premium-rate services through

which real money could be extracted from the system. So it has little in

the way of audit and non-repudiation. In effect, phone company A tells

phone company B, ‘please debit your customer no. X the sum of $Y and

send me the money’ — and it does. Even when these debits are mistaken

or fraudulent, phone company B has no incentive to quibble, as it gets a

cut. The result, as we saw with the cramming cases above, is that fraud

slowly rises as insiders abuse the system. This is no longer ﬁt for pur-

pose in a world with many phone companies, quite a few of which are

unscrupulous. The regulators aren’t effective, and the only real back-

ward pressure comes from the growing number of prepay customers.

The phone companies also want to be able to charge for relatively high-

value product and service delivery, extending the current premium ser-

vices through location-based services (‘give me a map showing me how

to drive to the nearest McDonalds’) to music and video downloads and

extra services such as the Finnish ferry tickets, cans of coke from vending

machines, and (most recently) parking meters in London. The account-

ing system will have to become a lot more robust, and dispute resolu-

tion mechanisms fairer and more transparent, if this potential is to be

realised.

All this interacts with platform security. If malware becomes widespread

on mobile phones, then the botnet herders who control subverted phones

will be able to pay for all sorts of goods and services by getting infected

machines to send text messages. Recent history suggests that any exploits

that can be industrialised to make money on a large scale, will be.

So how can phone payment systems be improved?

20.4 Security Economics of Telecomms 629

One proposed way of implementing this is to incorporate a micropayment

mechanism [92]. The idea is that the phone will send regular tick payments

to each of the networks or service providers which are due to be paid

for the call. The tick payments can be thought of as electronic coins and

are cryptographically protected against forgery. At the start of the call, the

handset will compute a number of phone ticks by repeated hashing: t1=h(t0),

t2=h(t1), and so on, with tk(for some credit limit k, typically 210 units) being

signed by the phone company. The phone will then release ticks regularly in

order to pay for the services as the call progresses. It starts by releasing tk,then

tk−1,thentk−2and so on. If a charge is subsequently disputed — whether by

a subscriber or a network operator — the party claiming an amount of (say) j

ticks must exhibit the ticks tk−jand tk, the latter with a certiﬁcate. As the hash

function his one-way, this should be hard to do unless the handset actually

released that many ticks. The tick tk−jcan how be checked by applying the hash

function to it jtimes and verifying that the result is tk4. One advantage of tick

payments is that as well as protecting the phone companies from conference

call frauds, it could protect the subscriber from many more. It could enable

phone application designers to empower users in new ways: for example, you

might by default decide to accept calls, or play games, or do other functions,

only so long as they cost less than a dollar, and require user intervention

for anything more expensive. At present, that kind of functionality just isn’t

available, except via the rather clunky mechanism of only loading so much

airtime into your phone.

The industry’s proposed solution is to redesign the call data record to

contain a lot more information. In addition to time, duration and called

number, it will have ﬁelds for data quantity, location and quality-of-service.

This is not just to support possible future differential charging for quality of

service, but also to help with emergency call tracking, and to comply with

a 2002 European directive on telecomms requires all mobile operators retain

location information on mobile phones for at least a year, for law enforcement

purposes. There was a proposal for an online cost-control mechanism to limit

the charges incurred for each user [901], but this appears to have stalled.

The cost-control mechanisms are not being standardized but can involve

forwarding charging data from either the local network or the gateway to

the home environment which will be able to have the call terminated if the

available credit is exhausted (as with a prepaid SIM card) or if the use appears

to be fraudulent. It’s tempting to draw a parallel with NATO’s failure to make

IFF work properly across different countries’ armed forces, which I discussed

in Chapter 19. The public good (in this case, transparent and dependable cost

4This protocol is an example of multiple simultaneous discovery, having been invented by our

group at Cambridge, by Pedersen, and by Rivest and Shamir, independently in 1995 [40, 1013,

1077].

630 Chapter 20 ■Telecom System Security

control) isn’t high on the agendas of the stakeholders who’d have to work

together to make it happen. Indeed, it’s even worse. It would be hard to ﬁnd

a general anywhere in North America or Europe who didn’t agree that decent

IFF would be a Good Thing; but many of the major stakeholders depend for

their existence on confusion pricing and would see a decent charging system

as a serious threat.

The extreme difﬁculty of engineering a global solution has left the market

open to a variety of local ones. In the USA, there is a move to integrate RFID-

based credit cards with NFC-compatible mobile phones. In the UK, a scheme

called PayForIt was launched in 2007 by the main mobile operators that aims

to replace premium SMS services with a WAP-based protocol that ‘provides a

uniform payment experience’ and requires ‘clear pricing, access to terms and

conditions and merchant contact details before a purchase is conﬁrmed’. O2,

one of the big networks, required all its users to switch from June 2007. They

said there would be ‘reduced customer care issues’.

Personally, I am sceptical, because of the lack of bankable guarantees for

the customer; the terms and conditions state that ‘any queries or complaints

regarding Goods and Services must be referred to the Supplier,’ while ﬁrms

advertising PayForIt transaction acquisition quote ﬁxed prices and seem ready

to accept all comers, respectable or otherwise. Perhaps the scheme will simply

hold up the growth of phone-based payments by making them more ﬁddly and

restricting them to more capable handsets, while not resolving the underlying

trust problem. A law that enforced customer rights would be better for all. In

its absence, the phone companies appear to be setting up a banking system

but without the regulations and controls that long and bitter experience has

shown necessary for bank stability and trustworthiness. In civilised countries,

maﬁosi are not allowed to run banks. But gangsters have a basic human right

(in America at least) to own a phone company. So phone companies should

not run banks.

20.5 Summary

Phone fraud is a fascinating case study. People have been cheating phone com-

panies for decades, and recently the phone companies have been vigorously

returning the compliment. To start off with, systems were not really protected

at all, and it was easy to evade charges and redirect calls. The mechanism

adopted to prevent this — out-of-band signalling — proved inadequate as the

rapidly growing complexity of the system opened up many more vulnerabili-

ties. These range from social engineering attacks on users through poor design

and management of terminal equipment such as PBXes to the exploitation of

various hard-to-predict feature interactions.

Research Problems 631

On the mobile front, the attempts to secure GSM and its third generation

successor make an interesting case study. Their engineers concentrated on

communications security threats rather than computer security threats, and

they concentrated on the phone companies’ interests at the expense of the

customers’. Their efforts were not entirely in vain but did not give a deﬁnitive

solution.

Overall, the security problems in telecomms are the result of environmental

changes, including deregulation, which brought in many new phone compa-

nies. But the main change was the introduction of premium rate numbers.

While previously phone companies sold a service with a negligible marginal

cost of provision, suddenly real money was involved; and while previously

about the only serious beneﬁt to be had from manipulating the system was

free calls, or calls that were hard for the police to tap, suddenly serious money

could be earned. The existing protection mechanisms were unable to cope

with this evolution. However, the major phone companies are so threatened

by price competition that their business models are now predicated on confu-

sion pricing. So the incentives for an overhaul of the billing mechanisms are

just not there.

Ultimately, I suspect, the regulator will have to step in. The best solution

in the USA could be the extension of Regulation E, which governs electronic

banking, to phone companies — as they have become de facto banks. When

you can use your mobile phone to buy ferry tickets and songs, and feed

parking meters, it’s performing all the functions of an electronic purse, which

if issued by a traditional payment service provider such as VISA or Mastercard

would fall squarely under Reg E. Also, I believe that either the FCC or the

Federal Reserve should have the right to ban known criminals from owning

or managing regulated phone companies. Europe is even further behind,

but there is some action in the regulatory pipeline: a draft Directive that

will impose a Europe-wide duty on companies to disclose security breaches

in telecomms to affected customers, similar to the breach notiﬁcation laws in

force in over 30 U.S states in 2007. This at least will be a start.

Research Problems

Relatively little research is done outside phone company and intelligence

agency labs on issues related speciﬁcally to phone fraud and wiretapping.

There is growing interest in trafﬁc analysis, which I’ll discuss later; and in

the likely effects of next-generation value added services, which are bound to

introduce new feature interactions and other vulnerabilities. The interaction

between all communications (especially mobile), platform security, and the

mechanisms used to protect distributed systems security, also looks like fertile

ground for both interesting research and expensive engineering errors. Society

632 Chapter 20 ■Telecom System Security

expects greater resilience and availability from the phone system than from

the Internet — for example, to get through to emergency services — and as the

two systems converge there will be some interesting assurance problems.

Further Reading

There are a lot of scattered articles about phone fraud, but nothing I know of

that brings everything together. The underlying technologies are described in

a number of reference books, such as [1061] on GSM, and more can be found

on websites such as [1190]. An overview of UMTS can be found in [642], and

a number of relevant research papers at [92]. To keep up with phone fraud, a

useful resource is the Discount Long Distance Digest [390]. NIST has a guide

on how to evaluate your PBX for vulnerabilities [943]. Finally, there’s a survey

of threats to mobile payment systems by the Mobile Payment Forum that gives

a summary of the state of play as of 2002 [897].

CHAPTER

Network Attack and Defense

Whoever thinks his problem can be solved using cryptography,

doesn’t understand his problem and doesn’t understand cryptography.

— Attributed by Roger Needham and Butler Lampson to Each Other

If you spend more on coffee than on IT security, then you will be hacked.

What’s more, you deserve to be hacked.

— Richard Clarke, Former U.S. Cybersecurity Tsar

21.1 Introduction

So far we’ve seen a large number of attacks against individual computers and

other devices. But attacks increasingly depend on connectivity. Consider the

following examples.

1. An ofﬁce worker clicks on an attachment in email. This infects her PC

with malware that compromises other machines in her ofﬁce by snoop-

ing passwords that travel across the LAN.

2. The reason she clicked on the attachment is that the email came from her

mother. The malware had infected her mother’s machine and then sent

out a copy of a recent email, with itself attached, to everyone in mum’s

address book.

3. Her mother in turn got infected by an old friend who chose a common

password for his ISP account. When there are many machines on a net-

work, the bad guys don’t have to be choosy; rather than trying to guess

the password for a particular account, they just try one password over

and over for millions of accounts. Given a webmail account, they can

send out bad email to the whole contact list.

633

634 Chapter 21 ■Network Attack and Defense

4. Another attack technique that makes sense only in a network context is

Google hacking. Here, the bad guys use search engines to ﬁnd web servers

that are running vulnerable applications.

5. The malware writers infect a whole lot of PCs more or less at random

using a set of tricks like these. They then look for choice pickings,

such as machines in companies from which large numbers of credit

card numbers can be stolen, or web servers that can be used to host

phishing web pages as well. These may be auctioned off to specialists

to exploit. Finally they sell on the residual infected machines for under a

dollar a time to a botnet herder — who operates a large network of com-

promised machines that he rents out to spammers, phishermen and

extortionists.

6. One of the applications is fast-ﬂux. This changes the IP address of a

web site perhaps once every 20 minutes, so that it’s much more

difﬁcult to take down. A different machine in the botnet acts as the host

(or as a proxy to the real host) with each change of IP address, so

blocking such an address has at most a temporary effect. Fast-ﬂux

hosting is used by the better phishing gangs for their bogus bank

websites.

There are many attacks, and defenses, that emerge once we have large

numbers of machines networked together. These depend on a number of

factors, the most important of which are the protocols the network uses. A

second set of factors relate to the topology of the network: is every machine

able to contact every other machine, or does it only have direct access to a

handful of others? In our example above, a virus spreads itself via a social

network — from one friend to another, just like the ﬂu virus.

I’ve touched on network aspects of attack and defense before, notably in the

chapters on telecomms and electronic warfare. However in this chapter I’m

going to try to draw together the network aspects of security in a coherent

framework. First I’m going to discuss networking protocols, then malware;

then defensive technologies, from ﬁltering and intrusion detection to the

widely-used crypto protocols TLS, SSH, IPsec and wireless LAN encryption.

Finally I’ll discuss network topology. The most immediate application of this

bundle of technologies is the defence of networks of PCs against malware;

however as other devices go online the lessons will apply there too. In addition,

many network security techniques can be used for multiple purposes. If you

invent a better ﬁrewall, then — like it or not — you’ve also invented a better

machine for online censorship and a better police wiretap device as well.

Conversely, if mobility and virtual private networks make life tough for the

ﬁrewall designer, they can give the censor and the police wiretap department

a hard time, too.

21.2 Vulnerabilities in Network Protocols 635

21.2 Vulnerabilities in Network Protocols

This book isn’t an appropriate place to explain basic network protocols. The

telegraphic summary is as follows. The Internet Protocol (IP) is a stateless

protocol that transfers packet data from one machine to another; IP version 4

uses 32-bit IP addresses, often written as four decimal numbers in the range

0–255, such as 172.16.8.93. People have started to migrate to IP version 6, as the

4 billion possible IPv4 addresses will have been allocated sometime between

2010 and 2015; IPv6 uses 128-bit addresses. Most modern kit is ready to use

IPv6 but the changeover, which companies will probably do one LAN at a

time, will no doubt throw up some interesting problems. The Domain Name

System (DNS) allows mnemonic names such as www.ross-anderson.com to be

mapped to IP addresses of either kind; there’s a hierarchy of DNS servers that

do this, ranging from thirteen top-level servers down through machines at

ISPs and on local networks, which cache DNS records for performance and

reliability.

The core routing protocol of the Internet is the Border Gateway Protocol

(BGP). The Internet consists of a large number of Autonomous Systems (ASs)

such as ISPs, telcos and large companies, each of which controls a range of IP

addresses. The routers — the specialized computers that ship packets on the

Internet — use BGP to exchange information about what routes are available

to get to particular blocks of IP addresses, and to maintain routing tables so

they can select efﬁcient routes.

Most Internet services use a protocol called transmission control protocol

(TCP) that is layered on top of IP and provides virtual circuits. It does this by

splitting up the data stream into IP packets and reassembling it at the far end,

automatically retransmitting any packets whose receipt is not acknowledged.

IP addresses are translated into the familiar Internet host addresses using the

domain name system (DNS), a worldwide distributed service in which higher-

level name servers point to local name servers for particular domains. Local

networks mostly use ethernet, in which devices have unique ethernet addresses

(also called MAC addresses) that are mapped to IP addresses using the address

resolution protocol (ARP). Because of the growing shortage of IP addresses,

most organisations and ISPs now use the Dynamic Host Conﬁguration Protocol

(DHCP) to allocate IP addresses to machines as needed and to ensure that

each IP address is unique. So if you want to track down a machine that has

done something wicked, you will often have to get the logs that map MAC

addresses to IP addresses.

There are many other components in the protocol suite for managing

communications and providing higher-level services. Most of them were

developed in the good old days when the net had only trusted hosts and

security wasn’t a concern. So there is little authentication built in. This is

636 Chapter 21 ■Network Attack and Defense

a particular problem with DNS and with BGP. For example, if a small ISP

mistakenly advertises to a large neighbour that it has good routes to a large

part of the Internet, it may be swamped by the trafﬁc. Here at least there are

sound economic incentives, in that ISPs either swap, or pay for, routes with

their peers; BGP security is in effect bodged up using manual intervention.

DNS is trickier, in that disruptions are often malicious rather than mistaken.

A DNS server may be fed wrong information to drive clients to a wicked

website. This can be done either wholesale, by an attack on the DNS servers

of a large ISP, or at the local level. For example, many homes have a wireless

router attached to a broadband connection, and the router contains the address

of the DNS server the customer uses. In an attack called Drive-By Pharming,the

villain lures you to view a web page containing javascript code that sets your

router’s DNS server to one under his control [1213]. The effect is that next time

you try to go to www.citibank.com, you may be directed to a phishing site

that emulates it. For this reason it’s a really good idea to change the default

password on your home router.

21.2.1 Attacks on Local Networks

Suppose the attacker controls one of your PCs. Perhaps one of your employees

was careless; or maybe he’s gone bad, and wants to take over an account in

someone else’s name to defraud you, or to do some other bad thing such as

downloading child porn in the hope of framing someone. There are several

possibilities open to him.

1. He can install packet sniffer software to harvest passwords, get the root

password, and thus take over a suitable account. Password-snifﬁng

attacks can be blocked if you use challenge-response password gener-

ators, or a protocol such as Kerberos or ssh to ensure that clear text pass-

words don’t go over the LAN. I described Kerberos in Chapter 3, and I’ll

describe SSH later.

2. Another approach is to masquerade as a machine where the target

user — say the sysadmin — has already logged on. It is often possi-

ble for the attacker simply to set his MAC address and IP address to

those of the target. In theory, the target machine should send ‘reset’

packets when it sees trafﬁc to its IP address that’s not in response to its

own packets; but many machines nowadays have personal ﬁrewalls,

which throw away ‘suspicious’ packets. As a result, the alarm doesn’t

get raised [300].

3. There’s a whole host of technical address-hijacking attacks that work

ﬁne against old-fashioned LANs. An example I gave in the ﬁrst edition

of my book was that the attacker gives wrong answers to ARP messages,

claiming to be the target, and may stop the target machine noticing and

21.2 Vulnerabilities in Network Protocols 637

raising the alarm by sending it a false subnet mask. Another possibility

is to send bogus DHCP messages. Attacks like this may or may not work

against a modern switched ethernet, depending on how it’s conﬁgured.

4. A further set of attacks target particular platforms. For example, if the

target company uses Linux or Unix servers, they are likely to use Sun’s

Network File System (NFS) for ﬁle sharing. This allows workstations to

use a network disk drive as if it were a local disk, and has a number

of well-known vulnerabilities to attackers on the same LAN. When a

volume is ﬁrst mounted, the client gets a root ﬁlehandle from the server.

This is in effect an access ticket that refers to the root directory of the

mounted ﬁlesystem, but that doesn’t depend on the time, or the server

generation number, and can’t be revoked. There is no mechanism for

per-user authentication: the server must trust a client completely or not

at all. Also, NFS servers often reply to requests from a different network

interface to the one on which the request arrived. So it’s possible to wait

until an administrator is using a ﬁle server and then masquerade as him

to overwrite the password ﬁle. Filehandles can also be intercepted by

network snifﬁng, though again, switched ethernet makes this harder.

Kerberos can be used to authenticate clients and servers, but many ﬁrms

don’t use it; getting it to work in a heterogeneous environment can be

difﬁcult.

So the ease with which a bad machine on your network can take over other

machines depends on how tightly you have the network locked down, and

the damage that a bad machine can do will depend on the size of the local

network. There are limits to how far a sysadmin can go; your ﬁrm might

need to run a complex mixture of legacy systems for which Kerberos just can’t

be got to work. Also, a security-conscious system administrator can impose

real costs. At our lab we argued with our sysadmins for years, trying to get

access to the Internet for visiting guests, while they resisted on both technical

protection and policy grounds (our academic network shouldn’t be made

available to commercial users). In the end, we solved the problem by setting

up a separate guest network that is connected to a commercial ISP rather than

to the University’s backbone.

This raises a wider problem: where’s the network boundary? In the old days,

many companies had a single internal network, connected to the Internet via a

ﬁrewall of some kind. But compartmentation often makes sense, as I discussed

in Chapter 9: separate networks for each department can limit the damage that

a compromised machine can do. There may be particularly strong arguments

for this if some of your departments may have high protection requirements,

while others need great ﬂexibility. In our university, for example, we don’t

want the students on the same LAN that the payroll folks use; in fact we

separate student, staff and administrative networks, and the ﬁrst two of these

638 Chapter 21 ■Network Attack and Defense

are also separated by college and department. Recently, mobility and virtual

networks have made deﬁnition of clear network boundaries even harder.

This debate goes by the buzz-word of deperimiterisation and I’ll return to

it later.

One ﬁnal attack is worth mentioning under the heading of attacks on local

networks, and that’s the rogue access point. Occasionally one ﬁnds WiFi access

points in public areas, such as airports, that have been deployed maliciously.

The operator might sit in the airport lounge with a laptop that accesses the

Internet via a paid WiFi service and advertises a free one; if you use it,

he’ll be able to sniff any plaintext passwords you enter, for example to your

webmail or Amazon account, and if you tried to do online banking he might

conceivably send you to a malicious site. So the effects can be somewhat

like drive-by pharming, although more reliable and less scalable. In addition,

rogue access points may also be devices that employees have installed for

their own convenience in deﬁance of corporate policy, or even ofﬁcial nodes

that have been misconﬁgured so that they don’t encrypt the trafﬁc. Whether

unencrypted WiFi trafﬁc is a big deal will depend on the circumstances; I’ll

discuss this in more detail later when we come to encryption.

21.2.2 Attacks Using Internet Protocols

and Mechanisms

Moving up now to the Internet protocol suite, the basic problem is similar:

there is no real authenticity protection in the default mechanisms. This is

particularly manifest at the lower level TCP/IP protocols, and has given rise

to many attacks.

Consider for example the 3-way handshake used by Alice to initiate a TCP

connection to Bob and set up sequence numbers.

This protocol can be exploited in a surprising number of different ways.

Now that service denial is becoming really important, let’s start off with the

simplest service denial attack: the SYN ﬂood.

21.2.2.1 SYN Flooding

The attack is quite simply to send a large number of SYN packets and never

acknowledge any of the replies. This leads the recipient (Bob, in Figure 21.1)

to accumulate more records of SYN packets than his software can handle. This

attack had been known to be theoretically possible since the 1980s but came to

public attention when it was used to bring down Panix, a New York ISP, for

several days in 1996.

A technical ﬁx has been incorporated in Linux and some other systems.

This is the so-called ‘SYNcookie’. Rather than keeping a copy of the incoming

SYN packet, Bsimply sends out as Yan encrypted version of X. That way, it’s

21.2 Vulnerabilities in Network Protocols 639

A→B: SYN; my number is X

B→A: ACK; now X+1

SYN; my number is Y

A→B: ACK; now Y+1

(start talking)

Figure 21.1: TCP/IP handshake

not necessary to retain state about sessions which are half-open. Despite this,

SYN ﬂoods are still a big deal, accounting for the largest number of reported

attacks (27%) in 2006, although they were only the third-largest in terms of

trafﬁc volume (18%, behind UDP ﬂoods and application-layer attacks) [86].

There is an important general principle here: when you’re designing a

protocol that anyone can invoke, don’t make it easy for malicious users to

make honest ones consume resources. Don’t let anyone in the world force your

software to allocate memory, or do a lot of computation. In the online world,

that’s just asking for trouble.

21.2.2.2 Smurﬁng

A common way of bringing down a host in the 90s was smurﬁng. This

exploited the Internet control message protocol (ICMP), which enables users to

send an echo packet to a remote host to check whether it’s alive. The problem

was with broadcast addresses that are shared by a number of hosts. Some

implementations of the Internet protocols responded to pings to both the

broadcast address as well as the local address — so you could test a LAN to

see what was alive. A collection of such hosts at a broadcast address is called

asmurf ampliﬁer. Bad guys would construct a packet with the source address

forged to be that of the victim, and send it to a number of smurf ampliﬁers.

These would then send a ﬂurry of packets to the target, which could swamp

it. Smurﬁng was typically used by kids to take over an Internet relay chat (IRC)

server, so they could assume control of the chatroom. For a while this was a

big deal, and the protocol standards were changed in August 1999 so that ping

packets sent to a broadcast address are no longer answered [1144]. Another

part of the ﬁx was socio-economic: vigilante sites produced lists of smurf

ampliﬁers. Diligent administrators spotted their networks on there and ﬁxed

them; the lazy ones then found that the bad guys used more and more of their

bandwidth, and thus got pressured into ﬁxing the problem too. By now (2007),

smurﬁng is more or less ﬁxed; it’s no longer an attack that many people use.

But there’s a useful moral: don’t create ampliﬁers. When you design a network

protocol, be extremely careful to ensure that no-one who puts one packet in

can get two packets out. It’s also important to avoid feedback and loops. A

classic example was source routing. A feature of early IP that enabled the sender

640 Chapter 21 ■Network Attack and Defense

of a packet to specify not just its destination but the route that it should take.

This made attacks too easy: you’d just send a packet from A to B to C to B to

C and so on, before going to its ﬁnal destination. Most ISPs now throw away

all packets with source routing set. (There was an alarm in early 2007 when it

turned out that source routing had found its way back into the speciﬁcation

for IPv6, but that’s now been ﬁxed [417].)

21.2.2.3 Distributed Denial of Service Attacks

As the clever ways of creating service-denial attacks have been closed off one

by one, the bad guys have turned increasingly to brute force, for example by

sending ﬂoods of UDP packets from infected machines. The distributed denial

of service (DDoS) attack made its appearance in October 1999 with the attack

already mentioned on a New York ISP, Panix. In DDoS, the attacker subverts

a large number of machines over a period of time and, on a given signal,

these machines all start to bombard the target with trafﬁc [391]. Curiously,

most of the machines in the ﬁrst botnets around 1999–2000 were U.S. medical

sites. The FDA insisted that medical Unix machines which were certiﬁed for

certain clinical uses have a known conﬁguration. Once bugs were found in

this, there was a guaranteed supply of vulnerable machines; an object lesson

in the dangers of monoculture.

Nowadays, botnets are assembled using all sorts of vulnerabilities, and a

market has arisen whereby people who specialise in hacking machines can sell

their product to people who specialise in herding them and extracting value.

Compromised machines typically pass down a kind of value chain; they are

ﬁrst used for targeted attacks, then for sending spam, then (once they get

known to spam ﬁlters) for applications like fast ﬂux, and then ﬁnally (once

they’re on all the blacklists) for DDoS.

DDoS attacks have been launched at a number of high-proﬁle web sites,

including Amazon and Yahoo, but nowadays the major sites have so much

bandwidth that they’re very hard to dent. The next development was extortion-

ists taking out online horserace-betting sites just before popular race meetings

that would have generated a lot of business, and demanding ransoms not to do

it again. Some bookmakers moved their operations to high-bandwidth hosting

services such as Akamai that are highly distributed and can cope with large

packet volumes, and others to specialist ISPs with packet-washing equipment

that ﬁlters out bad packets at high speed. However the real ﬁx for extortion

wasn’t technical. First, the bookmakers got together, compared notes, and

resolved that in future none of them would pay any ransom. Second, the

Russian government was leant on to deal with the main gang; three men were

arrested in 2004 and sent to prison for eight years in 2006 [791].

For a while, there was a technical arms race. Attackers started to spoof

source IP addresses, and to reﬂecting packets off innocuous hosts [1011]. One

21.2 Vulnerabilities in Network Protocols 641

countermeasure was traceback: the idea was that whenever a router forwards

a packet, it would also send an ICMP packet to the destination, with a prob-

ability of about 1 in 20,000, containing details of the previous hop, the next

hop, and as much of the packet as will ﬁt. Large-scale ﬂooding attacks could

thus be traced back to the responsible machines, even despite forged source IP

addresses [154]. However, this arms race has largely ﬁzzled out, and for a num-

ber of reasons. First, Microsoft changed their network stack to make it much

harder for an infected machine to send a packet with a spoofed IP address; you

now need to hack the operating system, not just any old application. Second,

more and more equipment at the edges of the network won’t accept spoofed

packets, and now about half of broadband ISPs ﬁlter them out [86].

However, the main reasons for the arms race stopping had to do with

economics. Tracing back packets didn’t work across different autonomous

systems; if you were a large telco, for example, you would not give network

access to your competitors’ technical staff. So the bad guys found that in

practice nobody came after them, and stopped using spooﬁng. Also, once

markets emerged round about 2004 for botnet machines to be bought and

sold (along with credit card numbers, spam contracts and other criminal

services), the price of compromised machines fell so low, and botnets started

to become so large, that the whole game changed. Instead of using a handful of

compromised machines to send out clever attacks via ampliﬁers using spoofed

source addresses, the bad guys simply burn thousands of end-of-life botnet

machines to send the bad packets directly. The rapier has been replaced with

the Kalashnikov.

Most recently, in 2005–7, there have been attempts to target core services

such as DNS and thus take down the whole Internet. DNS has now been

expanded to thirteen servers (the maximum the protocol will support), and

many of them use anycast — a protocol whereby when you ask to resolve the

domain name of a host into an IP address, the result that you get depends on

where you are. The net effect is that DNS has become a massively distributed

system using a lot of very fast machines connected to very high-capacity

networks. In the end, the brute force of the modern DDoS attack was simply

answered by even more brute force. If a hundred peasants with Kalashnikovs

are going to shoot at you, you’d better buy a tank with good enough armor to

absorb the ﬁre.

Large-scale DDoS attacks on critical services seem quiescent at present.

There are a few residual worries, though.

There’s a rising tide of DDoS attacks that happen by accident rather than

as a result of malice. For example, in 2003 the University of Wisconsin-

Madison found itself receiving hundreds of thousands of packets per

second requesting the time. It turned out that Netgear had sold some

700,000 routers that were hard-coded to ask their time server what time

642 Chapter 21 ■Network Attack and Defense

it was, and to ask again a second later if no response was received.

Netgear ended up paying the university to maintain a high-bandwidth

time server for them. There have been dozens of similar incidents [303].

There’s a steady stream of DDoS attacks by spammers and phishermen

on the websites of organisations that try to hinder their activities, such as

Artists Against 419 and Spamhaus.

There are continuing worries that DDoS attacks might come back on

an even larger scale. As of September 2007, there are several botnets

with over half a million machines [742]. The operators of such networks

can send out packet ﬂoods that will disable all but the biggest sites.

These worries have been ampliﬁed in some quarters by the 2007 attacks

on Estonia (even although that attack would not have harmed a large

commercial target like Microsoft or Google). The highest attack rate

seen in 2006 was 24 Gbit/sec, compared with 10 Gbit/sec in 2004 and

1.2 Gbit/sec in 2002 [86]. A further order-of-magnitude increase could

put all but the most distributed targets at risk.

Even if we never see the Internet taken down by a monster botnet,

attacks can still be carried out on smaller targets. Prior to the Estonia

incident, there had been a DDoS attack on the servers of an opposition

party in Kyrgyzstan, and these followed the site when it was relocated

to North America [1081]. Certainly, DDoS puts a weapon in the hands

of gangsters that can be rented out to various unsavoury people.

That said, one mustn’t forget online activism. If a hundred thousand

people send email to the White House protesting against some policy

or other, is this a DDoS attack? Protesters should not be treated as felons;

but drawing legislative distinctions can be hard.

21.2.2.4 Spam

Spam is in some respects similar to a DDoS attack: ﬂoods of generally unwanted

trafﬁc sent out for the most part by botnets, and often with clear criminal intent.

The technical aspects are related, in that both email and the web protocols

(smtp and http) assume wrongly that the lower levels are secure. Just as DDoS

bots may forge IP addresses, spam bots may forge the sender’s email address.

Spam differs in a various ways, though, from packet-level DDoS. First, it’s

enough of an annoyance and a cost for there to be real pressure on ISPs to

send less of it. If you’re a medium-sized ISP you will typically peer with other

ISPs as much as you can, and buy routes from large telcos only where you

can’t trade for them; if other ISPs see you as a source of spam they may drop

your peering arrangements, which costs real money. As a result, some ISPs

are starting to do egress ﬁltering: they monitor spam coming out of their own

networks and then quarantine the infected PCs into a ‘walled garden’ from

which they have access to antivirus software but little else [300].

21.2 Vulnerabilities in Network Protocols 643

Second, unlike DDoS, spam does not seem to be tailing off. It does appear

that spammers are consolidating, in that most spam comes from several dozen

large gangs. This is apparent from the ‘lumpiness’ of spam statistics: if there

were hundreds of thousands of mom-and-pop spam operations, you’d expect

to see spam volumes pretty constant, but this is no longer what we see [305].

So rather than spending more money on spam ﬁlters, it might be cheaper

to get the police to arrest the gangs. Trends do change over time, though.

Between 2006 and 2007, we’ve seen a drop in backscatter — in messages sent

to the people whose email addresses were forged by spammers. Quite a lot

of this came from anti-virus products, and it was pointed that the vendors

were often breaking the antispam laws by sending messages saying ‘Product X

found worm Y in the message you sent’. If worm Y was known to use address

forgery, the only conceivable purpose of sending such a message to the party

who hadn’t sent the offending message was to advertise product X. At the

same time, there’s been a huge increase in mule recruitment spam.

21.2.2.5 DNS Security and Pharming

I’ve given two examples so far of attacks in which a user is duped by being

directed to a malicious DNS server, with the result that when he tries to go to

his bank website he ends up entering his password into a fake one instead. This

is generally referred to as pharming. I mentioned drive-by pharming, in which

people’s home routers are reconﬁgured by malicious javascript in web pages

they download, and rogue access points in which the attacker offers a WiFi ser-

vice to the victim and then has complete control over all his unprotected trafﬁc.

There are a number of other variants on this theme, including feeding false

DNS records to genuine servers. Older DNS servers would accept additional

records without checking; if they asked your server where X was, you could

volunteer an IP address for Y as well. This has been ﬁxed but there are still

older servers in use that are vulnerable. Such attacks are often referred to as

DNS cache poisoning as they basically affect users who trust the information

about the target that’s cached by the attacked machine. They’ve been used

not just for pharming but also for simple vandalism, such as replacing the

web site of a target company with something offensive. They can also be

used for censorship; China has used DNS spooﬁng against dissident websites

for years, and by 2007 was also using it to make Voice of America news

unavailable [1297].

A number of researchers have worked on a proposed upgrade to the security

of DNS, but they have turned out to be hard to deploy for economic reasons;

most of the things that secure DNS would do can be done by TLS without

the need for new infrastructure, and individual network operators don’t get

enough beneﬁt from DNS security until enough other operators have adopted

them ﬁrst [997].

644 Chapter 21 ■Network Attack and Defense

21.3 Trojans, Viruses, Worms and Rootkits

Computer security experts have long been aware of the threat from malicious

code. The ﬁrst such programs were Trojan Horses, named after the horse the

Greeks left supposedly as a gift for the Trojans but which contained soldiers

who opened the gates of Troy to the Greek army. The use of the term for

malicious code goes back many years (see the discussion in [774] p 7). There

are also viruses and worms, which are self-propagating malicious programs,

and to which I’ve referred in earlier chapters. There is debate about their

precise deﬁnitions; the common usage is that a Trojan is a program that

does something malicious (such as capturing passwords) when run by an

unsuspecting user, while a worm is something that replicates and a virus is a

worm which replicates by attaching itself to other programs.

Finally, the most rapidly-growing problem is the rootkit — a piece of soft-

ware that once installed on a machine surreptitiously places it under remote

control. Rootkits can be used for targeted attacks (law enforcement agencies

use them to turn suspects’ laptops into listening devices) or for ﬁnancial fraud

(they may come with keyloggers that capture passwords). One of the most

salient features of rootkits nowadays is stealth; they try to hide from the

operating system so that they can’t be located and removed using standard

tools. But sooner or later rootkits are identiﬁed and tools to remove them are

written. On the other side of the coin, most PCs infected in this way end up

in botnets, so another way of framing the problem is how botnets are set up,

maintained and used. The rootkit vendors now do after-sales-service, supply-

ing their customers the botnet herders with the tools to upgrade rootkits for

which removal tools are becoming available.

21.3.1 Early History of Malicious Code

Malicious code, or malware, seems likely to appear whenever a large enough

number of users share a computing platform. It goes back at least to the

early 1960’s, when machines were slow and their CPU cycles were carefully

rationed between different groups of users — with students often at the tail

of the queue. Students invented tricks such as writing computer games with

a Trojan inside to check if the program is running as root, and if so to create

an extra privileged account with a known password. By the 1970s, large

time-sharing systems at universities were the target of more and more pranks

involving Trojans. All sorts of tricks were developed. In 1978, John Shoch

and Jon Hupp of Xerox PARC wrote a program they called a worm,which

replicated itself across a network looking for idle processors so it could assign

them tasks. They discussed this in a paper in 1982 [1164].

In 1984, Ken Thompson wrote a classic paper ‘On Trusting Trust’, in

which he showed that even if the source code for a system were carefully

21.3 Trojans, Viruses, Worms and Rootkits 645

inspected and known to be free of vulnerabilities, a trapdoor could still be

inserted [1247]. Thompson’s trick was to build the trapdoor into the compiler.

If this recognized that it was compiling the login program, it would insert a

trapdoor such as a master password that would work on any account. (This

developed an idea ﬁrst ﬂoated by Paul Karger and Robert Schell during the

Multics evaluation in 1974 [693].) Of course, someone might try to stop this

by examining the source code for the compiler, and then compiling it again

from scratch. So the next step is to see to it that, if the compiler recognizes

that it’s compiling itself, it inserts the vulnerability even if it’s not present in

the source. So even if you can buy a system with veriﬁably secure software

for the operating system, applications and tools, the compiler binary can still

contain a Trojan. The moral is that vulnerabilities can be inserted at any point

in the tool chain, so you can’t trust a system you didn’t build completely

yourself.

1984 was also the year when computer viruses appeared in public following

the thesis work of Fred Cohen. He performed a series of experiments with

different operating systems in which he showed how code could propagate

itself from one machine to another, and (as I mentioned in Chapter 8) from

one compartment of a multilevel system to another. This caused alarm and

consternation, and within about three years we started to see the ﬁrst real

live viruses in the wild1. Almost all of them were PC viruses as DOS was the

predominant operating system. They spread from one user to another when

users shared programs on diskettes or via bulletin boards.

One early innovation was the ‘Christma’ virus, which spread round IBM

mainframes in December 1987. It was a program written in the mainframe

command language REXX that had a header saying ‘Don’t read me, EXEC me’

and code that, if executed, drew a Christmas tree on the screen — then sent

itself to everyone in the user’s contacts ﬁle. It was written as a prank, rather

than out of malice; and by using the network (IBM’s BITNET) to spread, it was

ahead of its time.

The next year came the Internet worm, which alerted the press and the

general public to the problem.

21.3.2 The Internet Worm

The ﬁrst famous case of a service denial-attack was the Internet worm of

November 1988 [421]. This was a program written by Robert Morris Jr that

exploited a number of vulnerabilities to spread from one machine to another.

Some of these were general (e.g. 432 common passwords were used in a

guessing attack, and opportunistic use was made of .rhosts ﬁles), and others

1That’s when I ﬁrst came across them, as a security guy working in a bank; we now learn that the

ﬁrst ever computer virus in the wild was written for the Apple II by a 9th-grader in 1981 [1101].

646 Chapter 21 ■Network Attack and Defense

were system speciﬁc (problems with sendmail,andthefingerd bug mentioned

in section 4.4.1). The worm took steps to camouﬂage itself; it was called sh and

it encrypted its data strings (albeit with a Caesar cipher).

Its author claimed that this code was not a deliberate attack on the

Internet — merely an experiment to see whether code could replicate from

one machine to another. It was successful. It also had a bug. It should have

recognised already infected machines, and not infected them again, but this

feature didn’t work. The result was a huge volume of communications trafﬁc

that completely clogged up the Internet.

Given that the Internet (or more accurately, its predecessor the Arpanet) had

been designed to provide a very high degree of resilience against attacks — up

to and including a strategic nuclear strike — it was remarkable that a program

written by a student could disable it completely.

What’s less often remarked on is that the mess was cleaned up and normal

service restored within a day or two; that it only affected Berkeley Unix and

its derivatives (which may say something about the dangers of the Microsoft

monoculture today); and that sites that kept their nerve and didn’t pull their

network connection recovered more quickly as they could ﬁnd out what was

happening and get the ﬁxes.

21.3.3 How Viruses and Worms Work

A virus or worm typically has two components — a replication mechanism

and a payload. A worm simply makes a copy of itself somewhere else when

it’s run, perhaps by breaking into another system (as the Internet worm did)

or mailing itself as an attachment to the addresses on the infected system’s

address list (as many recent worms have done). In the days of DOS viruses, the

commonest way for a virus to replicate was to append itself to an executable

ﬁle and patch itself in, so that the execution path jumps to the virus code and

then back to the original program.

Given a speciﬁc platform, there are usually additional tricks available to

the virus writer. For example, if the target system was a DOS PC with a ﬁle

called ACCOUNTS.EXE, one could introduce a ﬁle called ACCOUNTS.COM,

which DOS will execute in preference. DOS viruses could also attack the boot

sector or the partition table, and there are even printable viruses — viruses

all of whose opcodes are printable ASCII characters, so that they can even

propagate on paper. A number of DOS viruses are examined in detail in [817].

The second component of a virus is the payload. This will usually be

activated by a trigger, such as a date, and may then do one or more of a

number of bad things:

make selective or random changes to the machine’s protection state

(this is what we worried about with multilevel secure systems);

21.3 Trojans, Viruses, Worms and Rootkits 647

make changes to user data (some early viruses would trash your hard

disk while some recent ones encrypt your disk and ask you to pay a ran-

som for the decryption key);

lock the network (e.g., start replicating at maximum speed);

perform some nefarious task (e.g. use the CPU for DES keysearch);

get your modem to phone a premium-rate number in order to transfer

money from you to a telephone scamster;

install spyware or adware in your machine. This might just tell mar-

keters what you do online — but it might steal your bank passwords and

extract money from your account;

install a rootkit — software that hides in your machine having taken it

over. This is typically used to recruit your machine into a botnet, so that

it can be used later for spam, phishing and distributed denial of service

attacks at the botnet herder’s pleasure.

The history of malware, and of countermeasures, has some interesting twists

and turns.

21.3.4 The History of Malware

By the late 1980s and early 1990s, PC viruses had become such a problem

that they gave rise to a whole industry of anti-virus software writers and

consultants. Many people thought that this couldn’t last, and that the move

from DOS to ‘proper’ operating systems such as Windows would solve the

problem. Some of the anti-virus pioneers even sold their companies; one of

them tells his story in [1198].

However, the move to 32-bit operating systems gave only temporary respite.

Soon, the spread of interpreted languages provided fertile soil for mischief. Bad

Java applets ﬂourished in the late 1990s as people found ways of penetrating

Java implementations in browsers [859]. By the start of the 21st century, the

main vector was the macro languages in products such as Word, and the main

transmission mechanism had become the Internet [95, 209]; by 2000, macro

viruses accounted for almost all incidents of mobile malicious code. Indeed,

an insider says that the net ‘saved’ the antivirus industry [669]. A more cynical

view is that the industry was never really under threat, as people will always

want to share code and data, and in the absence of trustworthy computing

platforms one can expect malware to exploit whichever sharing mechanisms

they use. Another view is that Microsoft is responsible as they were reckless

in incorporating such powerful scripting capabilities in all sorts of products.

As they say, your mileage may vary.

In passing, it’s worth noting that malicious data can also be a problem. An

interesting example is related by David Mazi`

eres and Frans Kaashoek who

648 Chapter 21 ■Network Attack and Defense

operated an anonymous remailer at MIT. This device decrypted incoming

messages from anywhere on the net, uncompressed them and acted on them.

Someone sent them a series of 25 Mbyte messages consisting of a single line

of text repeated over and over; these compressed very well and so were only

small ciphertexts when input, but when uncompressed they quickly ﬁlled up

the spool ﬁle and crashed the system [849]. There are similar attacks on other

programs that do decompression such as MPEG decoders. However, the most

egregious cases involve not malicious data but malicious code.

Anyway, the next phase of malware evolution may have been the ‘Love

Bug’ virus in 2000. This was actually a self-propagating worm; it propagated

by sending itself to everyone in the victim’s address book, and the subject line

‘I love you’ was calculated to get people to open it. In theory, companies can

defend themselves against such things by ﬁltering out Microsoft executables;

in practice, life isn’t so simple. A large Canadian company with 85,000 staff

did just this, but many of their staff had personal accounts at web-based

email services, and so the Love Bug virus got into the company without

going through the mail ﬁlter at the ﬁrewall. The company had conﬁgured its

employees’ mail clients so that each of them had the entire corporate directory

in her personal address book. The result was meltdown as 85,000 mail clients

each tried to send an email to each of 85,000 addresses. The Love Bug was

followed by a number of similar worms, which persuaded people to click on

them by offering pictures of celebs such as Anna Kournikova, Britney Spears

and Paris Hilton. There were also ‘ﬂash worms’ that propagated by scanning

the whole Internet for machines that were vulnerable to some exploit or other,

and taking them over; worms of this type, such as Code Red and Slammer,

infected all vulnerable machines within hours or even minutes, and caused

some alarm about what sort of defences might possibly react in time [1220].

At about the same time, in the early 2000s, we saw a signiﬁcant rise in

the amount of spyware and adware. Spyware is technology that collects and

forwards information about computer use without the owner’s authorization,

or with at best a a popup box that asks users to agree to perform some obscure

function, so that even those who don’t just reﬂexively click it away will not

really know what they’re agreeing to. This doesn’t pass muster as ‘consent’

under European data-protection and unfair-contracts laws, but enforcement

is weak. Adware may bombard the user with advertising popups and can

be bundled with spyware. The vendors of this tiresome crud have even

sued antivirus companies who blacklisted their wares. This all complicates

everything.

A large change came about in 2004 or so. Until then, we saw a huge range of

different viruses and payloads. Most virus writers did so for fun, for bragging

rights, to impress their girlfriends — basically, they were amateurs. Since then,

the emergence of an organised criminal economy in information goods has

made the whole business much more professional. The goal of the malware

21.3 Trojans, Viruses, Worms and Rootkits 649

writers is to recruit machines that can be sold on for cash to botnet herders

and for other exploits.

Most viruses in the 1980s and 1990s were very ﬂaky; although tens of

thousands of different viruses were reported, very few actually spread in the

wild. It’s actually rather difﬁcult to write a virus that spreads properly; if it’s

not infectious enough it won’t spread, while if you make it too infectious then

it gets noticed quickly and dealt with. However, those viruses that did spread

often spread very widely and infected millions of machines.

A widespread self-replicating worm may bring a huge ego boost to a teenage

programmer, but for the Maﬁa it’s not optimal. Such a worm becomes headline

news and within a few hours the world’s anti-virus vendors are upgrading

their products to detect and remove it. Even the mass media get in on the act,

telling the public not to click on any link in such-and-such an email. Now that

the writers are focussed on money rather than bragging rights, they release

more attacks but limited ones. Furthermore, rather than using self-replicating

worms — which attract attention by clogging up the Internet — the modern

trend is towards manually-controlled exploit campaigns.

In September 2007 the largest botnet was perhaps the Storm network,

with around a million machines. Its herders are constantly recruiting more

machines to it, using one theme after another. For example, following the

start of the National Football League season on September 6th, they sent

out spam on September 9th saying simply ‘Football ... Need we say more?

Know all the games, what time, what channel and all the stats. Never be in

the dark again with this online game tracker’, following by a link to a URL

from which the gullible download a ﬁle called tracker.exe that installs a

rootkit in their machine. Using techniques like this — essentially, professional

online marketing — they constantly grow their network. And although the

media refer to Storm as a ‘worm’, it isn’t really: it’s a Trojan and a rootkit.

Victims have to click away several warnings before they install it; Windows

warns them that it isn’t signed and asks them if they really want to install it.

However, Windows pops up so many annoying dialog boxes that most people

are well trained to click them away. In the case of Storm, it was targeted by

Microsoft’s malicious software removal tool on September 11th, and Redmond

reported that over a quarter of a million machines had been cleaned; they also

estimated that Storm had half a million active machines, with perhaps a few

hundred thousand that were not being actively used. The network — the most

powerful supercomputer on the planet — earned its living by being rented

out to pump-and-dump operators and pharmacy scammers [742]. Two other

networks were also identiﬁed as having over half a million bots; Gozi and

Nugache use the same peer-to-peer architecture as Storm, and by the end of

2007 these networks were getting increasingly sophisticated and exploring

new criminal business models [1134].

650 Chapter 21 ■Network Attack and Defense

So the malware business now operates on an industrial scale, with the top

botnet herders controlling roughly the same number of machines as Google.

Big business has been built on the fact that users have been trained to click on

stuff. As malware goes industrial, Trojans are becoming more common than

viruses; when the latter email themselves out from an infected machine, they

draw attention to themselves and the machine’s more likely to get cleaned up,

while with Trojans the botnet herder sends the infectious trafﬁc directly, which

also gives him better control [1239]. And once you install something, there’s

no telling whether it’s a rootkit, or malicious spyware that will use a keystroke

logger to steal your banking passwords, or a ‘normal’ piece of spyware that

will simply collect your personal data for sale to the highest bidder. Truth to

tell, the categories are hard to separate cleanly.

21.3.5 Countermeasures

Within a few months of the ﬁrst PC viruses appearing in the wild in 1987,

companies had set up to sell antivirus software. This led to an arms race in

which each tried to outwit the other. Early software came in basically two

ﬂavours — scanners and checksummers.

Scanners are programs that search executable ﬁles for a string of bytes

known to be from an identiﬁed virus. Virus writers responded in various

ways, such as speciﬁc counterattacks on popular antivirus programs; the most

general technique is polymorphism. The idea here is to change the code each

time the virus or worm replicates, to make it harder to write effective scanners.

The usual technique is to encrypt the code using a simple cipher, and have a

small header that contains decryption code. With each replication, the virus

re-encrypts itself under a different key, and tweaks the decryption code by

substituting equivalent sequences of instructions.

Checksummers keep a list of all the authorised executables on the system,

together with checksums of the original versions, typically computed using a

hash function. The main countermeasure is stealth, which in this context means

that the virus watches out for operating system calls of the kind used by the

checksummer and hides itself whenever a check is being done.

Researchers have also looked into the theory of malware replication. In

order for a virus infestation to be self-sustaining, it needs to pass an epidemic

threshold — at which its rate of replication exceeds the rate at which it’s

removed [711]. This depends not just on the infectivity of the virus itself but

on the number (and proportion) of connected machines that are vulnerable.

Epidemic models from medicine go over to some extent, though they are

limited by the different topology of software intercourse (sharing of software

is highly localised) and so predict higher infection rates than are actually

observed. (I’ll return to topology later.) People have also tried to use immune-

system models to develop distributed strategies for malware detection [482].

21.3 Trojans, Viruses, Worms and Rootkits 651

One medical lesson which does seem to apply is that the most effective

organisational countermeasure is centralised reporting and response using

selective vaccination [712].

In the practical world, antivirus software and managerial discipline are to

a certain extent substitutes, but to be really effective, you have to combine

tools, incentives and management. In the old days of DOS-based ﬁle viruses,

this came down to providing a central reporting point for all incidents, and

controlling all software loaded on the organisation’s machines. The main risks

were ﬁles coming in via PCs used at home both for work and for other things

(such as kids playing games), and ﬁles coming in from other organisations. But

how do you get staff to sweep all incoming email and diskettes for viruses?

One effective strategy, adopted at a London law ﬁrm, was to reward whoever

found a virus with a box of chocolates — which would then be invoiced to the

company that had sent the infected ﬁle.

Now that malware arrives mostly in email attachments or in web pages,

things are often more technical, with automatic screening and central reporting.

A company may ﬁlter executables out at the ﬁrewall, and see to it that users

have prudent default settings on their systems — such as disabling active

content on browsers and macros in word processing documents. Of course, this

creates a clash with usability. People will also create all sorts of unauthorized

communications channels, so you have to assume that screening can’t be

perfect; staff must still be trained not to open suspicious email attachments,

and in recovery procedures so they can deal with infected backups. In short,

the issues are more complex and diffuse. But as with the organic kind of

disease, prevention is better than cure; and software hygiene can be integrated

with controls on illegal software copying and unauthorised private use of

equipment.

Recently, antivirus software seems to be getting steadily less effective.

The commercialisation of botnets and of machine exploitation has meant

that malware writers have decent tools and training. Almost all Trojans and

other exploits are undetectable by the current antivirus products when ﬁrst

launched — as their writers test them properly — and many of them run their

course (by recruiting their target number of machines) without coming to

the attention of the antivirus industry. The net effect is that while antivirus

software might have detected almost all of the exploits in circulation in the

early 2000s, by 2007 the typical product might detect only a third of them.

And as for the rootkits that the exploits leave behind, they are also much

better written than a few years ago, and rarely cause trouble for the owner of

the machine on which they’re installed. Some rootkits even install up-to-date

antivirus software to stop any competing botnet from taking the machine over.

They also use all sorts of stealth techniques to hide from detectors. What’s

more, the specialists who sell the rootkits provide after-sales service; if a

removal kit is shipped, the rootkit vendor will rapidly ship countermeasures.

652 Chapter 21 ■Network Attack and Defense

It’s not at all clear that technical defences are keeping up with malware. On

the global scale, police action against the large gangs is needed, and although

it’s starting to ramp up, there’s a long way to go. Well-run ﬁrms can use

managerial discipline to contain the threat, but for private users of Windows

machines, the outlook isn’t particularly rosy. One survey suggested that 8% of

sales of new PCs are to people who’ve simply given up on machines that have

become so infested with adware and other crud as to become unusable [325];

and there is a growing threat from keyloggers that capture everything the user

does at his machine. Some of these are simply spyware that sells information

to marketers; others look out for bank passwords and other key data that can

be used to commit fraud directly.

21.4 Defense Against Network Attack

In defending against network attack, there are broadly speaking four sets of

available tools.

1. First is management — keeping your systems up-to-date and conﬁgured

in ways that will minimise the attack surface;

2. Next is ﬁltering — the use of ﬁrewalls to stop bad things like Trojans and

network exploits, and to detect signs of attack and compromise if any-

thing gets through;

3. Next is intrusion detection — having programs monitoring your net-

works and machines for signs of malicious behaviour;

4. Finally there’s encryption — protocols such as TLS and SSH that enable

you to protect speciﬁc parts of the network against particular attacks.

Let’s work through these in turn.

21.4.1 Conﬁguration Management

and Operational Security

The great majority of technical attacks on systems in the period 2000–07

exploited already known vulnerabilities. The typical cycle is that Microsoft

announces a set of security patches once a month; as soon as they come out, the

attackers start reverse engineering them; within a few days, the vulnerabilities

that they ﬁxed are understood and exploits appear. A well-run ﬁrm will

test its operational systems quickly on Patch Tuesday and apply the patches,

provided they don’t break anything important. If they do break something,

that will be ﬁxed as quickly as reasonably possible.

Tight conﬁguration management is not just about patches, though. Many

software products ship with unsafe defaults, such as well-known default

21.4 Defense Against Network Attack 653

passwords. It’s a good idea to have someone whose job it is to understand and

deal with such problems. It’s also common to remove unnecessary services

from machines; there is usually no reason for every workstation in your

company to be running a mail server, and ftp server and DNS, and stripping

things down can greatly reduce the attack surface. Frequent reinstallation is

another powerful tool: when this was ﬁrst tried at MIT during Project Athena,

a policy of overnight reinstallation of all software greatly cut the number of

sysadmins needed to look after student machines. Operations like call centres

often do the same; that way if anyone wants to install unauthorised software

they have to do it again every shift, and are more likely to get caught. There are

also network conﬁguration issues: you want to know your network’s topology,

and have some means of hunting down things like rogue access points. If all

this is done competently, then you can deal with most of the common technical

attacks. (You’ll need separate procedures to deal with bugs that arise in your

own code, but as most software is bought rather than written these days,

conﬁguration management is most of the battle.)

There are many tools to help the sysadmin in these tasks. Some enable you

to do centralized version control so that patches can be applied overnight and

everything kept in synch; others look for vulnerabilities in your network. When

the ﬁrst such tool came out (Satan [503]) there was quite a lot of controversy;

this has led to some countries passing laws against ‘hacking tools’. Now there

are dozens of such tools, but they have to be used with care.

However, a strategy of having your system administrators stop all vulnera-

bilities at source is harder than it looks; even diligent organisations may ﬁnd

it’s just too expensive to ﬁx all the security holes at once. Patches may break

critical applications, and it seems to be a general rule that an organisation’s

most critical systems run on the least secure machines, as administrators have

not dared to apply upgrades and patches for fear of losing service.

This leads us to operational security, and the use of ﬁltering tools such as

ﬁrewalls.

Operational security, as mentioned in Chapter 2 and Chapter 8, is about

training staff to not expose systems by foolish actions. There we were largely

interested in social engineering attacks involving the telephone; the main way

of getting unauthorised access to information is still to phone up and pretend

to be someone who’s entitled to know.

Now the main way machines get compromised in 2007 is because people

click on links in email that cause them to download and install rootkits. Of

course you must train your staff to not click on links in mail, but don’t expect

that this alone will ﬁx the problem; many banks and other businesses expect

their customers to click on links, and many of your staff will have to do some

clicking to get their work done. You can shield low-grade staff by not giving

them administrator access to their machines, and you can shield your creative

654 Chapter 21 ■Network Attack and Defense

staff by letting them buy Macs; but there is still going to be a residual risk. One

common way of dealing with it is to strip out all executables at your ﬁrewall.

21.4.2 Filtering: Firewalls, Spam Filters, Censorware

and Wiretaps

The most widely sold solution to the ‘problems of Internet security’ is the

ﬁrewall. This is a machine which stands between a local system and the Internet

and ﬁlters out trafﬁc that might be harmful. The idea of a ‘solution in a box’

has great appeal to many organisations, and is now so widely accepted that

it’s seen as an essential part of corporate due diligence. (This in itself creates a

risk — many ﬁrms prefer expensive ﬁrewalls to good ones.)

Firewalls are just one example of systems that examine streams of packets

and perform ﬁltering operations. Bad packets may be thrown away, or mod-

iﬁed in such a way as to make them harmless. They may also be copied to a

log or audit trail. Very similar systems are also used for Internet censorship

and for law-enforcement wiretapping; almost everything I’ll discuss in this

section goes across to those applications too. Developments in any of these

ﬁelds potentially affect the others; and actual systems may have overlap-

ping functions. For example, many corporate ﬁrewalls or mail ﬁlters screen

out pornography, and some even block bad language, while ISP systems

that censor child pornography or dissenting political speech may report the

perpetrators automatically to the authorities.

Filters come in basically three ﬂavours, depending on whether they operate

at the IP packet level, at the TCP session level or at the application level.

21.4.2.1 Packet Filtering

The simplest kind of ﬁlter merely inspects packet addresses and port numbers.

This functionality is also available in routers, in Linux and indeed in Windows.

A ﬁrewall can block IP spooﬁng by ensuring that only ‘local’ packets leave

a network, and only ‘foreign’ ones enter. It can also stop denial-of-service

attacks in which malformed packets are sent to a host. It’s also easy to block

trafﬁc to or from ‘known bad’ IP addresses. For example, IP ﬁltering is a major

component of the censorship mechanisms in the Great Firewall of China; a

list of bad IP addresses can be kept in router hardware, which enables packet

ﬁltering to be done at great speed.

Basic packet ﬁltering is also available as standard on most machines and can

be used for more mundane ﬁrewalling tasks. For example, packet ﬁlters can be

conﬁgured to block all trafﬁc except that arriving on speciﬁc port numbers. The

conﬁguration might be initially to allow the ports used by common services

such as email and web trafﬁc, and then open up ports as the protected machine

or subnet uses them.

21.4 Defense Against Network Attack 655

However, packet ﬁlters can be defeated by a number of tricks. For example,

a packet can be fragmented in such a way that the initial fragment (which

passes the ﬁrewall’s inspection) is overwritten by a subsequent fragment,

thereby replacing the source address with one that violates the ﬁrewall’s

security policy. Another limitation is that maintaining a blacklist is difﬁcult,

and especially so when it’s not the IP address speciﬁcally you want to block,

but something that resolves into an IP address, especially on a transient basis.

For example, the phishermen are starting to use tricks like fast-ﬂux in which a

site’s IP address changes several times an hour.

21.4.2.2 Circuit Gateways

The next step up is a more complex arrangement, a circuit gateway, that operates

at level 4, typically by reassembling and examining all the packets in each

TCP session. This is more expensive than simple packet ﬁltering; its main

advantage is that it can also provide the added functionality of a virtual private

network whereby corporate trafﬁc passed over the Internet is encrypted from

ﬁrewall to ﬁrewall. I’ll discuss the IPSEC protocol that’s used for this in the

last section of this chapter.

TCP-level ﬁltering can be used to do a few more things, such as DNS

ﬁltering. However, it can’t screen out bad things at the application level,

such as malicious code, image spam and unlawful images of child abuse.

Thus it may often be programmed to direct certain types of trafﬁc to speciﬁc

application ﬁlters. An example is British Telecom’s CleanFeed system, which

tries to prevent its customers getting access to child pornography. As some

bad sites are hosted on public web services and blocking all the web pages at

the service would be excessive, TCP/IP ﬁltering is used to redirect trafﬁc with

such sites to a proxy that can examine it in detail.

21.4.2.3 Application Relays

The third type of ﬁrewall is the application relay, which acts as a proxy for one

or more services. Examples are mail ﬁlters that try to weed out spam, and

web proxies that block or remove undesirable content. The classic example is

a corporate rule about stripping out code, be it straightforward executables,

active content in web pages, macros from incoming Word documents. Over

the period 2000–07, this has been a constant arms race between the ﬁrewall

vendors, the spammers, and people trying to circumvent controls to get their

work done.

The ﬂood of Word macro viruses around 2000 led many ﬁrms to strip out

Word macros (or even all Word documents) from email. Workers got round this

by zipping documents ﬁrst. Firewalls started unzipping them to inspect them,

whereupon people started encrypting them using zip’s password feature, and

656 Chapter 21 ■Network Attack and Defense

putting the password in the email plaintext. Once ﬁrewalls started to cope

with this, the spammers started putting zip passwords in images attached to

the mail along with the zip ﬁle. Eventually, many companies started adopting

a policy of not sending out Word documents, but Pdf documents instead; this

not only made it easier to get past ﬁrewalls, but also stopped people carelessly

sending out documents containing the last few dozen edits. Needless to say,

the spammers now send out Pdf attachments — and their botnets have the

power to make all the attachments different, for example by combining text,

and image, and a number of random color blocks for background. Rootkit

executables are now often distributed as web links; August 2007 saw ﬂoods

of messages telling people they’d got a card, while in September it was links

to a bogus NFL site. For complete protection, you have to ﬁlter executables in

your web proxy too (but this would really get in the way of users who wish to

run the latest applications). There is no sign of this arms race abating.

An application relay can also turn out to be a serious bottleneck. This applies

not just to the corporate application, but in censorship. An example is the Great

Firewall of China, which tries to block mail and web content that refers to

banned subjects. Although the ﬁrewall can block ‘known bad’ sites by simple

IP ﬁltering, ﬁnding forbidden words involves deep packet inspection — which

needs much more horsepower. An investigation by Richard Clayton, Steven

Murdoch and Robert Watson showed bad content wasn’t in fact blocked;

machines in China simply sent ‘reset’ packets to both ends of a connection

on which a bad word had appeared. This was almost certainly because they

needed a number of extra machines for the ﬁltering, rather than doing it in the

router; one side-effect was that you could defeat the ﬁrewall by ignoring these

reset packets [308]. (Of course, someone within China who did that might

eventually get a visit from the authorities.)

At the application level in particular, the pace of innovation leaves the

ﬁrewall vendors (and the censors and the wiretappers) trailing behind.

A good example is the move to edge-based computing. Google’s word

processor — Google Documents — is used by many people to edit docu-

ments online, simply to save them the cost of buying Microsoft Word. As a

side-effect, its users can instantly share documents with each other, creating

a new communications channel of which classical ﬁlters are unaware. So the

service might be used to smuggle conﬁdential documents out of a company,

to defeat political censors, or to communicate covertly. (It even blurs the dis-

tinction between trafﬁc and content, which is central to the legal regulation of

wiretapping in most countries.) Even more esoteric communications channels

are readily available — conspirators could join an online multi-user game, and

pass their messages via the silver dragon in the sixth dungeon.

Another problem is that application-level ﬁltering can be very expensive,

especially of high-bandwidth web content. That’s why a number of web

ﬁltering systems are hybrids, such as the CleanFeed mechanism mentioned

above where only those domains that contain at least some objectionable

21.4 Defense Against Network Attack 657

content are sent to full http proxy ﬁltering. Application proxies can also

interact with other protection mechanisms. Not only can spammers (and

others) use encryption to defeat content inspection; but some corporate web

proxies are set up to break encryption by doing middleperson attacks on TLS.

Even if you think you’re giving an encrypted credit card number to Amazon,

your encrypted session may just be with your employer’s web proxy, while

it runs another encrypted session with Amazon’s web server. I’ll discuss TLS

later in this chapter.

21.4.2.4 Ingress Versus Egress Filtering

At present, most ﬁrewalls look outwards and try to keep bad things out,

but a growing number look inwards and try to stop bad things leaving. The

pioneers were military mail systems that monitor outgoing trafﬁc to ensure

that nothing classiﬁed goes out in the clear; around 2005 some ISPs started

looking at outgoing mail trafﬁc to try to detect spam. The reason is that ISPs

which host lots of infected machines and thus pump out lots of spam damage

their peering relationships with other ISPs, which costs real money; so various

systems have been developed to help them spot infected machines, that can

then be restricted to a ‘walled garden’ from which they can access anti-virus

software but not much else [300].

If companies whose machines get used in service denial attacks start getting

sued, as has been proposed in [1285], then egress ﬁltering can at least in

principle be used to detect and stop such attacks. However, at present the

incentives just aren’t there, and so although people care about spam ﬂoods,

almost nobody at the ISP level bothers about packet ﬂoods. This might of

course change as attacks get worse or if the regulatory environment changes.

Another possible development is egress ﬁltering for privacy, given the

rising tide of spyware. Software that ‘phones home’, whether for copyright

enforcement or marketing purposes, can disclose highly sensitive material

such as local hard disk directories. Prudent organizations will increasingly

wish to monitor and control this kind of trafﬁc. In the long term we expect

that ‘pervasive computing’ will ﬁll our homes with all sorts of gadgets that

communicate, so I wouldn’t be surprised to see home ﬁrewalls that enable the

householder to control which of them ‘phone home’, and for what purpose.

21.4.2.5 Architecture

Many ﬁrms just buy a ﬁrewall because it’s on the tick-list of due-diligence

things their auditors want to see. In that case, the sensible choice is a simple

ﬁltering router, which won’t need much maintanence and won’t get in the

way. Where security’s taken seriously, one possible approach is to invest in a

really serious ﬁrewall system, which might consist of a packet ﬁlter connecting

the outside world to a screened subnet, also known as a demilitarized zone

658 Chapter 21 ■Network Attack and Defense

(DMZ), which in turn contains a number of application servers or proxies to

ﬁlter mail, web and other services. The DMZ may then be connected to the

internal network via a further ﬁlter that does network address translation.

In [323], there is a case study of how a ﬁrewall was deployed at Hanscom

Air Force Base. The work involved surveying the user community to ﬁnd

what network services were needed; devising a network security policy; using

network monitors to discover unexpected services that were in use; and lab

testing prior to installation. Once it was up and running, the problems included

ongoing maintenance (due to personnel turnover), the presence of (unmon-

itored) communications to other military bases, and the presence of modem

pools. Few non-military organizations are likely to take this much care.

An alternative approach is to have more networks, but smaller ones. At our

university, we have ﬁrewalls to separate departments, although we’ve got a

shared network backbone and there are some shared central services. There’s

no reason why the students and the ﬁnance department should be on the same

network, and a computer science department has got quite different require-

ments (and users) from a department of theology — so the network security

policies should be different too. In any case keeping each network small limits

the scope of any compromise.

You may even ﬁnd both a big corporate ﬁrewall and departmental bound-

aries. At defense contractors, you may expect to ﬁnd not just a fancy ﬁrewall

at the perimeter, but also pumps separating networks operating at different

clearance levels, with ﬁlters to ensure that classiﬁed information doesn’t escape

either outwards or downwards (Figure 21.2).

Mail

proxy

Filter

Web

server Mail

guard

Other

proxies

. . .

Filter

Internet

Intranet

Classified

intranet

Figure 21.2: Multiple ﬁrewalls

21.4 Defense Against Network Attack 659

Which is better? Well, it depends. Factors to consider when designing

a network security architecture are simplicity, usability, maintainability,

deperimeterisation, underblocking versus overblocking, and incentives.

First, since ﬁrewalls do only a small number of things, it’s possible to make

them very simple and remove many of the complex components from the

underlying operating system, removing a lot of sources of vulnerability and

error. If your organization has a large heterogeneous population of machines,

then loading as much of the security management task as possible on a small

number of simple boxes makes sense. On the other hand, if you’re running

something like a call centre, with a thousand identically-conﬁgured PCs, it

makes sense to put your effort into keeping this conﬁguration tight.

Second, elaborate central installations not only impose greater operational

costs, but can get in the way so much that people install back doors, such as

cable modems that bypass your ﬁrewall, to get their work done. In the 1990s,

UK diplomats got fed up waiting for a ‘secure’ email system from government

suppliers; they needed access to email so they could talk to people who

preferred to use it. Some of them simply bought PCs from local stores and

got accounts on AOL, thus exposing sensitive data to anyone who tapped the

network or indeed guessed an AOL password. In fact, the diplomats of over

a hundred countries had webmail accounts compromised when they were

foolish enough to rely on Tor for message conﬁdentiality, and got attacked

by a malicious Tor exit node (an incident I’ll discuss in section 23.4.2.) So a

prudent system administrator will ensure that he knows the actual network

conﬁguration rather than just the one stated by ‘policy’.

Third, ﬁrewalls (like other ﬁltering products) tend only to work for a while

until people ﬁnd ways round them. Early ﬁrewalls tended to let only mail

and web trafﬁc through; so writers of applications from computer games

to anonymity proxies redesigned their protocols to make the client-server

trafﬁc look as much like normal web trafﬁc as possible. Now, of course, in

the world of Web 2.0, more and more applications are actually web-based;

so we can expect the same games to be played out again in the web proxy.

There are particular issues with software products that insist on calling home.

For example, the ﬁrst time you use Windows Media Player, it tells you you

need a ‘security upgrade’. What’s actually happening is that it ‘individualizes’

itself by generating a public-private keypair and sending the public key to

Microsoft. If your ﬁrewall doesn’t allow this, then WMP won’t play protected

content. Microsoft suggests you use their ISA ﬁrewall product, which will

pass WMP trafﬁc automatically. Quite a few issues of trust, transparency and

competition may be raised by this!

Next, there’s deperimiterization — the latest buzzword. Progress is making it

steadily harder to put all the protection at the perimeter. The very technical

ability to maintain a perimeter is undermined by the proliferation of memory

sticks, of laptops, of PDAs being used for functions that used to be done on

660 Chapter 21 ■Network Attack and Defense

desktop computers, and by changing business methods that involve more

outsourcing of functions — whether formally to subcontractors or informally

to advertising-supported web apps. If some parts of your organisation can’t

be controlled well (e.g. the sales force and the R&D lab) while others must be

(the ﬁnance ofﬁce) then separate networks are needed. The crumbling of the

perimeter will be made even worse by mobility, and by the proliferation of web

applications. This is complemented by a blunting of the incentive to do things

at the perimeter, as useful things become harder to do. The difference between

code and data is steadily eroded by new scripting languages; a determination

to not allow javascript in the ﬁrm is quickly eroded by popular web sites that

require it; and so on.

And then there’s our old friend the Receiver Operating Characteristic or ROC

curve. No ﬁltering mechanism has complete precision, so there’s inevitably

a trade-off between underblocking and overblocking. If you’re running a

censorship system to stop kids accessing pornography in public libraries, do

you underblock, and annoy parents and churches when some pictures get

through, or do you overblock and get sued for infringing free-speech rights?

Things are made worse by the fact that the ﬁrewall systems used to ﬁlter web

content for sex, violence and bad language also tend to block free-speech sites

(as many of these criticise the ﬁrewall vendors — and some offer technical

advice on how to circumvent blocking.)

Finally, security depends at least as much on incentives as on technology.

A sysadmin who’s looking after a departmental network used by a hundred

people he knows, and who will personally have to clear up any mess caused by

an intrusion or a conﬁguration error, is much more motivated than someone

who’s merely one member of a large team looking after thousands of machines.

21.4.3 Intrusion Detection

It’s a good idea to assume that attacks will happen, and it’s often cheaper to

prevent some attacks and detect the rest than it is to try to prevent everything.

The systems used to detect bad things happening are referred to generically as

intrusion detection systems. The antivirus software products I discussed earlier

are one example; but the term is most usually applied to boxes that sit on

your network and look for signs of an attack in progress or a compromised

machine [1100]. Examples include:

spam coming from a machine in your network;

packets with forged source addresses — such as packets that claim to

be from outside a subnet coming from it, or packets that claim to be from

inside arriving at it;

a machine trying to contact a ‘known bad’ service such as an IRC channel

that’s being used to control a botnet.

21.4 Defense Against Network Attack 661

In cases like this, the IDS essentially tells the sysadmin that a particular

machine needs to be scrubbed and have its software reinstalled.

Other examples of intrusion detection, that we’ve seen in earlier chapters,

are the mechanisms for detecting mobile phone cloning and fraud by bank

tellers. There are also bank systems that look at customer complaints of credit

card fraud to try to ﬁgure out which merchants have been leaking card data,

and stock market systems that try to detect insider trading by looking for

increases in trading volume prior to a price-sensitive announcement and other

suspicious patterns of activity. And there are ‘suspect’ lists kept by airport

screeners; if your name is down there, you’ll be selected ‘at random’ for extra

screening. Although these intrusion detection systems are all performing very

similar tasks, their developers don’t talk to each other much. One sees the

same old wheels being re-invented again and again. But it’s starting slowly

to become a more coherent discipline, as the U.S. government has thrown

hundreds of millions at the problem.

The research program actually started in the mid-1990s and was prompted

by the realisation that many systems make no effective use of log and audit

data. In the case of Sun’s operating system Solaris, for example, we found in

1996 that the audit formats were not documented and tools to read them were

not available. The audit facility seemed to have been installed to satisfy the

formal checklist requirements of government systems buyers rather than to

perform any useful function. There was at least the hope that improving this

would help system administrators detect attacks, whether after the fact or even

when they were still in progress. Since 9/11, of course, there has been a great

switch of emphasis to doing data mining on large corpora of both government

and commercial data, looking for conspiracies.

21.4.3.1 Types of Intrusion Detection

The simplest intrusion detection method is to sound an alarm when a threshold

is passed. Three or more failed logons, a credit card expenditure of more than

twice the moving average of the last three months, or a mobile phone call

lasting more than six hours, might all ﬂag the account in question for attention.

More sophisticated systems generally fall into two categories.

Misuse detection systems operate using a model of the likely behaviour of

an intruder. A banking system may alarm if a user draws the maximum

permitted amount from a cash machine on three successive days; and a Unix

intrusion detection system may look for user account takeover by alarming

if a previously naive user suddenly started to use sophisticated tools like

compilers. Indeed, most misuse detection systems, like antivirus scanners,

look for a signature — a known characteristic of a particular attack.

Anomaly detection systems attempt the much harder job of looking for

anomalous patterns of behaviour in the absence of a clear model of the

662 Chapter 21 ■Network Attack and Defense

attacker’s modus operandi. The hope is to detect attacks that have not been

previously recognized and cataloged. Systems of this type often use AI

techniques — neural networks have been fashionable from time to time.

The dividing line between misuse and anomaly detection is somewhat

blurred. A good borderline case is Benford’s law, which describes the distribu-

tion of digits in random numbers. One might expect that numbers beginning

with the digits ‘1’, ‘2’, ... ‘9’ would be equally common. But in fact with

numbers that come from random natural sources, so that their distribution

is independent of the number system in which they’re expressed, the distri-

bution is logarithmic: about 30% of decimal numbers start with ‘1’. Crooked

clerks who think up numbers to cook the books, or even use random number

generators without knowing Benford’s law, are often caught using it [846].

Another borderline case is the honey trap — something enticing left to attract

attention. I mentioned, for example, that some hospitals have dummy records

with celebrities’ names in order to entrap staff who don’t respect medical

conﬁdentiality.

21.4.3.2 General Limitations of Intrusion Detection

Some intrusions are really obvious. If what you’re worried about is a script

kiddie vandalizing your corporate web site, then the obvious defence is to

have a machine somewhere that fetches the page regularly, inspects it, and

rings a really loud alarm when it changes. (Make sure you do this via an

outside proxy, and don’t forget that it’s not just your own systems at risk. The

kiddie could replace your advertisers’ pictures with porn, for example, and

then you’d want to pull the links to them pretty fast.)

But in the general case, intrusion detection is hard. Cohen proved that

detecting viruses (in the sense of deciding whether a program is going to do

something bad) is as hard as the halting problem, so we can’t ever expect a

complete solution [311].

Another fundamental limitation comes from the fact that there are basically

two different types of security failure — those which cause an error (which

we deﬁned in 6.3 to be an incorrect state) and those which don’t. An example

of the former is a theft from a bank which leaves traces on the audit trail. An

example of the latter is an undetected conﬁdentiality failure caused by a radio

microphone placed by a foreign intelligence service in your room. The former

can be detected (at least in principle, and forgetting for now about the halting

problem) by suitable processing of the data available to you. But the latter

can’t be. It’s a good idea to design systems so that as many failures as possible

fall into the former category, but it’s not always practicable [289].

There’s also the matter of deﬁnitions. Some intrusion detection systems are

conﬁgured to block any instances of suspicious behaviour and in extreme

cases to take down the affected systems. Quite apart from opening the door to

21.4 Defense Against Network Attack 663

service denial attacks, this turns the intrusion detection system into ﬁrewall or

an access control mechanism; and as we’ve already seen, access control is in

general a hard problem and incorporates all sorts of issues of security policy

which people often disagree on or simply get wrong.

I prefer to deﬁne an intrusion detection system as one that monitors the

logs and draws the attention of authority to suspicious occurrences. This is

closer to the way mobile phone operators work. It’s also critical in ﬁnancial

investigations; see [1095] for a discussion by a special agent with the U.S.

Internal Revenue Service, of what he looks for when trying to trace hidden

assets and income streams. A lot hangs on educated suspicion based on long

experience. For example, a $25 utility bill may lead to a $250,000 second house

hidden behind a nominee. Building an effective system means having the

people, and the machines, each do the part of the job they’re best at; and this

means getting the machine to do the preliminary ﬁltering.

Then there’s the cost of false alarms. For example, I used to go to San

Francisco every May, and I got used to the fact that after I’d used my UK

debit card in an ATM ﬁve days in a row, it would stop working. This not

only upsets the customer, but the villains quickly learn to exploit it. (So do

the customers — I just started making sure I got enough dollars out in the

ﬁrst ﬁve days to last me the whole trip.) As in so many security engineering

problems, the trade-off between the fraud rate and the insult rate is the critical

one — and, as discussed in section 15.9, you can’t expect to improve this trade-

off simply by looking at lots of different indicators. In general, you must expect

that an opponent will always get past the threshold if he’s patient enough and

either does the attack very slowly, or does a large number of small attacks.

A difﬁcult policy problem with commercial intrusion detection systems is

redlining. When insurance companies used claim statistics on postcodes to

decide the level of premiums to charge, it was found that many poor and

black areas suffered high premiums or were excluded altogether from cover.

In a number of jurisdictions this is now illegal. In general, if you build an

intrusion detection system based on data mining techniques, you are at serious

risk of discriminating. If you use neural network techniques, you’ll have no

way of explaining to a court what the rules underlying your decisions are, so

defending yourself could be hard. Opaque rules can also contravene European

data protection law, which entitles citizens to know the algorithms used to

process their personal data.

Already in 1997, systems introduced to proﬁle U.S. airline passengers for

terrorism risk, so they could be subjected to more stringent screening, were

denounced by the American-Arab Anti-Discrimination Committee [823]. Since

9/11 such problems have become much worse. How do we judge the balance

point beyond which we just radicalize people and breed more attacks? I’ll

come back to this in Part III.

664 Chapter 21 ■Network Attack and Defense

21.4.4 Speciﬁc Problems Detecting Network Attacks

Turning now to the speciﬁc problem of detecting network intrusion, the

problem is much harder than (say) detecting mobile phone cloning for a

number of reasons. Network intrusion detection products still don’t work

very well, with both high missed alarm and false alarm rates. It’s common not

to detect actual intrusions until afterwards — although once one is detected

by other means, the traces can be found on the logs.

The reasons for the poor performance include the following, in no particular

order.

The Internet is a very noisy environment — not just at the level of con-

tent but also at the packet level. A large amount of random crud arrives

at any substantial site, and enough of it can be interpreted as hostile to

provide a signiﬁcant false alarm rate. A survey by Bellovin [149] reports

that many bad packets result from software bugs; others are the fault

of out-of-date or corrupt DNS data; and some are local packets that

escaped, travelled the world and returned.

There are ‘too few attacks’. If there are ten real attacks per million

sessions — which is almost certainly an overestimate — then even if

the system has a false alarm rate as low as 0.1%, the ratio of false to

real alarms will be 100. We talked about similar problems with burglar

alarms; it’s also a well known problem for medics running screening

programs for diseases like HIV where the test error exceeds the organ-

ism’s prevalence. In general, where the signal is far below the noise, the

guards get tired and even the genuine alarms get missed.

Many network attacks are speciﬁc to particular versions of software, so a

general misuse detection tool must have a large and constantly changing

library of attack signatures.

In many cases, commercial organisations appear to buy intrusion

detection systems simply in order to tick a ‘due diligence’ box to satisfy

insurers or consultants. That means the products aren’t always kept up

to date.

Encrypted trafﬁc can’t easily be subjected to content analysis any more

than it can be ﬁltered for malicious code.

The issues we discussed in the context of ﬁrewalls largely apply to intru-

sion detection too. You can ﬁlter at the packet layer, which is fast but

can be defeated by packet fragmentation; or you can reconstruct each

session, which takes more computation and so is not really suitable for

network backbones; or you can examine application data, which is more

expensive still — and needs to be constantly updated to cope with the

arrival of new applications and attacks.

21.4 Defense Against Network Attack 665

You may have to do intrusion detection both locally and globally. The

antivirus side of things may have to be done on local machines, espe-

cially if the malware arrives on encrypted web sessions; on the other

hand, some attacks are stealthy — the opponent sends 1–2 packets per

day to each of maybe 100,000 hosts. Such attacks are unlikely to be found

by local monitoring; you need a central monitor that keeps histograms of

packets by source and destination address and by port.

So it appears unlikely that a single-product solution will do the trick. Future

intrusion detection systems are likely to involve the coordination of a number

of monitoring mechanisms at different levels both in the network (backbone,

LAN, individual machine) and in the protocol stack (packet, session and

application).

21.4.5 Encryption

In the context of preventing network attacks, many people have been con-

ditioned to think of encryption. Encryption usually does a lot less than you

might hope, as the quote from Butler Lampson and Roger Needham at the

head of this chapter suggests. But it can sometimes be useful. Here I’m going

to describe brieﬂy the four most relevant network encryption scenarios: SSH;

the local link protection offered by WiFi, Bluetooth and HomePlug; IPSec; and

TLS. Finally I’ll brieﬂy discuss public key infrastructures (PKI), which are used

to support the last two of these.

21.4.5.1 SSH

When I use my laptop to read email on my desktop machine, or do anything

with any other machine in our lab for that matter, I use a protocol called

secure shell (SSH) which provides encrypted links between Unix and Windows

hosts [1369, 1, 988]. So when I come in from home over the net, my trafﬁc is

protected, and when I log on from the PC at my desk to another machine in

the lab, the password I use doesn’t go across the LAN in the clear.

SSH was initially written in 1995 by Tatu Yl¨

onen, a researcher at Helsinki

University of Technology in Finland, following a password-snifﬁng attack

there. It not only sets up encrypted connections between machines, so that

logon passwords don’t travel across the network in the clear; it also supports

other useful features, such as forwarding X sessions, which led to its rapid

adoption. (In fact it’s a classic case study in how to get a security product

accepted in the marketplace; see [1083] for an analysis. Normally people don’t

want to use encryption products until a lot of other people are using them too,

because of network effects; so the trick is to bundle some real other beneﬁts

with the product.)

666 Chapter 21 ■Network Attack and Defense

There is a proprietary SSH product from Yl¨

onen’s company, and a number of

open implementations such as OpenSSH and Putty; there’s also an associated

ﬁle transfer protocol SCP (‘secure copy’). There are various conﬁguration

options, but in the most straightforward one, each machine has a public-

private keypair. The private key is protected by a passphrase that the user

types at the keyboard. To connect from (say) my laptop to a server at the

lab, I ensure that my public key is loaded on, and trusted by, the server.

Manual key installation is at once a strength and a weakness; it’s strong in

that management is intuitive, and weak as it doesn’t scale particularly well. In

any case, when I wish to log on to the server I’m prompted for my passphrase;

a key is then set up; and the trafﬁc is both encrypted and authenticated. Fresh

keys are set up after an hour, or after a Gigabyte of trafﬁc has been exchanged.

Possible problems with the use of SSH include the fact that the earliest

version, SSH 1.0, is vulnerable to middleperson attacks because of a poor

key-exchange protocol; and that if you’re typing at the keyboard one character

at a time, then each character gets sent in its own packet. The packet inter-

arrival times can leak a surprising amount of information about what you’re

typing [1203]. However, the worst is probably that most SSH keys are stored

in the clear, without being protected by a password at all. The consequence

is that if a machine is compromised, the same can happen to every other

machine that trusts an SSH key installed on it.

21.4.5.2 WiFi

WiFi is a technology used for wireless local area networks, and is very widely

used: people use it at home to connect PCs to a home router, and businesses

use it too, connecting devices such as tills and payment terminals as well as

PCs. Games consoles and even mobile phones make increasing use of wireless

LANs.

Wiﬁ has come with a series of encryption protocols since its launch in 1997.

The ﬁrst widely-used one, WEP (for wired equivalent privacy), was shown to be

fairly easily broken, even when conﬁgured correctly. Standardised with IEEE

802.11 in 1999, WEP uses the RC4 stream cipher to encrypt data with only a

cyclic redundancy check for integrity. Nikita Borisov, Ian Goldberg and David

Wagner showed that this led to attacks in depth [210]. Known plaintext allows

keystream to be stripped off and reused; in addition, the initial values used in

encryption were only 24 bits, which enabled IV collisions to be found leading

to further depth attacks. False messages could be encrypted and injected into

a wireless LAN, opening it to other attacks. What’s more, the key was only

40 bits long in early implementations, because of U.S. export rules; so keys

could be brute-forced.

That merely whetted cryptanalysts’ appetite. Shortly afterwards, Scott

Fluhrer, Itzhak Mantin and Adi Shamir found a really devastating attack.

21.4 Defense Against Network Attack 667

It is an initialisation weakness in the RC4 stream cipher that interacts with the

way in which WEP set up its initialization vectors [479]; in short order, Adam

Stubbleﬁeld, John Ioannidis and Avi Rubin turned this into a working attack

on WEP [1230]. Vendors bodged up their products so they would not use the

speciﬁc weak keys exploited in the initial attack programs; later programs

used a wider range of weak keys, and the attacks steadily improved. The

history of the attack evolution is told in [993]; the latest attack, by Erik Tews,

Ralf-Philipp Weinmann and Andrei Pyshkin, recovers 95% of all keys within

85,000 packets [1245]. Now there are publicly-available tools that will extract

WEP keys after observing a few minutes’ trafﬁc.

Stronger encryption systems, known as Wi-Fi Protected Access (WPA), aim

to solve this problem and are available on most new products. WPA shipped

in 2003, and was an intermediate solution that still uses RC4. WPA2 shipped in

2004; it is also called the Robust Security Network (RSN) and uses the AES

block cipher in counter mode with CBC-MAC. Within a few years, as older

machines become obsolete, WPA2 should solve the cipher security problem.

So what are we to make of WiFi security? There has been a lot of noise

in the press about how people should set passwords on their home routers, in

case a bad man stops outside your house and uses your network to download

child porn. However, a straw poll of security experts at WEIS 2006 showed

that most did not bother to encrypt their home networks; drive-by downloads

are a fairly remote threat. For most people in the UK or America, it’s just

convenient to have an open network for your guests to use, and so that you

and your neighbours can use each others’ networks as backups. Things are

different in countries where you pay for download bandwidth; there, home

router passwords are mostly set.

Things are different for businesses because of the possibility of targeted

attacks. If you use a Windows machine with Windows shares open, then

someone on your LAN can probably use that to infect you with malware.

A random home owner may not be at much risk — with botnets trading at

about a dollar a machine, it’s not worth someone’s while to drive around

town infecting machines by hand. But if you’re a high-value target, then

the odds change signiﬁcantly. In March 2007, retail chain TJ Maxx reported

that some 45.7 million credit card numbers had been stolen from its systems;

these card numbers largely related to sales in 2003 and 2004, and had been

stolen from 2005 but discovered only in December 2006. The Wall Street

Journal reported that an insecure WiFi connection in St Paul, Mn., was to

blame [1014]; the company’s SEC ﬁling about the incident is at [1252], and the

Canadian Privacy Commissioner concluded that ‘The company collected too

much personal information, kept it too long and relied on weak encryption

technology to protect it — putting the privacy of millions of its customers

at risk’ [1047]. Banks sued the company, with VISA claiming fraud losses of

over $68m from the compromise of 65 million accounts; the banks eventually

668 Chapter 21 ■Network Attack and Defense

settled for damages of $41m [544]. Since June 2007, the banking industry’s

Payment Card Industry Data Security Standard (PCI DSS) requires companies

processing credit card data to meet certain data security standards, and VISA

or Mastercard can ﬁne member banks whose merchants don’t comply with it.

(However, enforcement has historically been weak: it turned out that VISA had

known about TJX’s compliance problems, and had allowed them an extension

until 2009 [1349].)

The latest implementations of WiFi are coming with mechanisms that

encourage users to set up WPA encryption, and usability is a big deal for the

other local connectivity protocols too.

21.4.5.3 Bluetooth

Bluetooth is another protocol used for short-range wireless communication.

It’s aimed at personal area networks, such as linking a headset to a mobile phone,

or linking a mobile phone in your pocket to a hands-free phone interface in

your car. It’s also used to connect cameras and phones to laptops, keyboards

to PCs and so on. Like WiFi, the initially deployed security protocol turned

out to have ﬂaws. In the original version, devices discover each other, and

the users conﬁrm that they wish two devices to pair by entering the same

PIN at their keyboards. An attacker who’s present during this pairing process

can observe the trafﬁc and then brute-force the PIN. Worse, Ollie Whitehouse,

Yaniv Shaked and Avishai Wool ﬁgured out how to force two devices to rerun

the pairing protocol, so that PIN-cracking attacks could be performed even

on devices that were already paired [1341, 1156]. Denis K ¨

ugler also showed

how to manipulate the frequency hopping so as to do a man-in-the-middle

attack [747]. It’s possible to mitigate these vulnerabilities by only doing pairing

in a secure place and refusing requests to rekey.

Now, from version 2.1 (released in 2007), Bluetooth supports Secure Simple

Pairing, an improved protocol [802]. This uses elliptic curve Difﬁe-Hellmann

key exchange to thwart passive eavesdropping attacks, but man-in-the-middle

attacks are harder; they are dealt with by generating a six digit number

for numerical comparison, with a view to reducing the chance of an attack

succeeding to one in a million. However, because one or both of the devices

might lack a keyboard or screen (or both), it’s also possible for the six-digit

number to be generated at one device and entered as a passkey at another; and

there’s a ‘just works’ mode that’s fully vulnerable to a middleperson attack.

Finally, there’s a capability to load keys out of band, such as from some other

protocol that the devices use.

21.4.5.4 HomePlug

HomePlug is a protocol used for communication over the mains power line.

An early version had a low bitrate, but HomePlug AV, available from 2007,

21.4 Defense Against Network Attack 669

supports broadband. (Declaration of interest: I was one of the protocol’s

designers.) It aims to allow TVs, set-top boxes, personal video recorders,

DSL and cable modems and other such devices to communicate in the home

without additional cabling. We were faced with the same design constraints

as the Bluetooth team: not all devices have keyboards or screens, and we

needed to keep costs low. After much thought we decided to offer only two

modes of operation: secure mode, in which the user manually enters into her

network controller a unique AES key that’s printed on the label of every device,

and robust or ‘simple connect’ mode in which the keys are exchanged without

authentication. In fact, the keys aren’t even encrypted in this mode; its purpose

is not to provide security but to prevent wrong associations, such as when

your speakers wrongly get their audio signal from the apartment next door.

We considered offering a public-key exchange protocol, as with Bluetooth,

but came to the conclusion that it didn’t achieve much. If there’s a middleperson

attack going on where the attacker knocks out your set-top box using a jammer

and connects a bogus box of the same type to your mains, then the chances are

that you’ll go to your network controller (some software on your PC) and see a

message ‘Set-top box type Philips 123 seeks admission to network. Certiﬁcate

hash = 12345678. Admit/deny?’ In such a circumstance, most people will press

‘admit’ and allow the attacker in. The only way to prevent them is to get them

to read the certiﬁcate hash from the device label and type it in — and if they’re

going to do that, they might as well type in the key directly [967]. In short,

our design was driven by usability, and we weren’t convinced that public-key

crypto actually bought us anything.

Time will tell which approach was best. And if we turn out to have been

wrong, HomePlug (like Bluetooth and the latest versions of WiFi) lets keys be

set up from other protocols by out-of-band mechanisms. So all devices in the

ofﬁce or home could end up with their keys managed by a single mechanism or

device; and this could be convenient, or a source of vulnerabilities, depending

on how future security engineers build it.

21.4.5.5 IPsec

Another approach is to do encryption and/or authentication at the IP layer

using a protocol suite known as IPsec. IPsec deﬁnes a security association as the

combination of keys, algorithms and parameters used to protect a particular

packet stream. Protected packets are either encrypted or authenticated (or

both); in the latter case, an authentication header is added that protects data

integrity using HMAC-SHA1, while in the former the packet is encrypted and

encapsulated in other packets. (The use of encryption without authentication

is discouraged as it’s insecure [151].) There’s also an Internet Key Exchange

(IKE) protocol to set up keys and negotiate parameters. IKE has been through

a number of versions (some of the bugs that were ﬁxed are discussed in [465]).

670 Chapter 21 ■Network Attack and Defense

IPsec is widely used by ﬁrewall vendors who offer a virtual private network

facility with their products; that is, by installing one of their boxes in each

branch between the local LAN and the router, all the internal trafﬁc can pass

encrypted over the Internet. Individual PCs, such as workers’ laptops and

home PCs, can in theory join a VPN given a ﬁrewall that supports IPsec, but

this is harder than it looks. Compatibility has been a major problem with

different manufacturers’ offerings just not working with each other; although

ﬁrewall-to-ﬁrewall compatibility has improved recently, getting random PCs

to work with a given VPN is still very much a hit-or-miss affair.

IPsec has the potential to stop some network attacks, and be a useful com-

ponent in designing robust distributed systems. But it isn’t a panacea. Indeed,

virtual private networks exacerbate the ‘deperimeterization’ problem already

discussed. If you have thousands of machines sitting in your employee’s

homes that are both in the network (as they connect via a VPN) and connected

to the Internet (as their browser talks to the Internet directly via the home’s

cable modem) then they become a potential weak point. (Indeed, the U.S.

Department of Justice ruled in 2007 that employees can’t use their own PCs or

PDAs for work purposes; all mobile devices used for departmental business

must be centrally managed [108].)

21.4.5.6 TLS

Recall that when discussing public key encryption, I remarked that a server

could publish a public key KS and any web browser could then send a message

Mcontaining a credit card number to it encrypted using KS:{M}KS. This is

in essence what the TLS protocol (formerly known as SSL) does, although in

practice it is more complicated. It was developed to support encryption and

authentication in both directions, so that both http requests and responses can

be protected against both eavesdropping and manipulation. It’s the protocol

that’s activated when you see the padlock on your browser toolbar.

Here is a simpliﬁed description of the version as used to protect web pages

that solicit credit card numbers:

1. the client sends the server a client hello message that contains its name C,

a transaction serial number C#, and a random nonce NC;

2. the server replies with a server hello message that contains its name S,

a transaction serial number S#, a random nonce NS, and a certiﬁcate

CS containing its public key KS. The client now checks the certiﬁcate

CS back to a root certiﬁcate issued by a company such as Verisign and

stored in the browser;

3. the client sends a key exchange message containing a pre-master-secret

key, K0, encrypted under the server public key KS. It also sends a ﬁnished

message with a message authentication code (MAC) computed on all

21.4 Defense Against Network Attack 671

the messages to date. The key for this MAC is the master-secret,K1. This

key is computed by hashing the pre-master-secret key with the nonces

sent by the client and server: K1=h(K0,NC,NS). From this point onward,

all the trafﬁc is encrypted; we’ll write this as {...}KCS in the client-server

direction and {...}KSC from the server to the client. These keys are gener-

ated in turn by hashing the nonces with K1.

4. The server also sends a ﬁnished message with a MAC computed on all

the messages to date. It then ﬁnally starts sending the data.

C→S:C,C#, NC

S→C:S,S#, NS,CS

C→S:{K0}KS

C→S:{ﬁnished,MAC(K1,everythingtodate)}KCS

S→C:{ﬁnished,MAC(K1,everythingtodate)}KSC,{data}KSC

The design goals included minimising the load on the browser, and then

minimising the load on the server. Thus the public key encryption operation is

done by the client, and the decryption by the server; the standard encryption

method (ciphersuite) uses RSA for which encryption can be arranged to be very

much faster than decryption. (This was a wrong design decision as browsers

generally have a lot more compute cycles to spare than servers; it has created

a brisk aftermarket for crypto accelerator boards for web servers.) Also, once a

client and server have established a pre-master-secret, no more public key

operations are needed as further master secrets can be obtained by hashing it

with new nonces.

The full protocol is more complex than this, and has gone through a number

of versions. It supports a number of different ciphersuites, so that export

versions of browsers for example can be limited to 40 bit keys — a condition

of export licensing that was imposed for many years by the U.S. government.

Other ciphersuites support signed Difﬁe-Hellman key exchanges for transient

keys, to provide forward and backward secrecy. TLS also has options for

bidirectional authentication so that if the client also has a certiﬁcate, this can be

checked by the server. In addition, the working keys KCS and KSC can contain

separate subkeys for encryption and authentication. For example, the most

commonly used ciphersuite uses the stream cipher RC4 for the former and

HMAC for the latter, and these need separate keys.

Although early versions of SSL had a number of bugs [1308], version 3

and later (called TLS since version 3.1) appear to be sound (but they have

to be implemented carefully [189]). They are being used for much more than

electronic commerce — an example being medical privacy [280]. In our local

teaching hospital, clinical personnel were issued with smartcards containing

TLS certiﬁcates enabling them to log on to systems containing patient records.

This meant, for example, that researchers could access clinical data from home

during an emergency, or from their university ofﬁces if doing research. TLS

672 Chapter 21 ■Network Attack and Defense

has also been available as an authentication option in Windows from Windows

2000 onwards; you can use it instead of Kerberos if you wish.

Another application is in mail, where more and more mail servers now use

TLS opportunistically when exchanging emails with another mail server that’s

also prepared to use it. This stops passive eavesdropping, although it leaves

open the possibility of middleperson attacks. To stop them too, you need some

means of authenticating the public keys you use, and that brings us to the

topic of public-key certiﬁcates.

21.4.5.7 PKI

During the dotcom boom, a number of companies achieved astronomical

valuations by cornering the market in public-key certiﬁcates. The leading

European certiﬁcate provider, Baltimore, achieved an eleven-ﬁgure market

cap before crashing and burning in 2001. Investors believed that every device

would need a public-key certiﬁcate in order to connect to other devices; you’d

need to pay Baltimore (or Thawte, or Verisign) ten bucks every two years to

renew the certiﬁcate on your toaster, or it wouldn’t talk to your fridge.

As I discussed above, the keys in devices like fridges and toasters are best

set up by local mechanisms such as the Bluetooth and HomePlug pairing

mechanisms. But public key infrastructures are still used in a number of

applications. First, there are the certiﬁcates that web sites use with TLS

and that activate the security icon in your browser. Second, there are private

infrastructures, such as those used by banks to set up keys for SWIFT, by mobile

phone companies to exchange messages between Home Location Registers,

and by companies that use TLS to authenticate users of their networks.

There is frequent semantic confusion between ‘public (key infrastructure)’

and ‘(public key) infrastructure’. In the ﬁrst, the infrastructure can be used by

whatever new applications come along; I’ll call this an open PKI. In the second,

it can’t; I’ll call this a closed PKI.

PKI has a number of intrinsic limitations, many of which have to do with

the ﬁrst interpretation — namely that the infrastructure is provided as a public

service that anyone can use. I discussed many of the underlying problems in

Chapter 7. Naming is difﬁcult, and a certiﬁcate saying ‘Ross Anderson has the

right to administer the machine foo.com’ means little in a world with dozens

of people (and machines) of that name. Also, there’s Kent’s law: the more

applications rely on a certiﬁcate, the shorter its useful life will be.

This is the ‘one key or many’ debate. As the world goes digital, should I

expect to have a single digital key to replace each of the metal keys, credit

cards, swipe access cards and other tokens that I currently carry around? Or

should each of them be replaced by a different digital key? Multiple keys

protect the customer: I don’t want to have to use a key with which I can

remortgage my house to make calls from a payphone. It’s just too easy to

21.4 Defense Against Network Attack 673

dupe people into signing a message by having the equipment display another,

innocuous, one2.

However, the killer turned out to be business needs. Multiple keys are more

convenient for business, as sharing access tokens can lead to greater admin-

istrative costs and liability issues. There were many attempts to share keys;

the smartcard industry tried to market ‘multifunction smartcards’ through the

1990s that could work as bank cards, electricity meter cards and even building

access cards. Singapore even implemented such a scheme, in which even mili-

tary ID doubled as bank cards. However, such schemes have pretty well died

out. In one that I worked on — to reuse bank cards in electricity meters — the

issues were control of the customer base and of the process of developing,

upgrading and reissuing cards. In other cases, projects foundered because

no-one could agree which company’s logo would go on the smartcard.

Now the standard PKI machinery (the X.509 protocol suite) was largely

developed to provide an electronic replacement for the telephone book, so

it tends to assume that everyone will have a unique name and a unique

key in an open PKI architecture. Governments hoped for a ‘one key ﬁts all’

model of the world, so they could license and control the keys. But, in most

applications, the natural solution is for each business to run its own closed

PKI, which might be thought of at the system level as giving customers a

unique account number which isn’t shared with anyone else. Since then, the

CA market has fractured; whereas in the late 1990s, Internet Explorer shipped

with only a handful of CA keys (giving huge if temporary fortunes to the ﬁrms

that controlled them), now the version in Windows XP contains hundreds.

This in turn leads to issues of trust. You don’t really know who controls a

key whose signature is accepted by a typical browser. Further issues include

If you remove one of the 200-plus root certiﬁcates from Windows XP

Service Pack 2, then Windows silently replaces it — unless you’ve got

the skill to dissect out the software that does this [613]. Vista comes with

fewer root certiﬁcates — but you can’t delete them at all. This could

be bad news for a company that doesn’t want to trust a competitor, or

a government that doesn’t want to trust foreigners. For example, the

large CA Verisign also does wiretap work for the U.S. government;

so if I were running China, I wouldn’t want any Chinese PC to trust

their certiﬁcates (as Verisign could not just sign bad web pages — they

could also sign code that Chinese machines would install and run).

Usability is dreadful, as many sites use out-of-date certs, or certs that

correspond to the wrong company. As a result, users are well trained

to ignore security warnings. For example, when a New Zealand bank

2I just don’t know how to be conﬁdent of a digital signature I make even on my own PC — and

I’ve worked in security for over ﬁfteen years. Checking all the software in the critical path

between the display and the signature software is way beyond my patience.

674 Chapter 21 ■Network Attack and Defense

messed up its certiﬁcate with the result that users got warned it didn’t

correspond to the bank, only one user out of 300 stopped — the rest just

went ahead with their business [569].

It’s bad enough that the users don’t care whether certiﬁcates work; yet

the CAs don’t seem to care, either. The UK certiﬁer Tscheme was set up

by industry as a self-regulatory scheme under the Electronic Commu-

nications Act as ‘a source of independent assurance for all types of e-business

and e-government transactions — especially for those transactions that depend

on reliable, secure online identities.’ It was noticed in July 2006 that

https://www.tscheme.org/ had its certiﬁcation path misconﬁgured:

there was a certiﬁcate missing in the middle of the chain, so veriﬁca-

tion failed unless you manually added the missing cert. By December

2007, it still wasn’t properly ﬁxed. According to the documentation,

the ‘HMG Root CA’ should certify everything, yet it doesn’t certify the

Tscheme ‘Trustis FPS Root CA’, and neither is included in the stan-

dard Firefox distribution. In the CA world, it seems, everyone wants

to be root, and no-one wants anyone else’s signature on their keys, as

then they’d have no reason to exist. So stuff still doesn’t work.

Many users disable security features on their browsers, even if these

weren’t disabled by default when the software shipped. Recall that the

third step of the TLS protocol was for the client browser to check the

cert against its stored root certiﬁcates. If the check fails, the browser

may ask the client for permission to proceed; but many browsers are

conﬁgured to just proceed anyway.

Certs bind a company name to a DNS name, but their vendors are usu-

ally not authorities on either; they hand out certiﬁcates after cursory

due diligence, and in their ‘certiﬁcation practice statements’ they go

out of their way to deny all liability.

There are still technical shortcomings. For example, the dominant cer-

tiﬁcate format (X.509) does not have the kind of ﬂexible and scalable

‘hot card’ system which the credit card industry has evolved, but rather

assumes that anyone relying on a cert can download a certiﬁcate revo-

cation list from the issuing authority. Also, certs are designed to certify

names, when for most purposes one wants to certify an authorization.

Behind all this mess lies, as usual, security economics. During the dotcom

boom in the 1990s, the SSL protocol (as TLS then was) won out over a more

complex and heavyweight protocol called SET, because it placed less of a

burden on developers [72]. This is exactly the same reason that operating

systems such as Windows and Symbian were initially developed with too

little security — they were competing for pole position in a two-sided market.

21.5 Topology 675

The downside in this case was that the costs of compliance were dumped on

the users — who are unable to cope [357].

In short, while public key infrastructures can be useful in some applications,

they are not the universal solution to security problems that their advocates

claimed in the late 1990s. It’s a shame some governments still think they can

use PKI as a mechanism for empire-building and social control.

21.5 Topology

The topology of a network is the pattern in which its nodes are connected. The

Internet classically is thought of as a cloud to which all machines are attached,

so in effect every machine is (potentially) in contact with every other one.

So from the viewpoint of a ﬂash worm that propagates from one machine

to another directly, without human intervention, and by choosing the next

machine to attack at random, the network can be thought of as a fully

connected graph. However, in many networks each node communicates with

only a limited number of others. This may result from physical connectivity,

as with PCs on an isolated LAN, or with a camera and a laptop that are

communicating via Bluetooth; or it may come from logical connectivity. For

example, when a virus spreads by mailing itself to everyone in an infected

machine’s address book, the network that it’s infecting is one whose nodes

are users of the vulnerable email client and whose edges are their presence in

each others’ address books.

We can bring other ideas and tools to bear when the network is in effect

asocial network like this. In recent years, physicists and sociologists have

collaborated in applying thermodynamic models to the analysis of complex

networks of social interactions; the emerging discipline of network analysis is

reviewed in [965], and has been applied to disciplines from criminology to the

study of how new technologies diffuse.

Network topology turns out to be important in service denial attacks. Rulers

have known for centuries that when crushing dissidents, it’s best to focus

on the ringleaders; and when music industry enforcers try to close down

peer-to-peer ﬁle-sharing networks they similarly go after the most prominent

nodes. There’s now a solid scientiﬁc basis for this. It turns out that social

networks can be modelled by a type of graph in which there’s a power-law

distribution of vertex order; in other words, a small number of nodes have

a large number of edges ending at them. These well-connected nodes help

make the network resilient against random failure, and easy to navigate. Yet

Reka Albert, Hawoong Jeong and Albert-L´

aszl´

o Barab´

asi showed that they also

make such networks vulnerable to targeted attack. Remove the well-connected

nodes, and the network is easily disconnected [20].

676 Chapter 21 ■Network Attack and Defense

This has prompted further work on how topology interacts with conﬂict.

For example, Shishir Nagaraja and I extended Albert, Jeong and Barab´

asi’s

work to the dynamic case in which, at each round, the attacker gets to destroy

some nodes according to an attack strategy, and the defender then gets to

replace a similar number of nodes using a defense strategy. We played attack

and defense strategies off against each other; against a decapitation attack,

the best defense we found was a cell structure. This helps explain why peer-

to-peer systems with ring architectures turned out to be rather fragile — and

why revolutionaries have tended to organise themselves in cells [924]. George

Danezis and Bettina Wittneben applied these network analysis ideas to privacy,

and found that by doing trafﬁc analysis against just a few well-connected

organisers, a police force can identify a surprising number of members of a

dissident organisation. The reason is that if you monitor everyone who calls,

or is called by, the main organisers of a typical social network, you get most of

the members — unless effort was expended in organising it in a cell structure

in the ﬁrst place [345].

These techniques may well become even more relevant to network attack

and defence for a number of reasons. First, early social-network techniques

have produced real results; the capture of Saddam Hussein used a layered

social network analysis [623]. Second, as people try to attack (and defend)

local networks organised on an ad-hoc basis using technologies like WiFi and

Bluetooth, topology will matter more. Third, social networking sites —

and more conventional services like Google mail that use introductions to

acquire new customers — have a lot of social network information that can

be used to track people; if a phisherman uses Google mail, the police can

look for the people who introduced him, and then for everyone else they

introduced, when searching for contacts. Fourth, as social structure starts to be

used against wrongdoers (and against citizens by repressive regimes) people

will invest in cell-structured organisations and in other stealth techniques to

defeat it. Finally, there are all sorts of useful things that can potentially be done

with topological information. For example, people may be more able to take

a view on whether devices that are ‘nearby’ are trustworthy, and you may

not need to ﬁlter trafﬁc so assiduously if it can come from only a few sources

rather than the whole Internet. This isn’t entirely straightforward, as systems

change over time in ways that undermine their builders’ trust assumptions,

but it can still be worth thinking about.

21.6 Summary

Preventing and detecting attacks that are launched over networks, and partic-

ularly over the Internet, is probably the most newsworthy aspect of security

engineering. The problem is unlikely to be solved any time soon, as many

Research Problems 677

different kinds of vulnerability contribute to the attacker’s toolkit. Ideally,

people would run carefully written code on trustworthy platforms. In real

life, this won’t happen always, or even often. In the corporate world, there are

grounds for hope that ﬁrewalls can keep out the worst of the attacks, careful

conﬁguration management can block most of the rest, and intrusion detection

can catch most of the residue that make it through. Home users are less well

placed, and most of the machines being recruited to the vast botnets we see in

action today are home machines attached to DSL or cable modems.

Hacking techniques depend partly on the opportunistic exploitation of

vulnerabilities introduced accidentally by the major vendors, and partly on

techniques to social-engineer people into running untrustworthy code. Most

of the bad things that result are just the same bad things that happened a

generation ago, but moved online, on a larger scale, and with a speed, level of

automation and global dispersion that leaves law enforcement wrong-footed.

Despite all this, the Internet is not a disaster. It’s always possible for a

security engineer, when contemplating the problems we’ve discussed in this

chapter, to sink into doom and gloom. Yet the Internet has brought huge

beneﬁts to billions of people, and levels of online crime are well below the

levels of real-world crime. I’ll discuss this in more detail when we get to policy

in Part III; for now, note that the $200 m–$1 bn lost in the USA to phishing

in 2006 was way below ordinary fraud involving things like checks, not to

mention the drug trade or even the trade in stolen cars.

Herd effects matter. A useful analogy for the millions of insecure computers

is given by the herds of millions of gnu that once roamed the plains of Africa.

The lions could make life hard for any one gnu, but most of them survived for

years by taking shelter in numbers. The Internet’s much the same. There are

analogues of the White Hunter, who’ll carefully stalk a prime trophy animal;

so you need to take special care if anyone might see you in these terms.

And if you think that the alarms in the press about ‘Evil Hackers Bringing

Down the Internet’ are somehow equivalent to a hungry African peasant

poaching the game, then do bear in mind the much greater destruction done

by colonial ranching companies who had the capital to fence off the veld in

100,000-acre lots. Economics matters, and we are still feeling our way towards

the kinds of regulation that will give us a reasonably stable equilibrium. In fact,

colleagues and I wrote a report for the European Network and Information

Security Agency on the sort of policy incentives that might help [62]. I’ll return

to policy in Part III.

Research Problems

Seven years ago, the centre of gravity in network security research was

technical: we were busy looking for new attacks on protocols and applications

678 Chapter 21 ■Network Attack and Defense

as the potential for denial-of-service attacks started to become clear. Now, in

2007, there are more threads of research. Getting protocols right still matters

and it’s unfortunate (though understandable in business terms) that many

ﬁrms still ship products quickly and get them right later. This has led to calls

for vendor liability, for example from a committee of the UK Parliament [625].

On the security-economics front, there is much interesting work to be done on

decent metrics: on measuring the actual wickedness that goes on, and feeding

this not just into the policy debate but also into law enforcement.

Systems people do a lot of work on measuring the Internet to understand

how it’s evolving as more and more devices, people and applications join in.

And at the level of theory, more and more computer scientists are looking at

ways in which network protocols could be aligned with stakeholder interests,

so that participants have less incentive to cheat [971].

Further Reading

The early classic on Internet security was written by Steve Bellovin and

Bill Cheswick [157]; other solid books are by Simson Garﬁnkel and Eugene

Spafford on Unix and Internet security [517], and by Terry Escamilla on

intrusion detection [439]. These give good introductions to network attacks

(though like any print work in this fast-moving ﬁeld, they are all slightly

dated). The seminal work on viruses is by Fred Cohen [311], while Java

security is discussed by Gary McGraw and Ed Felten [859] as well as by

LiGong (who designed it) [539]. Eric Rescorla has a book on the details of

TLS [1070]; another useful description — shorter than Eric’s but longer than

the one I gave above — is by Larry Paulson [1010]. Our policy paper for ENISA

can be found at [62].

It’s important to know a bit about the history of attacks — as they

recur — and to keep up to date with what’s going on. A survey of

security incidents on the Internet in the late 1990s can be found in John

Howard’s thesis [626]. Advisories from CERT [321] and bugtraq [239] are one

way of keeping up with events, and hacker sites such as www.phrack.com

bear watching. However, as malware becomes commercial, I’d suggest you

also keep up with the people who measure botnets, spam and phishing.

As of 2007, I’d recommend Team Cymru at http://www.cymru.com/,the

Anti-Phishing Working Group at http://www.antiphishing.org/,the

Shadowserver Foundation at http://www.shadowserver.org/, Symantec’s

half-yearly threat report at www.symantec.com/threatreport/, and our blog

at www.lightbluetouchpaper.net.

CHAPTER

The DeCSS case is almost certainly a harbinger of what I would consider to be the

deﬁning battle of censorship in cyberspace. In my opinion, this will not be fought

over pornography, neo-Nazism, bomb design, blasphemy, or political dissent.

Instead, the Armageddon of digital control, the real death match between the Party

of the Past and Party of the Future, will be fought over copyright.

— John Perry Barlow

Be very glad that your PC is insecure — it means that after you buy it, you can break

into it and install whatever software you want. What YOU want, not what Sony or

Warner or AOL wants.

— John Gilmore

22.1 Introduction

contentious issues of the digital age. At the political level, there is the conﬂict

alluded to by Barlow in the above quotation. The control of information has

been near the centre of government concerns since before William Tyndale (one

of the founders of the Cambridge University Press) was burned at the stake

for printing the Bible in English. The sensitivity continued through the estab-

lishment of modern copyright law starting with the Statute of Anne in 1709,

through the eighteenth century battles over press censorship, to the Enlight-

enment and the framing of the U.S. Constitution. The link between copyright

and censorship is obscured by technology from time to time, but has a habit of

reappearing. Copyright mechanisms exist to keep information out of the hands

of people who haven’t paid for it, while censors keep information out of the

hands of people who satisfy some other criterion. If ISPs are ever compelled to

679

680 Chapter 22 ■Copyright and DRM

install ﬁlters that will prevent their customers from downloading copyrighted

content, these ﬁlters could also be used to prevent the download of seditious

content.

In the last few generations, the great wealth accruing to the owners of

literary copyright, ﬁlms and music has created another powerful interest in

control. As the music and ﬁlm industries in particular feared loss of sales to

digital copying, they lobbied for sweetheart laws — the DMCA in America,

and a series of IP Directives in Europe — that give special legal protection

to mechanisms that enforce copyright. These laws are now being used and

abused for all sorts of other purposes, from taking down phishing websites to

stopping people from reﬁlling printer cartridges.

The ostensible target of these laws, though, remains the DRM mechanisms

that are used in products such as Windows Media Player and Apple’s iTunes

to control copying of music and videos that have been purchased online.

I’ll describe how DRM works. The basic mechanism is to make available

an encrypted media ﬁle, and then to sell separately a ‘license’ which is the

key to the media ﬁle encrypted using a key unique to the user, plus some

statements in a ‘rights management language’ about what the user can do with

the content. I’ll also describe some interesting variants such as satellite TV

encryption systems, copyright marking, traitor tracing, and Blu-Ray. And, of

course, no discussion of copyright would be complete these days without some

mention of ﬁle-sharing systems, and the mechanisms used by Hollywood to

try to close them down.

Finally, there are some thorny policy issues tied up in all this. Economists

pointed out that stronger DRM would help the platform industry more than

the music industry, and their warnings have come true: Apple is making more

money and the music majors are making less. The possible implications for

video are interesting. And ﬁnally there are some serious privacy issues with

rights management systems. Do you really want a license management server,

whether in Redmond or Cupertino, to know every music track you’ve ever

listened to, and every movie you’ve ever watched?

22.2 Copyright

The protection of copyright has for years been an obsession of the ﬁlm,

music and book publishing industries (often referred to collectively — and

perjoratively — by computer industry people as Hollywood). But this didn’t

start with the Internet. There were long and acrimonious disputes in many

countries about whether blank audio- or videocassettes should be subjected

to a tax whose proceeds would be distributed to copyright owners; and the

issue isn’t conﬁned to electronic media. In the UK, several million pounds a

22.2 Copyright 681

year are distributed to authors whose books are borrowed from public lending

libraries [1050]. Going back to the nineteenth century, there was alarm that

the invention of photography would destroy the book publishing trade; and

in the sixteenth, the invention of movable type printing was considered to be

highly subversive by most of the powers that were, including princes, bishops

and craft guilds.

There’s a lot we can learn from historical examples such as book publishing,

and pay-TV. But I’m going to start by looking at software protection — as

most of the current copyright issues have been played out in the PC and

games software markets over the last twenty years or so. Also, the music

industry forced the computer industry to introduce DRM, saying that without

it they’d be ruined — and the computer industry for years retorted that the

music industry should just change its business model, so it’s interesting to

use software as the baseline. Finally, the computer industry frequently argued

in its tussles with the music majors that in an open platform such as the PC

it’s intrinsically hard to stop people copying bitstreams — so how did they

themselves cope?

22.2.1 Software

Software for early computers was given away free by the hardware vendors or

by users who’d written it. IBM even set up a scheme in the 1960’s whereby its

users could share programs they had written. (Most of them were useless as

they were too specialised, too poorly documented, or just too hard to adapt.)

So protecting software copyright was not an issue. Almost all organizations

that owned computers were large and respectable; the software tended to

require skilled maintenance; and so they often had full-time system engineers

employed by the hardware vendor on site. There are still sectors which

operate on this business model. For example, one supplier of bank dealing

room software takes the view that anyone who pirates their code is welcome,

as using it without skilled technical support would be a fast way for a bank to

lose millions.

But when minicomputers arrived in the 1960’s, software costs started to

become signiﬁcant. Hardware vendors started to charge extra for their oper-

ating system, and third party system houses sprang up. To begin with, they

mostly sold you a complete bespoke system — hardware, software and main-

tenance — so piracy was still not much of an issue. By the mid-1970’s, some of

them had turned bespoke systems into packages: software originally written

for one bakery would be parametrized and sold to many bakeries. The most

common copyright dispute in those days was when a programmer left your

company to join a competitor, and their code suddenly acquired a number of

your features; the question then was whether he’d taken code with him, or

reimplemented it.

682 Chapter 22 ■Copyright and DRM

The standard way to resolve such a problem is to look at software birth-

marks — features of how a particular implementation was done, such as the

order in which registers are pushed and popped. This continues to be impor-

tant, and there are various code comparison tools available — many of them

developed in universities to detect students cheating on programming assign-

ments. (This thread of research leads to general purpose plagiarism detection

tools, which can trawl through natural language as well as code and typically

recognise a passage of text by indexing it according to the least common words

which appear in it [589], on to systems used by humanities scholars to ﬁgure

out whether Bacon wrote Shakespeare, and back to tools which try to identify

the authors of viruses from their coding style [746].)

With time, people invented lots of useful things to do with software. So a

ﬁrm that had bought a minicomputer for stock control (or contracted for time

on a bureau service) might be tempted to run a statistical program as well to

prepare management reports. Meanwhile, the installed base of machines got

large enough for software sharing to happen more than just occasionally. So

some system houses started to put in copyright enforcement mechanisms. A

common one was to check the processor serial number; another was the time

bomb. When I worked in 1981 for a company selling retail stock control systems,

we caused a message to come up every few months saying something like

‘Fault no. WXYZ — please call technical support’. WXYZ was an encrypted

version of the license serial number, and if the caller claimed to be from that

customer we’d give them a password to re-enable the system for the next

few months. (If not, we’d send round a salesman.) This mechanism could

have been defeated easily if the ‘customer’ understood it, but in practice it

worked ﬁne: most of the time it was a low-level clerk who encountered the

fault message and called our ofﬁce.

Software piracy really started to become an issue when the arrival of

microcomputers in the late 1970’s and early 80’s created a mass market, and

software houses started to ship products that didn’t need support to install

and run. Initial responses varied. There was a famous open letter from Bill

Gates in 1976, a year after Microsoft was founded, in which he complained

that less than 10% of all microcomputer users had paid them for BASIC [502].

‘Who cares if the people who worked on it get paid?’ he asked. ‘Is this fair?’

His letter concluded: ‘Nothing would please me more than being able to hire

ten programmers and deluge the hobby market with good software’.

Appeals to people’s sense of fair play only got so far, and the industry next

seized on the obvious difference between minis and micros — the latter had no

processor serial numbers. There were three general approaches tried: to add

uniqueness on to the machine, to create uniqueness in it, or to use whatever

uniqueness happened to exist already by chance.

22.2 Copyright 683

1. The standard way to add hardware uniqueness was a dongle — a device,

typically attached to the PC’s parallel port, which could be interrogated

by the software. The simplest just had a serial number; the most com-

mon executed a simple challenge-response protocol; while some top-end

devices actually performed some critical part of the computation.

2. A cheaper and very common strategy was for the software to install

itself on the PC’s hard disk in a way that was resistant to naive copy-

ing. For example, a sector of the hard disk would be marked as bad, and

a critical part of the code or data written there. Now if the product were

copied from the hard disk using the utilities provided by the operating

system for the purpose, the data hidden in the bad sector wouldn’t be

copied and so the copy wouldn’t work. A variant on the same theme was

to require the presence of a master diskette which had been customized

in some way, such as by formatting it in a strange way or even burn-

ing holes in it with a laser. In general, though, a distinction should be

drawn between protecting the copy and protecting the master; it’s often

a requirement that people should be able to make copies for backup

if they wish, but not to make copies of the copies (this is called copy

generation control).

3. A product I worked on stored the PC’s conﬁguration — what cards were

present, how much memory, what type of printer — and if this changed

too radically, it would ask the user to phone the helpline. It’s actually

quite surprising how many unique identiﬁers there are in the aver-

age PC; ethernet addresses and serial numbers of disk controllers are

only the more obvious ones. Provided you have some means of dealing

with upgrades, you can use component details to tie software to a given

machine.

A generic attack that works against most of these defenses is to go through

the software with a debugger and remove all the calls made to the copy

protection routines. Many hobbyists did this for sport, and competed to put

unprotected versions of software products online as soon as possible after

their launch. Even people with licensed copies of the software often got

hold of unprotected versions as they were easier to back up and often more

reliable generally. You can stop this by having critical code somewhere really

uncopiable, such as in a dongle, but the lesson from this arms race was that

the kids with the debuggers would always break your scheme eventually.

The vendors also used psychological techniques.

The installation routine for many business programs would embed the

registered user’s name and company on the screen, for example, in the

toolbar. This wouldn’t stop a pirate distributing copies registered in a

684 Chapter 22 ■Copyright and DRM

false name, but it will discourage legitimate users from giving casual

copies to colleagues.

Industry publicists retailed stories of organizations that had come

unstuck when they failed to get a critical upgrade of software they hadn’t

paid for. One of the favourite stories was of the U.S. army bases in Ger-

many that didn’t pay for the VAX VMS operating system and got hacked

after they didn’t get a security patch (described above in section 2.5.4).

If early Microsoft software (Multiplan, Word or Chart) thought you were

running it under a debugger, trying to trace through it, it would put up

the message ‘The tree of evil bears bitter fruit. Now trashing program

disk.’ It would then seek to track zero on the ﬂoppy and go ‘rrnt, rrnt,

rrnt’.

In the mid- to late-1980s, the market split. The games market moved in the

direction of hardware protection, and ended up dominated by games console

products with closed architectures where the software is sold in proprietary

cartridges. The driver for this was that consumers are more sensitive about the

sticker price of a product than about its total cost of ownership, so it makes

sense to subsidise the cost of the console out of later sales of software for it (a

strategy since adopted by printer makers who subsidise the printers from the

ink cartridges). This strategy led to strict accessory control in which hardware

protection was used to prevent competitors selling software or other add-ons

unless they had paid the appropriate royalty.

Business software vendors, however, generally stopped trying to protect

mass market products using predominantly technical means. There were

several reasons.

Unless you’re prepared to spend money on seriously tamper resistant

dongle hardware which executes some of your critical code, the mecha-

nisms will be defeated by people for whom it’s an intellectual challenge,

and unprotected code will be anonymously published. Code that isn’t

protected in the ﬁrst place is less of a challenge.

As processors got faster and code got more complex, operating system

interfaces became higher level, and software protection routines of the

‘bad disk sector’ variety became harder to write. And now that it’s possi-

ble to run a Windows NT system on top of Linux using VMware or Xen,

application software can be completely shielded from machine speciﬁcs

such as ethernet addresses. The net effect is an increase in the cost and

complexity of both protection and piracy.

Protection is a nuisance. Multiple dongles get in the way or even inter-

fere with each other. Software protection techniques tend to make a

22.2 Copyright 685

product less robust and cause you problems — as when their hard disk

fails and they recover from backup to a new disk. Protection mecha-

nisms can also cause software from different vendors to be unnecessarily

incompatible and in some cases unable to reside on the same machine.

Technical support became more and more important as software prod-

ucts became more complex, and you only get it if you pay for the

software.

The arrival of computer viruses was great for the industry. It forced cor-

porate customers to invest in software hygiene, which in turn meant

that casual copying couldn’t be condoned so easily. Within a few years,

antivirus programs made life much harder for copy protection design-

ers in any case, as non-standard operating system usage tended to set off

virus alarms.

There was not much money to be made out of harassing personal users

as they often made only casual use of the product and would throw it

away rather than pay.

A certain level of piracy was good for business. People who got a pirate

copy of a tool and liked it would often buy a regular copy, or persuade

their employer to buy one.

In Microsoft’s case, customer reaction to their scare message was pretty

negative.

Many vendors preferred not to have to worry about whether the soft-

ware was licensed to the user (in which case he could migrate it to a new

machine) or to the machine (in which case he could sell the computer

second-hand with the software installed). As both practices were com-

mon, mechanisms that made one or the other very much harder caused

problems. Mechanisms that could easily deal with both (such as dongles)

tended to be expensive, either to implement, or in call-centre support

costs.

Finally, Borland shook up the industry with its launch of Turbo Pas-

cal. Before then a typical language compiler cost about $500 and came

with such poor documentation that you had to spend a further $50 on a

book to tell you how to use it. Borland’s product cost $49.95, was tech-

nically superior to the competition, and came with a manual that was

just as good as a third party product. (So, like many other people, once

I’d heard of it, borowed a copy from a friend, tried it and liked it, I went

out and bought it.) ‘Pile it high and sell it cheap’ simply proved to be a

more proﬁtable business model — even for speciality products such as

compilers.

686 Chapter 22 ■Copyright and DRM

The industry then swung to legal solutions. The main initiative was to

establish anti-piracy trade organizations in most countries (in the USA, the

Software Publishers’ Association) that brought high-proﬁle prosecutions of

large companies that had been condoning widespread use of pirate PC soft-

ware. This was followed up by harassing medium and even small businesses

with threatening letters demanding details of the company’s policy on enforc-

ing copyright — holding out a carrot of approved software audit schemes and

a stick of possible raids by enforcement squads. All sorts of tricks were used

to get pirates to incriminate themselves. A typical ruse was the salted list;

for example, one trade directory product I worked on contained details of a

number of bogus companies with phone numbers directed to the publisher’s

help desk, whose staff would ask for the caller’s company and check it off

against the list of paid subscribers.

Eventually, the industry discovered that the law not only provides tools

for enforcement, but sets limits too. The time-honoured technique of using

timebombs has now been found to be illegal in a number of jurisdictions.

In 1993, for example, a software company director in Scunthorpe, England,

received a criminal conviction under Britain’s Computer Misuse Act for

‘making an unauthorized modiﬁcation’ to a system after he used a time-bomb

to enforce payment of an disputed invoice [313]. Many jurisdictions now

consider time bombs unacceptable unless the customer is adequately notiﬁed

of their existence at the time of purchase.

The emphasis is now swinging somewhat back in the direction of technical

mechanisms. Site licence agreements are enforced using license servers,which

are somewhat like dongles but are implemented on PCs which sit on a

corporate network and limit the number of copies of an application that can

run simultaneously. They can still be defeated by disassembling the application

code, but as code becomes larger this gets harder, and combined with the threat

of legal action they are often adequate.

The model to which the software industry is converging is thus one that

combines technical and legal measures, understanding the limits of both, and

accepting that a certain amount of copying will take place (with which you try

to leverage fully-paid sales). One of Bill’s more revealing sayings is:

Although about three million computers get sold every year in

China, people don’t pay for the software. Someday they will,

though. And as long as they’re going to steal it, we want them to

steal ours. They’ll get sort of addicted, and then we’ll somehow

ﬁgure out how to collect sometime in the next decade [518].

The latest emphasis is on online registration. If you design your product so

that customers interact with your web site — for example, to download the

latest exchange rates, virus signatures or security patches, then you can keep

22.2 Copyright 687

a log of everyone who uses your software. But this can be dangerous. When

Microsoft tried it with Registration Wizard in Windows 95, it caused a storm

of protest. Also, a colleague found that he couldn’t upgrade Windows 98 on

a machine on his yacht since it was always ofﬂine. And when I ﬁrst tried

Microsoft Antispyware Beta on a machine we had at home with Windows XP,

it was denounced as a pirate copy — despite the fact that the PC had been

bought legally in a local store. Microsoft did sort that out for me, but having

ﬂaky registration mechanisms clearly costs money, and building robust ones is

not trivial if you’re selling large volumes of many products through complex

supply chains. (In fact, it’s against the public interest for security patches to

be limited to registered licensees; a security-economics analysis we did of the

problem recommended that the practice be outlawed [62].)

It’s also worth noting that different methods are used to counter different

threats. Large-scale commercial counterfeiting may be detected by monitoring

product serial numbers registered online, but such operations are found and

closed down by using investigative agencies to trace their product back through

the supply chains. For example, once they got their product registration sorted

out, Microsoft found that a third of the copies of Ofﬁce sold in Germany were

counterfeit, and traced them to a small factory a few miles up the road from us

in Cambridge. Almost all the factory’s staff were unaware of the scam — they

believed the company was a bona ﬁde Microsoft supplier. They were even

proud of it and their salesmen used it to try to get disk duplication business

from other software vendors.

That is more or less what’s done with the personal and small business

sectors, but with medium sized and large businesses the main risk is that

fewer legal copies will be purchased than there are machines which run them.

The usual countermeasure is to combine legal pressure from software trade

associations with site licences and rewards for whistleblowers. It’s signiﬁcant

that companies such as Microsoft make the vast bulk of their sales from

business rather than personal customers. Many large businesses prefer not

to have machines registered online individually, as they want to keep their

staff numbers and structures conﬁdential from the vendors; many vendors

respect this, but the downside is that an ‘unprotected’ binary originally

issued to a large company is often the standard ‘warez’ that people swap.

Many ﬁrms still hold back from using online registration to enforce copyright

aggressively against personal users; the potential extra revenues are small

given the possible costs of a public backlash. Other considerations include the

difﬁculty of tracing people who change addresses or trade PCs secondhand. It

is just very expensive to maintain a high-quality database of millions of small

customers.

For companies that do have deep pockets, such as Google, one option is to

provide not just the software but also the processor to run it. It’s worth noting

that software-as-a-service may be the ultimate copyright protection or DRM

688 Chapter 22 ■Copyright and DRM

for software (or any other content that can live online): you can’t buy it, freeze

the version you’re running, or use it ofﬂine. You may also get to control all

your customers’ data too, giving you impressive lockin. (I will discuss web

services in the next chapter.)

With that exception, none of the mass-market protection technologies avail-

able at the beginning of the 21st century is foolproof, especially against a

determined opponent. But by using the right combination of them a large

software vendor can usually get a tolerable result — especially if prices aren’t

too extortionate and the vendor isn’t too unpopular. Small software companies

are under less pressure, as their products tend to be more specialised and the

risk of copying is lower, so they can often get away with making little or no

effort to control copying. (And if they have only a few customers, it may even

be economic for them to supply software as a service.)

There are also many alternative business models. One is to give away a

limited version of the product, and sell online a password which unlocks its

full functionality. Unix was popularized by giving it away free to universities,

while companies had to pay; a variant on this theme is to give basic software

away free to individuals but charge companies, as Netscape did. An even

more radical model is to give your software away completely free, and make

your money from selling services. The Linux industry makes its money from

consultancy and support, while Web applications such as Google Documents

make their money from advertising.

This experience has led many computer people to believe that the solution

for Hollywood’s problem lies in a change of business model. But before we

dive into the world of protecting multimedia content, let’s look brieﬂy at a few

historical precedents.

22.2.2 Books

Carl Shapiro and Hal Varian present a useful historical lesson in the rise

of book publishing [1159]. In 1800, there were only 80,000 frequent readers

in England; most of the books up till then were serious philosophical or

theological tomes. After the invention of the novel, a mass market appeared

for books, and circulating libraries sprung up to service it. The educated

classes were appalled, and the printers were frightened that the libraries

would deprive them of sales.

But the libraries so whetted people’s appetite for books that the number

of readers grew to 5,000,000 by 1850. Sales of books soared as people bought

books they’d ﬁrst borrowed from a library. The library movement turned out

to have been the printers’ greatest ally and helped create a whole new market

for mass-market books.

22.2 Copyright 689

22.2.3 Audio

Pirates have also been copying music and other audio much longer than

software. Paganini was so worried that people would copy his violin concertos

that he distributed the scores himself to the orchestra just before rehearsals

and performances, and collected them again afterwards. (As a result, many of

his works were lost to posterity.)

In recent years, there have been several ﬂurries of industry concern. When

the cassette recorder came along in the 1960’s, the record industry lobbied

for (and in some countries got) a tax on audiocassettes, to be distributed to

Sergeant Pepper contained a 20 KHz spoiler tone that should in theory have

combined with the 21 KHz bias frequency of the tape to produce a 1 KHz

whistle that would spoil the sound. In practice it didn’t work, as many record

players didn’t have the bandwidth to pick up the spoiler tone. But in practice

this didn’t matter. Cassettes turned out not to be a huge problem because the

degradation in quality is noticeable on home equipment; many people just

used them to record music to listen to in their cars. Then, in the 1980s, the

arrival of the Sony Walkman made cassettes into big business, and although

there was some copying, there were huge sales of pre-recorded cassettes and

the music industry cleaned up.

The introduction of digital audio tape (DAT) caused the next worry, because

a perfect copy of the contents of a CD could be made. The eventual response

was to introduce a serial copy management system (SCMS) — a single bit in the

tape header that said whether a track could be copied or not [648]. The idea

was that copies made from a CD would be marked as uncopiable, so people

could copy CDs they already owned, to listen to on the move, but couldn’t

make copies of the copies. This didn’t work well, as the no-more-copies bit

is ignored by many recorders and can be defeated by simple ﬁltering. Again,

this didn’t matter as DAT didn’t become widely used. (CDROMs also have a

no-copy bit in the track header but this is almost universally ignored.)

Audio copying has become a headline concern again, thanks to the MP3

format for compressing audio. Previously, digital audio was protected by its

size: a CD full of uncompressed music can take 650 Mb. However, MP3 enables

people to squeeze an audio CD track into a few megabytes, and universal

broadband enables ﬁles of this size to be shared easily. Usage in universities is

particularly heavy; by 1998, some 40% of the network trafﬁc at MIT was MP3

trafﬁc. Some students became underground disc jockeys and relayed audio

streams around campus — without paying royalties to the copyright owners.

The initial response of the industry was to push for technical ﬁxes. This led

to the growth of the rights-management industry. It had its origins in work on

690 Chapter 22 ■Copyright and DRM

digital publishing and in the mechanisms used to protect pay-TV and DVDs,

so let’s look at those ﬁrst.

22.2.4 Video and Pay-TV

The early history of videocassettes was just like that of audio cassettes. At ﬁrst

Hollywood was terriﬁed, and refused to release movies for home viewing.

Again, there were technical measures taken to prevent copying — such as the

Macrovision system which adds spurious synchronization pulses to confuse

the recording circuitry of domestic VCRs — but again these turned out to

be straightforward for technically savvy users to defeat. Then Hollywood

became paranoid about video rental stores, just as book publishers had been

about libraries: but being able to rent videos greatly increased the number of

VCRs and whetted people’s desire to own their favorite movies. VCRs and

videocassettes became mass market products rather than rock stars’ toys, and

now sales of prerecorded cassettes make up most of the income of ﬁrms like

Disney. The business model has changed so that the cinema release is really

just advertising for the sales of the video.

And now that many of the world’s pre-teens demand that their parents

build them a collection of Disney cassettes, just like their friends have, a

videocassette pirate must make the packaging look original. This reduces the

problem to an industrial counterfeiting one. As with mass market software

before the onset of online registration, or with perfumes and Swiss watches

today, enforcement involves sending out ﬁeld agents to buy cassettes, look for

forgeries, trace the supply chain and bring prosecutions.

Much more interesting technical protection mechanisms have been built

into the last few generations of pay-TV equipment.

The advent of pay-TV, whether delivered by cable or satellite, created a

need for conditional access mechanisms which would allow a station operator

to restrict reception of a channel in various ways. If he’d only bought the

rights to screen a movie in Poland, then he’d have to block German or Russian

viewers within the satellite footprint from watching. Porn channel operators

needed to prevent reception in countries like Britain and Ireland with strict

censorship laws. Most operators also wanted to be able to charge extra for

speciﬁc events such as boxing matches.

22.2.4.1 Typical System Architecture

The evolution of early systems was determined largely by the hardware cost of

deciphering video (for a history of set-top boxes, see [293]). The ﬁrst generation

of systems, available since the 1970’s, were crude analog devices which used

22.2 Copyright 691

tricks such as inverting the video signal from time to time, interfering with

the synchronization, and inserting spikes to confuse the TV’s automatic gain

control. They were easy enough to implement, but also easy to defeat; breaking

them didn’t involve cryptanalysis, just an oscilloscope and persistence.

The second generation of systems appeared in the late 1980’s and employed

a hybrid of analog and digital technologies: the broadcast was analogue, the

subscriber control was digital. These included systems such as Videocrypt and

Nagravision, and typically have three components:

a subscription management service at the station enciphers the outgo-

ing video, embeds various entitlement management messages (EMMs) and

entitlement control messages (ECMs) in it, and issues access tokens such as

smartcards to subscribers;

aset-top box converts the cable or satellite signal into one the TV can deal

with. This includes descrambling it;

the subscriber smartcard personalises the device and controls what

programmes the set-top box is allowed to descramble. It does this by

interpreting the ECMs and providing keys to the descrambling circuit in

the set-top box.

This arrangement means that the complex, expensive processes such as bulk

video scrambling can be done in a mass-produced custom chip with a long

product life, while security-critical functions that may need to be replaced in a

hurry after a hack can be sold to the customer in a low-cost token that is easy

to replace. If the set-top box itself had to replaced every time the system was

hacked, the economics would be much less attractive.

The basic mechanism is that the set-top box decodes the ECMs from the

input datastream and passes them to the card. The card deciphers the ECMs to

get both control messages (such as ‘smartcard number 123356, your subscriber

hasn’t paid, stop working until further notice’) and keys, known as control

words, that are passed to the set-top box. The set-top box then uses the

control words to descramble the video and audio streams. There’s a detailed

description in a patent ﬁled in 1991 by NDS [314].

22.2.4.2 Video Scrambling Techniques

Because of the limitations on the chips available at low cost in the early 1990s,

hybrid systems typically scramble video by applying a transposition cipher to

picture elements. A typical scheme was the cut-and-rotate algorithm used in

692 Chapter 22 ■Copyright and DRM

Videocrypt. This scrambles one line of video at a time by cutting it at a

point determined by a control byte and swapping the left and right halves

(Figure 22.1):

Plain

Cut point

tCipher t

Figure 22.1: Cut-and-rotate scrambling

This involved analog-to-digital conversion of the video signal, storage in a

buffer, and digital-to-analog conversion after rotation — a process which could

just about be shoehorned into a low-cost custom VLSI chip by 1990. However,

a systemic vulnerability of such systems is that video is highly redundant, so it

may be possible to reconstruct the image using signal processing techniques.

This was ﬁrst done by Markus Kuhn in 1995 and required the use of a

supercomputer at the University of Erlangen to do in real time. Figure 22.2

shows a frame of enciphered video, and Figure 22.3 the same frame after

processing. By now, it’s possible to do this on a PC [1222]. If this attack had

been feasible earlier, it would have given a complete break of the system, as

regardless of how well the smartcard managed the keys, the video signal could

be retrieved without them. Hybrid systems are still used by some stations,

particularly in less developed countries, together with frequent key changes

to make life inconvenient for the pirates — whose problem is to somehow

distribute the keys to their customers as they crack them.

Figure 22.2: Scrambled video frame Figure 22.3: Processed video frame

22.2 Copyright 693

As for major developed-world operators, they moved to digital systems in

the early 2000s. These digital systems work on the same principle — a set-top

box with the crypto hardware and a smartcard to hold the personal keys that in

turn decipher the content keys from ECMs. However the crypto now typically

uses a block cipher to protect the entire digital video stream. I’ll describe the

current digital video broadcast systems in the next section.

The hybrid scrambling techniques lasted (just) long enough. However,

they have some interesting lessons to teach, as they were subjected to quite

determined attack in decade after 1995, so I’ll spend a page or two going

through what went wrong.

22.2.4.3 Attacks on Hybrid Scrambling Systems

Given a population of set-top boxes that will unscramble broadcast video

given a stream of control words, the next problem is to see to it that only

paying customers can generate the control words. In general, this can be done

with white lists or black lists. But the bandwidth available to last-generation

pay-TV systems was low — typically of the order of ten ECMs per second

could be sent, or a bit over half a million a day. So the blacklist approach was

the main one. With a subscriber base of ﬁve million customers, sending an

individual message to each customer would take over a week.

The basic protocol is that the customer smartcard interprets the ECMs. If

the current programme is one the subscriber is allowed to watch, then a keyed

hash — essentially a message authentication code (MAC) — is computed on a

series of ECMs using a master key held in the card and supplied to the set-top

box as the control word:

CW =MAC(K;ECM1,ECM2,ECM3,ECM4)

So if a subscriber stops paying his subscription, his card can be inactivated

by sending an ECM ordering it to stop issuing control words; and it needs

access to the ECM stream in order to compute the control words at all. So

provided the cards can be made tamper-resistant, only compliant devices

should have access to the master key K, and they should commit suicide on

demand. So what could go wrong?

Well, the ﬁrst attacks were on the protocol. Since the control word sent from

the smartcard to the set-top box is the same for every set-top box currently

unscrambling the program, one person can record the stream of control words,

by placing a PC between the smartcard and the set-top box, and post them

to the Internet. Other people can video-record the scrambled program, and

unscramble it later after downloading the control word stream [850]. Servers

sprung up for this key-log attack exist, but were only a minor nuisance to the

pay-TV industry; not many viewers were prepared to get a special adapter to

connect their PC to their set-top box.

694 Chapter 22 ■Copyright and DRM

Linear feedback shift register 1

↓↓↓

(address)

Multiplexer −→ output

(select)

↑↑↑↑↑

Linear feedback shift register 2

Figure 22.4: The multiplexer generator

Cryptanalysis also gave opportunities to the hackers. Every half-second or

so the smartcard supplies the set-top box with a new control word, and this

is loaded into a keystream generator which works as follows. There are two

linear feedback shift registers, of lengths 31 and 29 in the Eurocrypt system,

which generate long linear sequences. Some of the bits of register 1 are used

as address lines to a multiplexer, which selects a bit from register 2; this bit

becomes the next bit of the keystream sequence. Each successive byte of output

becomes a control byte for the scrambler (Figure 22.4).

The designers intended that breaking this cipher should involve guessing

the key, and as this is 60 bits long a guess would take on average 259 trials

which is uneconomic — as it has to be done about twice a second. But it

turns out that the cipher has a shortcut attack. The trick is to guess the

contents of register 1, use this address information to place bits of the observed

keystream in register 2, and if this causes a clash, reject the current guess for

cryptography.) The effect of this is that as the high order four bits or so of

each control word are easy to deduce from inter-line correlations — it’s the

least signiﬁcant bits you really have to work hard for. So you can easily get

about half the bits from a segment of keystream, and reconstruct the control

word using cryptanalysis. But this computation is still comparable with the

full signal processing attack, and of interest to hobbyists rather than the mass

market.

Other hobbyist attacks included blockers, which would prevent ECMs

addressed to your card from being delivered to it; this way, you could

cancel your subscription without the station operator being able to cancel your

service [850]. Perhaps the most powerful of the ‘amateur’ attacks exploited

a master key leakage: someone bought a second-hand PC, looked out of

curiosity to see if there were any interesting deleted ﬁles on the hard disk,

and managed to undelete a complete subscriber management system for one

pay-TV operator, including embedded master keys. This enabled software to

be written that would completely emulate a subscriber smartcard — in fact, it

could be ‘improved’ in that it would not turn itself off when ordered to do so

by an ECM.

22.2 Copyright 695

Anyway, once this ‘low-hanging fruit’ had been picked, the commercial

pirates turned to reverse engineering smartcards using microprobing tech-

niques. In Chapter 16 I described the arms race between attackers and

defenders. But hardware level ﬁxes were limited to new card issues, and

the operators didn’t want to issue a new card more than once a year as it cost

several dollars per subscriber, and the subscriptions were usually less than

$20 a month. So other defensive techniques had to be found.

Litigation was tried, but it took time. A lawsuit was lost against a pirate in

Ireland, which for a while became a haven from which pirates sold cards by

mail order all over Europe. The industry’s lobbying muscle was deployed to

bring in European law to override Dublin, but this took years. By the middle

of 1995, for example, the main UK satellite TV station (Sky-TV) was losing 5%

of its revenue to pirate cards.

So all through the mid 1990s, pirates and the operators engaged in a war

of countermeasures and counter-countermeasures. The operators would buy

pirate cards, analyze them, and develop all sorts of tricks to cause them to

fail. The problem faced by the operators was this: when all the secrets in your

system are compromised, how can you still ﬁght back against the pirates?

The operators came up with all sorts of cunning tricks. One of their more

effective ones was an ECM whose packet contents were executed as code

by the smartcard; in this way, the existing card base could be upgraded on

the ﬂy and implementation differences between the genuine and pirate cards

could be exploited. Any computation that would give a different answer

on the two platforms — even if only as a result of an unintentional timing

condition — could be fed into the MAC algorithm and used to make the pirate

cards deliver invalid control words.

One of the systems (Eurocrypt) had an efﬁcient revocation scheme designed

in from the start, and it’s worth looking at brieﬂy. Each of the subscriber smart-

cards contains a subscriber key ki, and there is a binary tree of intermediate

group keys KGij linking the subscriber keys to the currently active master key

KM. Each operational card knows all the group keys in the path between it and

the master key, as in Figure 22.5.

In this scheme, if (say) key k2appears in pirate cards and has to be revoked,

then the operator will send out a stream of packets that let all the other

subscriber cards compute a new master key KM. The ﬁrst packet will be

{K

M}KG12 which will let half the subscribers compute K

Mat once; then there will

be a K

Mencrypted under an updated version of KG11: {K

M}KG11; then this new

group key KG11 encrypted under GK22; and so on. The effect is that even with

ten million customers the operator has to transmit less than ﬁfty ECMs in order

to do a complete key change. Of course, this isn’t a complete solution: one

also needs to think about how to deal with pirate cards that contain several

subscriber keys, and hopefully how leaked keys can by identiﬁed without

having to go to the trouble of breaking into pirate cards. But it’s a useful tool

in the countermeasures war.

696 Chapter 22 ■Copyright and DRM

KG12KG11

KG21

KG22

KGn1

• • •

Figure 22.5: Binary revocation tree

Psychological measures were also used. For example, one cable TV station

broadcast a special offer for a free T-shirt, and stopped legitimate viewers from

seeing the 0800 number to call; this got them a list of the pirates’ customers.

Economic factors also matter here, as everywhere. Pay-TV pirates depend for

their success on time-to-market as much as conventional software ﬁrms: a

pirate who could produce a 99% correct forgery in three weeks would wipe

out a competitor who produced a 99.9% forgery after three months. So pirates

race to market just as legitimate vendors do, and pirate cards have bugs just

as Windows does. An understanding of economics helps you exploit them

effectively: it’s best to let a pirate build up a substantial user base before you

pull the plug on him, as this gives him time to wipe out his competitors, and

also as switching off his cards once he’s established will destroy his credibility

with more potential customers than an immediate response would. But if you

leave him too long, he may acquire both the ﬁnancial and technical resources

to become a persistent problem.

The pay-TV industry learned to plan in advance for security recovery, and

to hide a number of features in their products that weren’t used initially

but could be activated later. (As usual, the same lesson had been learned

22.2 Copyright 697

years previously by another industry — in this particular case the banknote

printers.)

Eventually, the smartcards were made much harder to forge by including

proprietary encryption algorithms in the processor hardware. As the attacker

couldn’t simply read out the algorithm with a probing station but had to

reverse engineer thousands of gates in the chip, they reduced to a few dozen

the number of laboratories with the technical capability to do attacks. Many of

these laboratories were drawn into the industry’s orbit by consultancy deals

or other kinds of sponsorship. Those who remained outside the tent, and

appeared to pose a threat, were watched carefully. Vigorous legal enforcement

provided the last link in the chain. The industry hunted down the main

commercial pirates and put them out of business, whether by having them

jailed or by drowning them in litigation.

In the last big pay-TV piracy case in the 20th century, British pirate Chris

Cary was convicted of forging Sky-TV smartcards whose design he had

had reverse engineered by a company in Canada for $105,000. He then sold

forgeries through a front company in Ireland, where counterfeit cards were

not illegal at the time [922]. So Sky TV’s security consultants inﬁltrated a spy

into his Dublin sales ofﬁce, and she quietly photocopied enough documents to

prove that the operation was really being run from the UK [645]. The British

authorities didn’t want to prosecute, so Sky brought a private prosecution and

had him convicted. When he later escaped from jail, Sky’s private detectives

relentlessly hunted him down and eventually caught him in New Zealand,

where he’d ﬂed using a passport in a dead person’s name [575].

So pay-TV history reinforces the lesson that one must make the engineering

and legal aspects of copyright protection work together. Neither is likely to be

adequate on its own.

22.2.4.4 DVB

Digital video broadcasting (DVB) largely operates using a set of standards

that have evolved over the ten years since 1996 and that are controlled by the

DVB Consortium, and industry group of over 250 members. The standards

are many and complex, relating to IPTV and digital terrestrial TV as well as

satellite TV, and to free-to-air services as well as pay-TV.

The protection mechanisms are still a work in progress, and some of them are

covered by nondisclosure agreements, but here is a telegraphic summary. The

conditional access mechanisms for pay-TV are similar to the hybrid system:

the content encryption is digital, but the keys are generated by subscriber

smartcards operating on EMMs and ECMs as before. The encryption uses the

DVB Common Scrambling Algorithm, which is available only under NDA.

The smartcards are not standardised (except at the interface level) so each

broadcaster can use his favorite crypto tricks and suppliers; the piracy to date

698 Chapter 22 ■Copyright and DRM

seems to have involved smartcard cloning, but none of the major systems

appear to have been broken since 2005 (the relentless viciousness shown by

NDS in the Cary case may have had a deterrent effect).

Current standardization work focusses on Content Protection & Copy

Management (CPCM), a set of mechanisms for protecting digital content after

it’s been descrambled and made available within the home. The aim is to

enable set-top boxes and other consumer electronic devices to work together

so that a customer who’s bought a ﬁlm from pay-TV can store it on a personal

video recorder or PC and watch it later at his leisure. (At present, pay-TV

operators that offer this do it by selling a more expensive set-top box that

contains an extra tuner and a built-in PVR). The basic idea is that all the

CPCM-compliant devices in a home will join an ‘authorized domain’ within

which media can be shared and usage information will be logged. This work is

still at the proof-of-concept stage. Established DRM vendors, such as Microsoft,

already have mechanisms whereby protected content can be moved between

compliant devices within their proprietary ecosystem, and there are strong

economic incentives on DRM vendors to keep their systems incompatible, so

as to maximise the lockin and thus the switching costs. There are also incentive

issues with suppliers: how do you stop each individual equipment vendor

from making his protection as weak as he can get away with, so that the

resulting ‘race to the bottom’ undermines protection completely?

A good example of ‘how not to do it’ comes from the world of DVD.

22.2.5 DVD

The consumer electronics industry introduced the digital video disk (DVD),

later renamed the digital versatile disk, in 1996. As usual, Hollywood took

fright and said that unless DVD had a decent copy protection mechanism,

ﬁrst-class movies wouldn’t be released for it. So a mechanism called the content

scrambling system (CSS) was built in at the last minute; arguments over this

held up the launch of DVD and it was designed in a rush. (The story of how

the DVD standards evolved is told in Jim Taylor’s standard reference [1242],

which also describes most of them.)

DVD has region coding: it divides the world into ﬁve regions, and disks are

supposed to run only on players from some designated list of regions. The

goal was to support the traditional practice of releasing a movie in the USA

ﬁrst, then in Europe and so on, in order to minimise the cost of producing

physical ﬁlm prints for use in movie theatres, and the ﬁnancial loss if the ﬁlm

bombs. But a strong implementation of region coding was not in the vendors’

interests; it became clear that users preferred to buy DVD players in which

region coding could be turned off, so the vendors raced to ensure that everyone

knew how to turn it off in their products. Region coding is less important

now, as the Internet has pushed the studios towards near-simultaneous global

22.2 Copyright 699

release of ﬁlms; but the failure of region coding is yet another example of what

happens when the people who have to implement some protection measure

are not the people who suffer the costs of failure.

This left CSS, which was known to be vulnerable by the time that DVD

was launched [1004]. I heard that the designers were told to come up with a

copy protection scheme in two weeks, to use no more than 3,000 gates, and

to limit the keylength to 40 bits so the equipment wouldn’t fall foul of U.S.

expert regulations; another story was that they only ever wanted to compel

DVD player manufacturers to licence the CSS patent, so that makers could

be compelled to implement other copy protection mechanisms as a condition

of the license [193]. No matter whose fault the design was, it’s actually quite

curious that their system held up for three years. The detailed description

of CSS is the subject of much litigation, with numerous injunctions issued in

the USA against web sites that have published it. (In fact the hated Digital

Millennium Copyright Act, under which U.S. actions are often brought, was

introduced in anticipation of the DVD to reinforce its technical protection.)

The ﬂood of legal threats was counterproductive, as Hollywood just got

everybody’s back up. In 1999, a California court enjoined the posting of DeCSS,

one of the CSS decryption programs, but not links pointing to implementations

of it; a New York court added a prohibition of links in a case against the hacker

magazine 2600 the next year. Such injunctions were seen as censorship, and

software to decrypt CSS started appearing on websites outside the USA, on

T-shirts, in songs, and in other forms of speech that more traditionally enjoy

constitutional protection. This just got the software distributed ever more

widely, and made Hollywood look foolish [772]. Their lawyers blundered

on, persuading the government of Norway to prosecute a teenager, Jon

Lech Johansen, who was one of the authors of DeCSS. He released the code

in October 1999; was acquitted on appeal in 2003; and ﬁnally in 2004 the

Norwegian government decided not to attempt a retrial.

Here’s an abbreviated description of CSS1. It is based on a stream cipher

similar to that in Figure 22.4 except that the multiplexer is replaced with a full

adder: each successive keystream bit is obtained by adding together the next

two outputs from the shift registers with carry. Combining the xor operations

of the shift registers with the add-with-carry of the combiner can actually give

a strong cipher if there are (say) ﬁve shift registers with coprime lengths greater

than 70 [1093]; but in CSS there are only two registers, with lengths 17 and 25,

so there is a 216 shortcut attack of exactly the same kind as the one discussed

above. Where the cipher is used to protect keys rather than data, there is a

further mangling step; but this only increases the complexity to 225.

Each player has one or more keys speciﬁc to the manufacturer, and each

DVDdiskhasadiskkeykd encrypted under each of the manufacturer keys

1Lawyers note: this was published in the ﬁrst edition of this book in 2001.

700 Chapter 22 ■Copyright and DRM

(409 of them in 1999) kmi:{kd}km1,{kd}km2,{kd}km3,... ,{kd}km409. There is also a

hash of kd computed by encrypting it with itself: {kd}kd. The actual content is

protected under sector keys derived from kd. Of course, given that the cipher

can be broken with 225 effort, any disk key can be found from a single disk

hash.

The DVD consortium hoped to keep enough of the manufacturer keys

secret by economic pressure: the idea was that if any manufacturer’s master

key got leaked, then it wouldn’t be used on future disks, so his players

wouldn’t be able to play new releases. So manufacturers would implement

decent tamper resistance — or so it was hoped. But the design of CSS doesn’t

support this: given any key in the system, all the others can be found at once.

Also, the economics of mass-produced consumer electronics just don’t allow a

few dollars more for a tamper-resistant processor. In effect, CSS contravened

Kerckhoffs’ principle, in that it depended for its protection on remaining secret,

which was never realistic. I also don’t think it was realistic for the consortium

to think it could blacklist a large electronics ﬁrm, as this would not only have

sparked off vicious litigation; it would also have meant that the millions of

honest consumers who’d bought that company’s products would then ﬁnd

they had to go out and buy a new DVD player. The outcry would have been

too much; had this nuclear button ever been pressed, I expect governments

could have rushed through laws demanding that all DVDs have all master

keys. (I’ll discuss later how the industry hopes to manage revocation better

with the new Blu-Ray standard.)

Another set of problems came from the fact that the PC is an open platform.

The DVD consortium required people producing DVD player software to

obfuscate their code so that it would be hard to reverse engineer. Papers duly

appeared on tricks for systematic software obfuscation [96]. These tricks may

have pushed up the cost of reverse engineering from a few days of effort to a

few weeks, but once the CSS design was out, that was it.

An even more serious problem with came from Linux, the open source PC

operating system used by millions of people. The DVD consortium’s philoso-

phy and architecture were not consistent with making DVD drivers available

to the Linux community. So as PCs with CD drives started being replaced in

the shops with PCs ﬁtted with DVD drives, the Linux user community either

had to break CSS, or give up using Linux in favour of Windows. Under the

circumstances, it was only a matter of time before someone ﬁgured out CSS

and DeCSS appeared.

Anyway, DVD followed the usual pattern: Hollywood terriﬁed, and refusing

to release their best movies; technical measures taken to prevent copying,

which got broken; then litigation. I wrote in 2001: ‘A reasonable person might

hope that once again the studios will see sense in the end, and make a lot of

money from selling DVDs. There will be copying, of course, but it’s not entirely

trivial yet — even a DSL modem takes hours to send a 4Gb DVD movie to a

22.2 Copyright 701

friend, and PC disk space is also an issue’. This has come true; although some

studios held out for a year or two, they all climbed on the DVD bandwagon

within a few years, and Disney now makes most of its money from DVD sales.

Peer-to-peer sharing is an issue, but because of the bandwidth and disk space

constraints it’s less so for ﬁlms than with music.

So will we see the same pattern repeated with high-deﬁnition video? Let’s

look next at the new generation of DVD formats which are being introduced in

the hope of grabbing the market for recordings made for high-deﬁnition TV,

as well as for longer video recordings and larger mass storage of other data.

22.2.6 HD-DVD and Blu-ray

As I was writing this chapter in 2007, a format war was raging between two

proposed successors to DVDs, each backed by a large industrial consortium.

HD-DVD and Blu-ray are in many respects similar; they both use shorter

wavelength lasers to encode information more densely than an old-fashioned

DVD does, so a standard disk will store 25 Gb and a double-layered one 50

Gb. Both of them use a content encryption system called AACS, while Blu-ray

adds an interesting extra mechanism called SPDC. (As I was correcting the

proofs in early 2008, it became clear that Blu-ray had won.)

22.2.6.1 AACS — Broadcast Encryption and Traitor Tracing

The Advanced Access Content System (AACS) has an open design, docu-

mented in [647]. Its design goals included the provision of robust, renewable

protection on both embedded and general-purpose computers so as to limit

output and recording of protected material to approved methods. The encryp-

tion is done using AES.

Key management is based on a broadcast encryption scheme. The basic idea,

due to Li Gong and David Wheeler, is that you can give each user a number

of different keys, chosen from a large pool, and arrange things so that any

two of them will ﬁnd some subset of keys that they have in common [540].

So a large number of people can talk to each other without needing a unique

key for every other user (or public-key certiﬁcates, or a Kerberos server).

Amos Fiat and Moni Naor applied this to pay-TV: the broadcaster gives each

decoder a small set of keys from the pool, and sets up content keys so that each

decoder can compute them using a subset of its keys [469]. When a decoder’s

compromised, its keys are no longer used, and many researchers have worked

on optimising things (so that decoders don’t need to hold too many keys, and

quite a few of them can be revoked before key material has to be updated).

AACS gives each decoder 256 device keys, and information bundled with

the disk tells the decoder which keys to use and how in order to create a

Processing Key. The idea was that each processing key would be used only for

a set of disks. The mechanism is a Media Key Block that tells the decoder how to

702 Chapter 22 ■Copyright and DRM

combine its existing media keys to get the Processing Key. The goal is to be able

to remove single devices, as with the revocation tree in Figure 22.5, although

the details are somewhat different; AACS basically uses a subset-difference

tree that enables each decoder to arrive at the same result (for details, see the

speciﬁcation [647]). When a decoder is revoked, a new MKB can be distributed

which won’t work for that decoder. The processing key in turn protects a

Volume Unique Key (VUK), that in turn protects a title key, that encrypts the

content.

This creates a less catastrophic way to revoke a player whose keys have been

incorporated in unlicensed ripping software. While with a traditional DVD

the studios would have had to revoke all players by that manufacturer, AACS

lets them revoke an individual machine; the broadcast encryption scheme lets

them target one player out of billions, rather than one vendor out of several

hundred.

Has it worked? Well, at present a steady stream of keys are being extracted

from software DVD players, essentially by reading them from memory, and

published. To begin with, these were VUKs, and that was bad enough:

once a VUK is known, anyone can decrypt that disk’s content. But recently

processing keys have been published too. In theory, publishers should have

used many different processing keys, but it now appears that many were

using a single processing key for all disks. When it’s compromised, they

change it, but hackers don’t seem to have too much difﬁculty digging out the

processing keys from DVD player software — and each of these lets people

decrypt all the content published up till the next key change. There has

been the usual circus of lawyers issuing threatening letters, injunctions and

takedown notices to websites where processing keys appear, while people

who don’t like Hollywood have delighted in publishing these keys in all sorts

of innovative ways. The ﬁrst processing key to be published appeared on

T-shirts and registered in domain names; it appeared on Digg.com, and when

the administrators removed it, Digg’s users revolted and forced them to climb

down. It’s been d´

eja vu all over again.

So broadcast encryption, although a neat idea, doesn’t seem to have been

enough on its own to stop people decrypting movies.

There are features in AACS that don’t seem to have been used yet. For

example, there’s a traitor tracing scheme. The idea behind traitor tracing is

that you add some unique marks to each copy, so if decrypted content fetches

up on a peer-to-peer network, the studios know who to sue. Such methods

have been used with the electronic distribution of ﬁlms to movie theatres,

where the goal is to identify theatres that let the pirates record new ﬁlms on

camcorders. Another application is in the Oscars, where Academy members

are given ‘screeners’ — DVDs of candidate ﬁlms. These used to leak regularly;

but in 2004, after the studios started to individualize them, the actor Carmine

Caridi was ordered to pay $300,000 after forensics identiﬁed him as the source

22.2 Copyright 703

of leaked screeners that ended up providing masters for illegal copies in 21

countries [1250]. Since then, Academy members have been more careful.

With a small distribution, it’s possible to mark copies at production, but this

is not feasible when mass-producing disks. A method is therefore needed to

add uniqueness during the decryption process. How AACS supports this is

as follows. Each cell (time segment) is usually decrypted by a title key, but

issuers can insert a sequence key cell — of which there will be up to eight, each

encrypted with a different segment key, and there can be up to 32 sequence key

cells in a disk. The key management mechanism is arranged so that different

players will derive different sets of segment keys. So if decrypted content

appears somewhere, the content owner can see which segment keys were used

and from that work out which player was used. The actual marking of each

segment might be something overt from the production process, such as the

color of shirt an actor’s wearing, or more likely a robust hidden mark (which

I’ll describe later). However, the traitor-tracing mechanisms in AACS do not

seem to have been deployed so far.

22.2.6.2 Blu-ray and SPDC

So the content protection in HD-DVD was starting to look slightly shaky, which

helped the competing format, Blu-ray, as it also supports a novel protection

mechanism called Self-Protecting Digital Content (SPDC).

SPDC is a very neat idea that tries to tackle the underlying incentive

problem — that while it’s the studios that want to protect the content, it’s the

equipment makers that provide the protection. The innovation is that each

player contains a virtual machine that can run content-protection code that’s

unique to each title. The studios write this code and can change it from one

disk to the next, or even between versions of the same disk. This gives the

studios a number of options for responding to a compromise, and they can do

so directly rather than having to work through trade associations, standards

bodies or the vendors themselves.

If a particular player is seen to be leaking content, they can blacklist it

in future discs; if a particular type of player is leaking, they can arrange

that it only outputs low-resolution video. When it comes to tracing they can

do their own; content can be marked after decoding. This not only avoids

relying on the AACS mechanisms to identify compromised players or player

types, but is potentially much more ﬂexible as marks can be embedded much

more pervasively in audio and video (I’ll discuss marking technology below).

For these reasons, SPDC provides more resilient protection than the key-

management and device-revocation mechanisms in AACS alone; it’s described

in an evaluation report [637]. As of the beginning of 2008, the studios have been

struggling with teething problems with movies using SPDC, but have decided

to favor it; it now looks like SPDC has helped Blu-ray win the format war.

704 Chapter 22 ■Copyright and DRM

There are two engineering challenges facing rights-management engineers.

First, people increasingly expect to move content they’ve purchased from one

device to another. If you buy and download the latest recording of Mahler’s

ﬁfth, you expect to play it not just on your laptop and iPod, but also on your

home audio system; and if you buy and download a baseball game, you’ll want

to see it on your wide-screen TV. Second, as you make content, and the keys

that protect it, available on more and more platforms, so the risk of leakage

rises — and the incentives can potentially drift ever more out of alignment.

In any case, as more and more digital content is distributed online using

DRM, we need to look at that next.

22.3 General Platforms

In the mid-1990s, a number of researchers started work on ways to control

the sale and distribution of digital goods over the Internet to customers with

personal computers. The original applications included the distribution of

newspapers and articles from scientiﬁc journals [221], although it was always

understood that music and video would follow once networks had enough

bandwidth.

The basic problem is that a PC, being a general-purpose computer, can in

principle copy any ﬁle and send it to any other computer; unlike with analogue

copying, copies are perfect, so millions of copies might be made from one orig-

inal. The problem is compounded by the fact that, from the viewpoint of the

content vendor, the PC owner is the ‘enemy’. The music industry believed that

unlimited copying would destroy their business; the computer industry told

them that DRM was intrinsically impossible on a general-purpose computer,

so they’d better get a new business model. The music and ﬁlm industries,

despite being a tenth of the computer industry’s size, had much more clout in

Congress (a Microsoft guy told me this was because the average Congressman

was much keener to be photographed with Madonna than with Bill). So Hol-

lywood got its way. The result is a number of products generally referred to

as digital rights management or DRM.

Curiously, despite the acrimony of the argument between computer people

and Hollywood in the 1990s about whether DRM was possible or desirable,

it now seems that both of them may have been taking the wrong side in the

argument! Stronger DRM has turned out to beneﬁt the platform vendors, such

as Apple, more than the content companies (and economists had predicted

this). And although all DRM mechanisms seem to get broken sooner or later,

the existence of a modest bar to copying does have effects on the business.

But as download sites move to selling unprotected MP3s, it’s not clear that it’s

strictly necessary. However, I’ll come back to the policy issues later. First let’s

look at the most common DRM mechanism, Windows Media Player.

22.3 General Platforms 705

22.3.1 Windows Media Rights Management

At the time of writing, Windows Media Player (WMP) comes bundled with

every copy of Windows sold on general-purpose PCs. This will change, as

the European Court has found the bundling to be an anticompetitive practice

and ordered Microsoft to make Windows available without it. However,

people will still be able to download it for free, and given the large number

of websites that use Windows Media ﬁle formats, it is bound to remain an

important platform.

WMP replaced an earlier media player application when Windows 98 was

released. It enables a user to play music, watch video and view photos, and has

all sorts of features from MP3 player support to synchronisation of lyrics for

Karaoke. However our main interest here is its ability to play ﬁles protected

using Windows Media Rights Management (WMRM). This works as follows.

A store wanting to sell digital media encrypts each item using a content

key and puts the encrypted ﬁles on a streaming media server that is typically

linked to their web site. In order to access a media object, the customer must

get hold of a license, which consists of the object identiﬁer, the license key

seed, and a set of instructions in a rights management language which state what

she can do with it; how many times she may play it, whether she’s allowed

to burn it to a CD, and so on. The license is generated by a license server

and encrypted using her public key. The license acquisition may involve her

going to a registration or payment page, or it may happen silently in the

background [1049].

In order to use this system, the customer must ﬁrst personalize her media

player, which is done automatically the ﬁrst time she accesses protected

content. The player generates a public/private keypair (WMP uses elliptic

curve cryptography) and the public key is sent to the license server.

The architecture is very similar to pay-TV conditional access, in that the

bulk encryption task of protecting the music or video is separated from the

personal task of key management, so the video doesn’t have to be encrypted

anew for each customer. And just as pay-TV smartcards can be replaced when

keys are leaked or the key management mechanism compromised, so the

key management functions of WMRM are performed in an ‘individualized

blackbox’ (IBX) component of the software, which gets replaced as needed.

The IBX internals are not documented publicly by Microsoft, but according

to a description by people who reverse-engineered WMP version 2 in 2001, the

basic components are elliptic curve cryptography for public-key encryption;

DES, RC4 and a proprietary cipher for symmetric crypto; and SHA-1 for

hashing [1141]. The customer’s private key is obscured by the blackbox and

hidden in a ﬁle. Licenses the customer has previously acquired are kept in

a license store; when a new object is encountered, the software looks up the

object ID and key ID, and if a license isn’t already stored for them, it sends a

706 Chapter 22 ■Copyright and DRM

session key encrypted under the license server’s public key. The session key is

used to encrypt the license that’s sent back. Once the client software has used

the session key to decrypt the license, it’s added to the license store. Stored

licenses have a further layer of encryption, presumably to stop people reading

clear keys or manipulating permissions: the content key is encrypted using the

customer’s public key, and there’s also a digital signature. By now, of course,

the protocol may have been tweaked — as Microsoft has had to recover several

times from hacks, and because WMP has now acquired many more features.

WMRM is used at its most basic to provide a streaming media service,

to make it slightly harder for people to record music and video from news

sites and Internet radio. (Vendors seem to have given up on audio, as it’s

very easy to record audio after decryption as it’s passed to the sound card;

but Microsoft is making a serious effort in Vista to stop people grabbing

video in this way [570].) More sophisticated uses of WMRM include music

subscription services, where you can download as many songs as you wish,

but they will all become unplayable if you stop paying; and geographically-

linked services, such as MLB.com which makes Major League baseball games

available everywhere except in the team’s home area — for which the rights

have usually been sold for megabucks to local TV stations.

22.3.2 Other Online Rights-Management Systems

The Microsoft offering is fairly typical of rights-management systems. Apple’s

FairPlay, which is used in the iPod and in its media player QuickTime, also has

tunes encrypted under master keys, and when a tune is bought the user is sent

the master key encrypted under a random session key, plus the session key

encrypted under his iTunes player’s RSA public key. Session keys are backed

up online on Apple’s servers. As with Windows, a number of programs

appeared that unlocked protected content, and Apple duly upgraded iTunes

to stop these programs working in September 2006.

In the mid-2000s there were growing calls from consumer and industry

groups for DRM systems to become more interoperable. Real Networks is the

other major supplier of media player software for PCs, and its RealPlayer uses

its own proprietary DRM. In a tussle reminiscent of the word-processing ﬁle-

format wars of the 1980s, Real made their music tracks readable by iTunes in

2004; Apple responded by threatening litigation, and ended the compatibility

in a 2006 upgrade. This should surprise no-one: recall the discussion in

section 7.3.2 of how the value of a software company depends on the total

lock-in of its customers. In 2007, Steve Jobs of Apple called on the music

industry to sell music without DRM; this was surprising, given the incentives

facing Apple. However, the gamble seems to be paying off, in that some music

labels are now starting to make unprotected music available, but at higher

prices and not on a large enough scale to threaten Apple’s lock-in of its iTunes

22.3 General Platforms 707

customer base. Apple’s revenues continue to soar, while the music majors see

sales tumbling.

The Open Mobile Alliance (OMA) is a consortium that promotes standards

for interoperable DRM on mobile devices. Its OMA DRM Version 1.0 is widely

used to protect mobile phone ringtones; it mostly consists of a simple rights

management language whose most important feature is ‘forward lock’: a

way of marking content so that the phone knows it must not forward the

object to any other device. Download protection is typically using standard

TLS mechanisms. Version 2.0 is similar in its expressive power to WMRM or

FairPlay, but is still not widely used. Network operators have objected to the

idea that anyone else should have direct access to their customer base.

22.3.3 Peer-to-Peer Systems

Peer-to-peer ﬁle-sharing has become one of the main ways in which music

is distributed online. After Napster’s initial music-sharing service was closed

down by music industry lawyers, systems such as Gnutella and Freenet

borrowed ideas from the world of censorship-resistant systems to set up

networks with no central node that could be closed down using legal attacks.

I was the designer of an early censorship-resistant system, the Eternity Ser-

vice. I had been alarmed when an early anonymous remailer, anon.penet.fi,

was closed down following legal action brought by the Scientologists [590]. It

had been used to post a message that upset them. This contained an afﬁdavit

by a former minister of their church, the gist of which was reported to be

an allegation that once members had been fully initiated they were told that

the rest of the human race was suffering from false consciousness; that, in

reality, Jesus was the bad guy and Lucifer was the good guy. Well, history has

many examples of religions that denounced their competitors as both deluded

and wicked; the Scientologists’ innovation was to claim that the details were

their copyright. They were successful in bringing lawsuits in a number of

jurisdictions.

The Eternity Service was designed to provide long-term ﬁle storage by

distributing ﬁle fragments across the net, encrypted so that the people host-

ing them would not be able to tell which fragments they had, and so that

reconstruction could only be performed via remailer mechanisms [41]. A later

version of this was Publius2, which also provided a censor-resistant anony-

mous publishing mechanism [1309].

In 1999, Shawn Fanning, a 18-year-old drop-out, revolutionised the music

business by creating the Napster service, which enabled people to share MP3

2For non-U.S. readers: the revolutionaries Alexander Hamilton, John Jay, and James Madison

used the pen name Publius when they wrote the Federalist Papers, a collection of 85 articles

published in New York State newspapers in 1787–8 and which helped convince New York voters

to ratify the United States constitution.

708 Chapter 22 ■Copyright and DRM

audio ﬁles with each other [931]. Rather than keeping the ﬁles centrally, which

would invite legal action, Napster just provided an index so that someone

wanting a given track can ﬁnd out who else has got it and is prepared to share or

trade. It attracted tens of millions of users, and then lawsuits from Hollywood

that closed it down in 2001. The gap that it left behind was promptly ﬁlled

by peer-to-peer networks such as Gnutella and Freenet [297] that were in turn

inspired by the earlier censorship resistant systems. These were followed by

commercial systems such as Kazaa and eMule, which were also targeted by

music industry lawyers (and many of which acquired a reputation for being

full of spyware).

The United States Copyright Ofﬁce deﬁnes peer-to-peer networks as net-

works where computers are linked to one another directly rather than through

a central server. The absence of a server that can be closed down by court

order creates an interesting problem for music industry enforcers. They have

tried persuasion, by playing up stories of people who’d carelessly set a P2P

program to share all their hard disk, rather than just their music ﬁles. But the

two tactics on which the music industry has relied are suing uploaders and

technical attacks on the systems.

In section 21.5 I explained how social networks are often vulnerable to

decapitation attack, as they rely for their connectivity on a small number of

particularly well-connected nodes. Taking down these nodes will disconnect

the network. Similarly, peer-to-peer networks have well-connected nodes, and

in ﬁlesharing systems there are also nodes whose owners contribute more than

their share of the uploaded content. The music industry has for some years

been targeting key nodes for legal action, having ﬁled over 20,000 lawsuits

since 2003. In many cases people agree to cease and desist and pay a small

penalty rather than ﬁght a case; but in October 2007 a federal jury in Duluth,

MN., convicted 30-year-old Jammie Thomas of copyright infringement for

sharing material on Kazaa and ordered her to pay $9,250 for each of the 24

songs involved in the case.

On the technical-countermeasures side, it was long known that ﬁrms work-

ing for the music industry were uploading damaged music ﬁles to spam out

systems (which will usually be legal), and it was suspected that they were

also conducting denial-of-service attacks (which in many jurisdictions isn’t).

But there was no evidence. Then in September 2007, a company called Media

Defender that worked for the music industry on ‘ﬁle-sharing mitigation’ suf-

fered the embarrassment of having several thousand of its internal emails

leaked, after an employee forwarded his email to Gmail and his password was

compromised. The emails not only disclosed many details of the ‘mitigation’

tactics, but also revealed that the company worked closely with the New York

Attorney General’s ofﬁce — potentially embarrassing in view of the industry’s

possibly illegal attacks on computers and invasions of privacy. It turned out

that Media Defender’s business model was to charge $4,000 per album per

22.3 General Platforms 709

month, and $2,000 per track per month, for ‘protection’ that involved attacks

on twelve million users of ﬁfteen P2P networks [1009].

Peer-to-peer systems have also allegedly been attacked by Comcast, which is

said to have disrupted its customers’ use of BitTorrent by sending forged reset

packets to tear down connections. Comcast might prefer its customers to watch

TV over its cable network, so they see its ads, rather than via ﬁlesharing; but

the allegations raise some public policy issues if true: BitTorrent partners with

content owners such as Fox and MGM, while Comcast is not a law-enforcement

agency [155]. In any case, this harassment of ﬁle sharers is fuelling an arms

race; the proportion of BitTorrent trafﬁc that’s encrypted rose from 4% to 40%

during 2006–7 [1346].

22.3.4 Rights Management of Semiconductor IP

Another live problem is the protection of designs licensed for use in semi-

conductors. Companies like ARM make their living by designing processors

and other components that they sell to ﬁrms making custom chips, whether

by designing application-speciﬁc integrated circuits (ASICs) or by using Field-

Programmable Gate Arrays (FPGAs).

The ﬁrst problem is overrun production. A camera company licenses a

circuit that they integrate into a bitstream that’s loaded into an FPGA, that

then becomes a key component in a new camera that they have made in a

factory in China. They pay for 100,000 licenses, yet 200,000 cameras arrive

on the market. There are two failure modes: the camera company could have

ordered the extra production and lied to the IP owner, or the Chinese factory

could be cheating the camera company. In fact, they could both be cheating,

each having decided to make an extra 50,000 units. Now there are technical

mechanisms that the camera company could use, such as personalising each

camera with a serial number and so on after manufacture — but these could

make it harder to cheat.

The second problem is knowing when a product contains a particular

circuit. The camera company might have licensed a processor, or a ﬁlter, for

one model, then built it into another cheaper model too without declaring it.

These risks cause a partial market failure, in that large IP vendors often prefer

to license their best designs only to other large ﬁrms that they trust, so small

startups can ﬁnd it difﬁcult to compete on equal terms [1348]. They also depress

sales of FPGAs, whose manufacturers would dearly love a rights management

system for design IP. The best that’s been done so far are mechanisms to

tackle the ﬁrst problem by distributing encrypted bitstreams and updates for

whole chips; the second problem is a lot harder, because the chip design

tools would be squarely within the trust boundary. Customers would need

to be able to evaluate designs, and debug designs, while maintaining some

control on dissemination. At present, the best that can often be done is to

710 Chapter 22 ■Copyright and DRM

use side-channels for forensics. Owners of semiconductor IP can buy up

samples of suspect goods, operate them, and observe the chips’ precise power

consumption, electromagnetic emissions, thermal signature and so on, which

can often reveal the presence of a given functional component.

This brings us to the question of copyright marking, which has blossomed

into a large and complex research area.

22.4 Information Hiding

Hollywood’s interest in ﬁnding new mechanisms for protecting copyright

came together in the mid-1990’s with the military’s interest in unobtrusive

communications and public concerns over government efforts to control cryp-

tography, and started to drive rapid developments in the ﬁeld of information

hiding. This largely refers to techniques which enable data to be hidden in

other data, such as when a secret message is hidden in an MP3 audio ﬁle,

or a program’s serial number is embedded in the order in which certain

instructions are executed.

The Hollywood interest is in copyright marks which can be hidden unob-

trusively in digital audio, video and artwork. These are generally either

watermarks, which are hidden copyright messages, or ﬁngerprints which are

hidden serial numbers. For example, when you download an MP3 ﬁle from

Apple’s iTunes music store, it contains a ﬁngerprint embedded in the audio

that identiﬁes you. The idea is that if you then upload your copy to a ﬁle-

sharing system, the copyright owner can sue you. (This isn’t universal: some

people believe that ﬁngerprinting depresses sales overall because of the legal

hazards it creates for honest purchasers. Amazon, for example, does not mark

MP3 downloads [577].)

The privacy interest is in steganography whose purpose is to embed a

message in some cover medium in such a way that its very existence remains

undetectable. The conceptual model, proposed by Gus Simmons [1169, 1176],

is as follows. Alice and Bob are in jail and wish to hatch an escape plan; all their

communications pass through the warden, Willie; and if Willie detects any

encrypted messages, he will frustrate their plan by throwing them into solitary

conﬁnement. So they must ﬁnd some way of hiding their secret messages

in an innocuous-looking covertext. As in the related ﬁeld of cryptography,

we assume that the mechanism in use is known to the warden, and so the

security must depend solely on a secret key that Alice and Bob have somehow

managed to share.

There is some similarity with electronic warfare. First, if steganography is

seen as a low-probability-of-intercept communication, then copyright marking

is like the related jam-resistant communication technique: it may use much

the same methods but in order to resist focussed attacks it is likely to have

a much lower bit rate. We can think of Willie as the pirate who tries to

22.4 Information Hiding 711

mangle the audio or video signal in such a way as to cause the copyright mark

detector to fail. Second, techniques such as direct sequence spread spectrum

that were originally developed for electronic warfare are ﬁnding wide use in

the information hiding community.

Of course, copyright marks don’t have to be hidden to be effective. Some

TV stations embed their logo in a visible but unobtrusive manner in the

corner of the picture, and many DRM systems have control tags bundled quite

visibly with the content. In many cases this will be the appropriate technology.

However, in what follows we’ll concentrate on hidden copyright marks.

22.4.1 Watermarks and Copy Generation Management

The DVD consortium became concerned that digital video or audio could be

decoded to analog format and then redistributed (the so-called ‘analog hole’).

They set out to invent a copy generation management system that would work

even with analog signals. The idea was that a video or music track might be

unmarked, or marked ‘never copy’, or marked ‘copy once only’; compliant

players would not record a video marked ‘never copy’ and when recording one

marked ‘copy once only’ would change its mark to ‘never copy’. Commercially

sold videos would be marked ‘never copy’, while TV broadcasts and similar

material would be marked ‘copy once only’. In this way, the DVD players

available to consumers would allow unlimited copying of home videos and

time-shifted viewing of TV programmes, but could not easily be abused for

commercial piracy.

The proposed mechanisms depended on hiding one or more copyright

marks in the content, and are reviewed in [193, 800]. For each disk, choose a

ticket X, which can be a random number, plus copy control information, plus

possibly some information unique to the physical medium such as the wobble

in the lead-in track. Use a one-way hash function hto compute h(X)andthen

h(h(X)). Embed h(h(X)) in the video as a hidden copyright mark. See to it that

compliant machines look for a watermark, and if they ﬁnd one will refuse to

play a track unless they are supplied with h(X) which they check by hashing

it and comparing it with the mark. Finally, arrange things so that a compliant

device will only record a marked track if given X, in which case only a h(X)

is written to the new disc. In this way, a ‘copy once only’ track in the original

medium becomes a ‘copy no more’ track in the new medium. DVD-audio uses

such a marking mechanism; SDMI also uses a fragile watermark that’s damaged

by unauthorised processing. (I’ll discuss mark removal techniques in the next

section.)

The main use of marking with video content will be if Blu-ray wins the

standards war with HD-DVD. Then the SPDC processor will be able to embed

unique ﬁngerprints into decrypted content in real time. Each plaintext copy of

a video derived from a decoder can be unique, and the studios can use forensic

techniques to determine which decoder it came from. Unless the decoder itself

712 Chapter 22 ■Copyright and DRM

is compromised, the marking mechanism can give a much more resilient way

of identifying subscribers who’re leaking content. This raises a number of

questions. First, how well does marking work? Second, how well does it scale?

And third, what about the policy aspects?

Quality brings us back to our old friend, the ROC. It’s not enough for a

marking mechanism, whether used for copy management or for traitor tracing,

to have a low missed alarm rate; it needs a low false alarm rate [892] too. If

your legitimate DVD player were to detect a ‘no-copy’ mark in your wedding

video by mistake, then you’d have to buy a pirate player to watch it. So what

sort of marks are possible, and how robust are they against forgery, spooﬁng

and other attacks?

22.4.2 General Information Hiding Techniques

Information hiding goes back even further than cryptology, having its roots

in camouﬂage. Herodotus records tricks used during the wars between the

Greeks and the Persians, including hiding a message in the belly of a hare

carried by a hunter, tattooing it on the shaven head of a slave whose hair was

then allowed to grow back, and writing it on the wooden base under the wax

of a writing tablet [595]. Francis Bacon proposed a system which embedded

a binary message in a book at one bit per letter by alternating between two

different fonts [1016]. Until quite modern times, most writers considered hid-

ing conﬁdential information much more important than enciphering it [1345].

Military organizations still largely hold this view and have used all sorts

of technologies from the microdots used by spies in much of the twentieth

century to low-probability-of-intercept radios.

When it comes to hiding data in other data, the modern terminology

of the subject is as follows [1023]. The copyright mark, or in the case of

steganography, the embedded text, is hidden in the cover-text producing the

marked text or in the case of steganography the stego-text. In most cases,

additional secret information is used during this process; this is the marking

key or stego-key, and some function of it is typically needed to recover the mark

or embedded text. Here, the word ‘text’ can be replaced by ‘audio’, ‘video’ and

so on, as appropriate.

A wide variety of embedding schemes has been proposed.

Many people have proposed hiding mark or secret message in the least

signiﬁcant bits of an audio or video signal. This isn’t usually a very good

strategy, as the hidden data is easy to detect statistically (the least sig-

niﬁcant bits are no longer correlated with the rest of the image), and it’s

trivial to remove or replace. It’s also severely damaged by lossy com-

pression techniques.

22.4 Information Hiding 713

A better technique is to hide the mark at a location determined by a

secret key. This was ﬁrst invented in classical China. The sender and

receiver had copies of a paper mask with holes cut out of it at random

locations. The sender would place his mask over a blank sheet of paper,

write his message in the holes, then remove it and compose a cover mes-

sage including the characters of the secret embedded message. This trick

was reinvented in the 16th century by the Italian mathematician Car-

dan and is now known to cryptographers as the Cardan grille [676].

A modern version of this hides a mark in a .gif format image as

follows. A secret key is expanded to a keystream which selects an appro-

priate number of pixels. The embedded message is the parity of the

color codes for these pixels. In practice even a quite large number of

the pixels in an image can have their color changed to that of a simi-

lar one in the palette without any visible effects [654]. However, if all

the pixels are tweaked in this way, then again the hidden data is easy

to remove by just tweaking them again. A better result is obtained if

the cover image and embedding method are such that (say) only 10%

of the pixels can safely be tweaked. Then, if the warden repeats the

process but with a different key, a different 10% of the pixels will be

tweaked and only 10% of the bits of the hidden data will be corrupted.

In general, the introduction of noise or distortion — as happens with

lossy compression — will introduce errors into the hidden data almost

regardless of the embedding method unless some kind of error correct-

ing code is added. A system proposed for banknote marking, Patch-

work, uses a repetition code — the key selects two subsets of pixels,

one of which is marked by increasing the luminosity and the other by

decreasing it. This embeds a single bit; the note is either watermarked

using that key, or it isn’t [160, 562]. You can think of this as like dif-

ferential power analysis: the key tells you how to sort your input data

into two piles, and if the key was right they’re noticeably different.

In the general case, one may want to embed more than one bit, and have

the embedded data to survive very high levels of induced errors. So a

common technique is to use direct-sequence spread-spectrum techniques

borrowed from electronic warfare [1251]. You have a number of secret

sequences, each coding a particular symbol, and you add one of them to

the content to mark it.

Spread spectrum encoding is often done in a transform space to make its

effects less perceptible and more robust against common forms of com-

pression. These techniques are also commonly used in conjunction with

perceptual ﬁltering, which emphasises the encoding in the noisiest or

714 Chapter 22 ■Copyright and DRM

perceptually most signiﬁcant parts of the image or music track, where it

will be least obtrusive, and de-emphasises it in quiet passages of music

or large expanses of color [208].

Some schemes use the characteristics of particular media, such as a

scheme for marking print media by moving text lines up or down by a

three-hundredth of an inch [221], or adding extra echoes to music below

the threshold of perception [160]. So far, such techniques don’t seem to

have become as robust, or as widely used, as generic techniques based

on keyed embedding using transform spaces, spread spectrum and per-

ceptual ﬁltering.

Progress in copyright marking and steganography was very rapid in the

late 1990s: people invented marking schemes which other people broke, until

eventually the technology became more mature and robust.

22.4.3 Attacks on Copyright Marking Schemes

Throughout this book, we’ve seen attacks on cryptographic systems that occa-

sionally involved cryptanalysis but more often relied on mistaken assumptions,

protecting the wrong things, protocol failures and implementation bugs. And

in the history of technology as a whole, inventions tend to end up being used

to solve problems somewhat different from the problems the inventor was

originally thinking about. Copyright marking has been no different on either

count.

In the beginning, many people assumed that the main market would

be embedding hidden copyright messages so that ownership of a work

could be proved in court. This was mistaken. Intellectual property

lawyers almost never have any difﬁculty in proving ownership of an

exhibit; they don’t rely on technical measures which might confuse a

jury, but on documents such as contracts with bands and model release

forms.

As usual, many designers ignored Kerckhoffs’ principle — that the secu-

rity of a system should reside in the choice of key, not in the algorithm in

use. But this principle applies with greater than usual force when marks

are to be used in evidence, as this means disclosing them in court. In fact,

as even the marking keys may need to be disclosed, it may be necessary

to protect objects with multiple marks. For example, one can have a mark

with a secret key that is system-wide and which serves to identify which

customer re-sold protected content in violation of his licence, and a sec-

ond mark with a unique key that can be disclosed in court when he’s

prosecuted.

22.4 Information Hiding 715

Many marks are simply additive. This opens a whole series of possible

vulnerabilities. For example, if all the frames in a video carry the same

mark, you can average them to get the mark and then subtract it out. An

even simpler attack is to supply some known content to a marking sys-

tem, and compare its input and output. Even if this isn’t possible — say

the mark is applied in a tamper-resistant processor immediately after

decryption and every device adds a different mark — then if the mark

consists of small signals added at discrete points in the content, an oppo-

nent can just decrypt the content with several different devices and com-

pare them to remove the marks.

There have been various attempts to develop a marking equivalent of

public key cryptography, so that (for example) anyone could insert

a mark which only one principal could detect, or anyone could detect a

mark that only one principal could have inserted. The former seems just

about feasible if the mark can be inserted as the cover audio or video

is being manufactured [334]. The latter is the case of particular interest

to Hollywood. However, it seems a lot harder than it looks as there is a

very general attack. Given a device that will detect the mark, an attacker

can remove a mark by applying small changes to the image until the

decoder cannot ﬁnd it anymore [1015, 803]. Hence the drive to have

mechanisms that enable you to put a different mark in each instance of

the content, as the content is decrypted.

Some neat steganalysis techniques were developed to break particular

embedding schemes. For example, where the mark was added by either

increasing or decreasing the luminosity of the image by a small ﬁxed

amount, the caused the peaks in the luminosity graph to become twin

peaks, which meant that the mark could be ﬁltered out over much of

many images [826].

The ﬁrst large vendor of marking systems — Digimarc — set up a ser-

vice to track intellectual property on the web. They supplied tools to

let picture owners embed invisible ﬁngerprints, but their initial imple-

mentation could be easily defeated by guessing the master password,

or by modifying the marking software so that it would overwrite exist-

ing marks. They also had a ‘Marc spider’, a bot which crawled the web

looking for marked pictures and reporting them to the copyright owner,

but there were many ways to defeat this. For example, the typical web

browser when presented with a series of graphics images will display

them one after another without any gaps; so a marked image can often

be chopped up into smaller images which will together look just like the

original when displayed on a web page but in which a copyright mark

won’t be detected (Figure 22.6) [1019].

716 Chapter 22 ■Copyright and DRM

Figure 22.6: The Mosaic attack (courtesy Jet Photographic, www.jetphotographic

.com)

Digimarc has now ﬁxed the bugs and concentrates on monitoring broad-

cast streams; this enables advertisers, for example, to check whether the

ads they’ve paid for have actually gone out. But they’ve found that a

larger business is to move digital technology into the world of secu-

rity printing: they put watermarks in ID documents to prevent them

being copied, and licensed their marking technology to central banks

as a counterfeit detection measure. For example, it’s found in the new

Euro banknotes, which it prevents from being scanned or copied using

the latest equipment [1373]. Software packages such as Photoshop and

Paintshop Pro now refuse to handle marked images.

However, the most general known attacks on copyright marking

schemes involve suitably chosen distortions. Audio marks can be re-

moved by randomly duplicating or deleting sound samples to introduce

inaudible jitter; techniques used for click removal and resampling are

also powerful mark removers. For images, a tool my students devel-

oped, called Stirmark, introduces the same kind of errors into an image

as printing it on a high quality printer and then scanning it again with a

high quality scanner. It applies a minor geometric distortion: the image

is slightly stretched, sheared, shifted, and/or rotated by an unnoticeable

random amount (see Figure 22.7). This defeated almost all the mark-

ing schemes in existence when it was developed and is now a standard

benchmark for copyright mark robustness [1019]. In general, it’s not

22.4 Information Hiding 717

clear how to design marking schemes that will resist a chosen distor-

tion attack in which the attacker who understands the marking scheme

mangles the content in such a way as to cause maximum damage to

the mark while doing minimal damage to the marked content.

For a fuller account of attacks on copyright marking schemes, see [1019,

1020]. The technology’s improving slowly but the limiting factor appears to be

the difﬁculty of designing marking schemes that remain robust once the mark

detection algorithm is known. If any copy control scheme based on marking

is implemented in PC software or low-cost tamper-resistant processors, it’s

only a matter of time before the algorithm gets out; then expect to see people

writing quite effective unmarking software.

(a) Picture of Lena (b) Lena after Stirmark

Figure 22.7: The effect of Stirmark

718 Chapter 22 ■Copyright and DRM

The security-by-obscurity issue has led to at least one political row. The

Digimarc software used in commercial graphics packages to prevent them

loading images of currency is only available under NDA. Thus proposed

legislation to compel its use will place open-source graphics packages such as

Gimp in danger.

22.4.4 Applications of Copyright Marking Schemes

The applications of marking techniques are much broader than just DVDs and

banknotes. Color copiers sold in the U.S. have hidden their serial number in

the bit patterns of copies, for example, as an extra means of detecting currency

forgers [1331]. Another technique is to embed fragile marks in still pictures as

they are taken, so that alterations are readily detected by forensic labs [769].

A further class of proposed applications have to do with convenience or

safety rather than preventing malicious behaviour. It’s been proposed that

music broadcast over the radio should be marked with the CD’s number

so that someone who likes it could order the CD automatically by pressing

a button. And in medicine, digital versions of X-rays often get separated

from the patient’s details, as the various proprietary ﬁle formats get mangled

through numerous protocol conversions; this safety problem might be solved

by embedding patient details directly in the image.

Finally, a fair proportion of information hiding research doesn’t aim at

Hollywood’s requirements, or those of the Bureau of Engraving and Printing,

but at hiding information for privacy purposes. I’ll discuss such applications

in the next chapter.

22.5 Policy

The IP policy debate got heated in the 1990s are early 2000s, as a series of laws

from copyright term extension to America’s Digital Millennium Copyright Act

(DMCA) shifted power to the owners of ‘intellectual property’ — copyrights,

patents and trademarks — in ways that many people in the computer industry

and elsewhere felt to be threatening. Stricter copyright enforcement impinges

on free-speech rights, for example: the law used to provide ‘fair use’ or ‘fair

dealing’ exemptions that enable you to parody or criticise a work by someone

else, but these exemptions weren’t respected in all countries during the rush

to update copyright laws. In addition, the DMCA gives copyright owners the

power (‘Notice and Take Down’) to compel ISPs to take down websites with

infringing material. Although there is also a provision (‘Notice and Put Back’)

for the subscriber to ﬁle a counter notice and have his stuff put back within

14 days unless the copyright owner ﬁles suit, in practice many ISPs will just

terminate a customer’s service rather than get involved in litigation.

22.5 Policy 719

There are continuing concerns about the effect of DMCA on libraries,

especially as more and more material becomes electronic: the legal controls that

allowed, for example, library lending are being replaced by technical controls

that don’t [730]. For example, when I applied for planning permission to extend

my kitchen, I had to ﬁle four copies of a local plan; and the map software at

our university library only lets you print three copies. This is of course quite

deliberate. Legal controls are supplemented by access controls, and the legal

privilege given to those access controls by the DMCA creates a new bundle

of de-facto rights, criticised by many legal scholars as ‘paracopyright’ [364]. In

effect, copyright regulations are no longer made by lawmakers in Washington

or Brussels, but on the hoof by programmers working for Microsoft or Apple.

The result, according to copyright law critics such as Larry Lessig and Pamela

Samuelson, has been to greatly decrease the rights of copyright users.

At the same time, copyright law has suddenly become relevant to millions

of people. Whereas in the past it was only a concern of specialists such as

publishers, it now touches the lives of everyone who downloads music, time-

shifts movies using a TiVo, or maintains a personal web page. As the law has

failed to keep up with technology, the gap between what it permits and what

people actually do has become wider. In the UK, for example, it’s technically

illegal to rip a CD on to an iPod, yet as this is one of the main reasons that

people buy CDs nowadays, the British Phonographic Industry (the trade body)

graciously says it won’t sue anybody. The law-norm gap can only become

wider as we move increasingly to a remix culture, and as the many minor

infringements that used to take place in private, or undocumented public,

spaces (such as singing a song in a pub) go online (as when a phone video clip

of the song gets on someone’s social-network page). John Tehranian calculates

that a typical law professor commits over 80 copyright infringements a day,

carrying statutory penalties of over $10 m [1243].

On the other side of the coin, there are the privacy concerns. In the old

days, people would buy a book or a record for cash; the move to down-

loads means that DRM license servers run by ﬁrms such as Microsoft and

Apple have a record of what people watch and listen to, and this can be

subpoena’ed. (It’s equally true that the move to online bookselling has created

similar records at Amazon.) These records are also used for marketing. A

survey for the Privacy Commissioner of Canada found many examples of

intrusive behavior, including e-book software proﬁling individuals, Double-

Click advertising in a library service, systems tracking individuals via IP

addresses, and contradictions between vendors’ started privacy policies and

observed behaviour — including undisclosed communications to third par-

ties [468]. Not one of the organisations whose products and services were

tested complied with requests from the testers to disclose personal infor-

mation held about them. It looks inevitable that such DRM systems break

European privacy law too, as it’s based on the same principles. A particularly

720 Chapter 22 ■Copyright and DRM

extreme case arose in 2005 when Microsoft decided that Sony’s XCP system

was ‘spyware’ and and ‘rootkit’, and would thus be removed by Windows

Defender and the Malcicious Software Removal Tool [884]. So next time you

hear a large company bellyaching about how its users break the law by copying

its music or its software, bear in mind that it may well be a lawbreaker too.

Where will it all lead? Until recently, the copyright law debate was a

straight ﬁght between corporate lobbyists on the one hand, and activists such

as academics and the free software community on the other; the latter had the

arguments, but the former always got their way with legislatures. This has

now changed, and in two interesting ways. First, the IP lobby started to run

out of steam, and second, Hollywood started to realise that stronger DRM was

likely to shift power away from content owners towards the platform vendors.

22.5.1 The IP Lobby

First, the IP lobby. This has its modern origins in an effort by the drug company

Pﬁzer to extend patent protection on its drugs from the USA to less developed

countries like Brazil and India in the 1970s. The story is told in a history by Peter

Drahos and John Braithwaite [398]; in summary, Pﬁzer and the other drug

companies allied themselves with the music and ﬁlm industry (who wanted

to cut bootlegging and copying), the luxury-goods industry (who wanted to

reduce the number of cheap knock-offs), and a number of other U.S. players

(including, it should be said, the Business Software Alliance), and persuaded

the U.S. government to start applying pressure to less developed countries

to bring their patent, copyright and trade-mark laws in line with America’s.

From the mid-1980s onwards this was largely a matter of bilateral action, but

in 1994 the a treaty on Trade-Relates Aspects of Intellectual Property Rights

(TRIPS) was signed, followed by two treaties of the World Intellectual Property

Organisation (WIPO) in 1996. Essentially the USA and the EU got together

and bullied holdouts like India and Brazil.

The implementation of these treaties stirred up a lot of opposition in devel-

oped countries as people began to realise how they might be affected. In the

USA, the Digital Millennium Copyright Act of 1998 made it an offence to

circumvent a copyright-protection mechanism, as required by WIPO, while

in the European Union the Copyright Directive of 2001 had a similar effect.

This was seen as enabling vendors to create closed platforms and control

competition; it was also seen as a threat by the free and open source soft-

ware movement, and by security researchers — especially after the Russian

researcher Dmitri Sklyarov was arrested at a U.S. conference at the request of

Adobe after his employer had sold tools circumventing password protection

on pdf documents.

There were many other high-proﬁle incidents; for example, I was on the

program committee of the 2001 Information Hiding Workshop when an

22.5 Policy 721

attempt was made by the Recording Industry Association of America (RIAA)

to force the program chair to pull a paper by Ed Felten and his students

describing vulnerabilities in a copyright marking scheme being touted for a

digital music standard [335]. This led to a lawsuit in which Ed sued RIAA,

which became a landmark academic-freedom case [424]. The irony is that the

promoters of this challenge had issued a public challenge to academics and

others to break their scheme. The next case was Bunnie Huang’s book ‘Hacking

the Xbox’: this described in detail how, as an MIT student, he’d overcome the

protection mechanisms in the ﬁrst version of Microsoft’s games console [629].

The book he wrote caused his publisher to take fright, but he found another

one and the publicity can’t have done his sales any harm. In any case, the

encroachment on liberties threatened by rights-management mechanisms and

anti-hacking laws led to the growth of digital rights NGOs in a number of

countries (others had them already as a result of the ‘Crypto Wars’; I’ll discuss

all this in more detail in Part III).

The turning point appears to have come in 2003–4, as the IP lobby was trying

to steer a further measure through Brussels, the IP Enforcement Directive. This

would have further ratcheted up the penalties on infringers and removed the

prospects for public-interest defences based on free speech or fair use. This

time opponents of the measure managed to assemble a sufﬁciently strong

coalition of interests opposed to stronger IP enforcement that the measure was

substantially amended. By then there were the beginnings of decent economic

arguments, such as the ﬁrst result mentioned in section 7.5.5 that downloading

doesn’t actually harm music sales; and meanwhile the IP lobby had seriously

overreached.

For example, the IP folks tried to compel every country in Europe to make

patent infringement a crime, rather than just a civil matter. This was designed

by the leading drugs companies to undermine ﬁrms who manufacture and

sell generic versions of drugs once they have come off patent. At present, drug

companies try to prolong their patents by ‘evergreening’ — ﬁling subsidiary,

later patents, with often dubious derivative claims — which the generic drug-

makers deal with by offering their distributors indemnities against having to

pay damages. Making infringement a criminal matter would have upset these

arrangements. This caused the generic drugmakers to oppose the directive

vigorously, along with supermarkets, car parts dealers and consumer groups.

Even the software industry started to get nervous: we pointed out to Microsoft

that thousands of companies believe that Microsoft is infringing their patents,

but don’t have the money to go the distance in a civil court. If patent infringe-

ment became a crime, surely they would take their grievances to the police?

Would Bill risk arrest on some future trip to Europe?

With hindsight, this was bound to be the eventual fate of the IP movement.

A rich, powerful lobby isn’t stopped by ﬁne words, or by outrage from

university professors and free-software activists (such as the words of John

722 Chapter 22 ■Copyright and DRM

Perry Barlow at the head of this chapter). It’s stopped when it comes up against

another rich, powerful lobby pushing in the opposite direction. That point now

appears to have been passed; for example, in June 2007 WIPO abandoned the

attempt by the IP lobby to conclude a new treaty on broadcasting.

The IP movement have have passed its high water mark, but it is not quite

dead yet; following the election of Nicholas Sarkozy as President of France,

they persuaded him to back a law that would make ISPs responsible for

less competitive [115]. It will be interesting to see whether French ISPs — or

the European Commission — manage to push back on this one; even though

the tide is no longer ﬂowing in the IP lobby’s direction, there are bound to be

a lot of skirmishes like this before the law ﬁnally stabilises.

There are some other problems with copyright that people will have to

worry about eventually. Some copyright activists have assumed that once

under a Creative Commons license — then everything will be hunky-dory.

I doubt it. Curating old bits costs money, just as curating old manuscripts

does; indeed the ﬁlm industry has recently discovered that archiving digital

productions actually costs more than they used to pay in the old days, when

they just locked away the master copies in an old salt mine. There’s just

an awful lot of bits generated during digital production, and copying them

to new disks every few years isn’t cheap. In the long term, once bitstrings

belong to nobody, who will pay for their upkeep? Some lawyers would like

to extend copyright term indeﬁnitely, but that violates the social contract on

which copyright is based and it also doesn’t solve the problem properly: many

publishers have failed to look after their own back catalogue properly and had

to retrieve copies from national deposit collections. When NGO colleagues

and I considered this problem the best solution we could come up with was a

digital preservation law plus a taxpayer-funded digital deposit library [418].

22.5.2 Who Beneﬁts?

As I discussed in section 7.5.5, a further important development came in 2005.

In January of that year, Google’s chief economist Hal Varian addressed a

DRM conference in Berlin and asked who would beneﬁt from stronger DRM.

He pointed out that, in classical economic theory, a technical link between

two industries would usually beneﬁt the more concentrated industry (for

example, car makers and car parts). Now the platform industry is concentrated

(Apple, Microsoft) while the music industry is less so (four majors and many

independents): so why should the music industry expect to be the winners from

better DRM? Economic theory says that platform vendors should win more.

The music industry scoffed, and yet by the end of that year they were hurt-

ing — lobbying governments and the European Commission to ‘do something’

22.6 Accessory Control 723

about Apple, such as forcing it to open its FairPlay DRM scheme. It’s now

becoming clear that music downloading — with or without DRM — is chang-

ing the structure and dynamics of the music industry. Bands used to rely on

the majors to promote them, but now they can do that themselves by giving

away their albums on their websites; they always made most of their money

from performances, and now they make more than ever.

In the ﬁrst edition of this book, I predicted that big bands might follow the

Linux business model: ‘it may make sense to give the ‘‘product’’ away free

and make money on the ‘‘maintenance’’ (tours, T-shirts, fan club ...)’, while

for more specialised acts ‘the trick may be slightly keener pricing and/or

packaging that appeals to the collector’. So far, that was called right. I also

wrote ‘I also expect that Hollywood will follow the software industry and

adopt a somewhat more mature attitude to copying. After all, 70% of a market

worth $100 billion is better than 98% of a market worth $50 billion. And just as

a certain amount of copying helped market software, it can help music sales

too: the Grateful Dead encouraged bootleg taping because they had learned it

didn’t harm their sales’. On that one, the current state of play is that CD sales

are declining steadily, and while download sales are starting to rise, they’re

not rising fast enough yet to compensate. We shall have to wait and see.

22.6 Accessory Control

There was concern in the late 1990s and early 2000s that the prohibitions

against reverse engineering introduced at Hollywood’s request via the DMCA

would have evil effects on competition. Indeed, one of the most important and

rapidly-growing uses of cryptographic mechanisms and of rights-management

technology generally since the DMCA was passed is in accessory control. The

familiar example is the printer cartridge.

The practice started in 1996 with the Xerox N24 (see [1221] for the history

of cartridge chips). In a typical system, if the printer senses a third-party

cartridge, or a reﬁlled cartridge, it may silently downgrade from 1200 dpi to

300 dpi, or even refuse to work at all. In 2003, expiry dates and ink usage

controls were added: cartridges for the HP BusinessJet 2200C expire after being

in the printer for 30 months, or 4.5 years after manufacture [827]; and modern

cartridges now limit the amount of ink dispensed electronically rather than

waiting for it to run out physically. The latest development is region coding:

you can’t use U.S. ink cartidges in a recently UK-purchased HP printer.

All this has caused grumbling; the European Parliament approved a ‘Direc-

tive on waste electrical and electronic equipment’ designed to force member

states to outlaw the circumvention of EU recycling rules by companies who

design products with chips to ensure that they cannot be recycled [230, 332].

But by the time this was translated into actual regulation, the authorities had

724 Chapter 22 ■Copyright and DRM

relented. The printer companies lobbied hard and the regulations eventually

said that so long as the vendors accept empty cartridges back for disposal,

they will have discharged their obligations to the environment [1323], while

the UK government has said that as cartridges are consumables, the law won’t

apply [1097].

In the USA, the matter was decided in court. The printer maker Lexmark

sued SCC, which (as I recounted in the chapter on tamper resistance at 16.5.1)

had reverse-engineered their print-cartridge crypto, alleging violation of the

Digital Millennium Copyright Act. Although they won at ﬁrst instance, they

lost on appeal in 2004 [790]. In a similar case, Chamberlain (who make garage

door openers) sued Skylink (who made compatible openers) and also lost,

losing the appeal too in 2004. This appears to settle U.S. law in favour of a free

market for cryptologists, which was always the position before the DMCA

came along [1110]. A ﬁrm wanting to control its aftermarket using crypto chips

is free to hire the smartest cryptographers it can ﬁnd to build authentication

chips that are really hard to hack, and its competitors are free to hire the

smartest cryptanalysts they can ﬁnd to try to reverse-engineer them. Other

ﬁelds in which crypto is used for accessory control include games console

software and add-ons, and mobile phone batteries3.

Other industries are eyeing up the technology. For example, the carmaker

Volkswagen makes a range of models at different prices, and this is reﬂected

in the price of spares; you can pay $12 for an air ﬁlter for a Skoda, $40 for

the same air ﬁlter for a Volkswagen, and over $100 for the same air ﬁlter for

an Audi. Will we in future see cars authenticating their major spare parts,

to prevent arbitrage? (If the motor industry’s smart it will be marketed as a

‘safety measure’ to stop unsafe foreign parts — this is the line Motorola took

when it introduced authentication chips into phone batteries.)

Is accessory control objectionable? The economist Hal Varian analyses it as

follows [1286]:

The answer depends on how competitive the markets are. Take the

inkjet printer market. If cartridges have a high proﬁt margin but

the market for printers is competitive, competition will push down

the price of printers to compensate for the high-priced cartridges.

Restricting after-purchase use makes the monopoly in cartridges

stronger (since it inhibits reﬁlls), but that just makes sellers compete

more intensely to sell printers, leading to lower prices in that

3There are limits to the freedom to reverse engineer. In the UK, a man was convicted in October

2007 of selling modchips that allowed people to run programs on games consoles without their

being signed by the vendor. This relies on a 2003 law implementing the EU Copyright Directive

that made Member States prohibit the sale of devices designed to circumvent copy protection. He

is appealing [879]. However, a court in Helsinki made a possibly conﬂicting ruling [1216] — so

it’s not clear that we have a consistent law in Europe.

Research Problems 725

market. This is just the old story of ‘‘give away the razor and sell

the blades.’’

However, in many other industries it might be anticompetitive; it just

depends on how concentrated the industry is, and in winner-take-all platform

markets it could be particularly objectionable [53]. So it’s a good job that early

fears of a legal prohibition against reverse engineering for compatibility have

proved largely unfounded. Of course, the right to reverse engineer is not the

same as the right to succeed at reverse engineering, and so there may be

cases in the future — as shrinking feature sizes and growing complexity make

reverse engineering harder — where ﬁrms locked out of a market will have to

use antitrust law to get access.

22.7 Summary

The technical protection of digital content against unauthorised copying is

a difﬁcult problem both technically and politically. It’s difﬁcult technically

because general-purpose computers can copy bitstrings at no cost, and it’s dif-

ﬁcult politically because Hollywood’s attempts to impose rights-management

technology on a reluctant computer industry have done a lot of collateral

damage. Some of problems people agonised about — including the effects on

reverse engineering for compatibility — have failed to materialise, but there

is still real complexity. At the technical level, compatibility between DRM

systems is a big issue, and it’s not at all clear how DRM will work in the home

of the future where people will expect to play music and videos that they’ve

bought on multiple devices. At the political level, it appears that the music

industry has brought down grief on itself by insisting on stronger DRM, as this

simply shifted power in the supply chain from itself to the platform vendors

(and speciﬁcally Apple). It remains to be seen whether the same will happen to

Hollywood: will the move to high-deﬁnition video result in Microsoft stealing

their lunch? At least there’s one strategy for them to ﬁght back — by using the

SPDC mechanisms to assume part of the role of platform owner.

At any rate, the development of copyright and rights-management systems

over the last ten years has been a fascinating if bumpy ride, and its evolution

over the next ten will surely be too.

Research Problems

There are many interesting research problems in copyright management. Some

of them we’ve already touched on, such as how to build cheaper tamper-proof

hardware tokens, and better ways of embedding copyright marks in digital

726 Chapter 22 ■Copyright and DRM

pictures and sound. But a huge amount of research was done on these topics

from the mid-90s until about 2005; there are many competing proposals, and

lawyers are ﬁghting through patent thickets. Novel ideas would surely be

of interest to many stakeholders, but the low-hanging fruit may have been

plucked by now.

Further Reading

Software copy protection techniques are discussed at length in [561]; there’s a

brief history of technical protection mechanisms for audio and video in [487];

and a racy account of the coevolution of attack and defense in pay-TV systems

in [850]. More information about pay-TV, and the available information on

DVD, can be found at various web sites (which may move because of legal

harassment), while there’s a lawyer’s view at [565].

There is an overview of information hiding techniques, including steganog-

raphy and information hiding, in a special issue of the Proceedings of the

IEEE [822]; for attacks on marking schemes in particular, see [1019, 1020]. For

more detail there’s a recent book on the subject [697]. Kahn is, as usual, good

historical background reading [676]. A useful guided tour of U.S. copyright law

is by Gordon [546]. Ongoing research work can be found in the proceedings of

the workshops on information hiding [42, 97, 1022]. And ﬁnally, no chapter on

the subject [784, 785] and Pam Samuelson’s writings [1107, 1108, 1109, 1110].

CHAPTER

The Bleeding Edge

What information consumes is rather obvious: it consumes the attention of its

recipients. Hence a wealth of information creates a poverty of attention, and a need

to allocate that attention efﬁciently among the overabundance of information

sources that might consume it.

— Herb Simon

Voting machine software is a special case because the biggest danger to security

comes from the people who are supposed to be responsible for it.

— Richard Stallman

23.1 Introduction

Our security group at Cambridge runs a blog, www.lightbluetouchpaper.org,

where we discuss the latest hacks and cracks. We even found some vulner-

abilities in the Wordpress blog software we use and reported them to the

maintainers. But we weren’t alone in ﬁnding ﬂaws, and in October 2007,

the blog itself was compromised by a Russian script kiddie who tried to put

on some drug ads. The attack itself was only an inconvenience, as we spotted

it quickly and recovered from backup, but it brought home how dependent

we’ve all become on a vast range of applications that we just don’t have time

to evaluate. And the blog posts themselves show that many of the attacks, and

much of the cutting-edge work in security research, hinge on speciﬁc applica-

tions. There will still be exploits against platforms like Windows and Symbian,

but there are many more vulnerabilities out there in apps. As Microsoft cleans

up its act, and as search engines make it easier to ﬁnd machines running

speciﬁc apps, that’s where the action may well shift.

727

728 Chapter 23 ■The Bleeding Edge

In the case of blog software, the Wordpress security engineering was not

very impressive, but its competitors are even worse; and this one application

alone exposes thousands of machines to compromise. There are many, many

applications, and their developers usually don’t care about security until they

get hacked. The same learning process that Microsoft’s gone through since

2000 will be repeated in one domain after another. But not all applications

are the same; while some (like blog software) simply open up PCs to botnet

recruitment, there are others from which money can be extracted directly,

others that people rely on for privacy, and others that mediate power.

I’ve already discussed a number of more or less ‘embedded’ apps, from

banking through alarms to prepayment meters. In this chapter I’m going to

brieﬂy describe four types of application that make up the bleeding edge of

security research. They are where we ﬁnd innovative attacks, novel protection

problems, and thorny policy issues. They are: online games; web applications

such as auction, social networking and search; privacy technologies such as

anonymizing proxies; and, ﬁnally, electronic elections.

Games and Web 2.0 highlight the fact that the real ‘killer application’ of the

Internet is other people. As more people come online in ever more contexts,

we’re creating complex socio-technical systems of a kind that never existed

before. That means quite novel attacks, exploits, tussles and disputes. We’ve

already seen several examples, such as click fraud and impression spam, as

well as new variants on old scams.

Anonymity systems follow naturally: if you want to reap some of the beneﬁts

of web applications but not end up exposing your privacy to them, you may

wish to play under a pseudonym. Mechanisms to do this have been discussed

for decades and real systems have emerged; but as you’d expect from our

discussion of inference control in Chapter 9, anonymity is much harder than it

looks.

Finally, elections are a classic example of an application where anonymity

is required, but coupled with accountability: you want voters in an election to

be able to satisfy themselves that their vote was counted, yet not to be able

to prove to anyone else who they voted for (so they can’t be bribed or bullied).

Elections are also the key interface between social computing and power.

23.2 Computer Games

Games were one of the ﬁrst applications of all — pretty well as soon as the

world’s ﬁrst proper computer, the EDSAC, was operational, research students

were writing games for it. The early AI researchers started writing chess

programs, believing that when this problem was solved, computers would be

able to function more or less as people. And in my own spotty youth, the ﬁrst

23.2 Computer Games 729

cryptanalysis program I wrote was to let me peek ahead into the rooms of a

dungeon game.

There are limited opportunities for cheating at games of perfect information

like chess, especially when playing against a computer. But there are many

ways to cheat in other games, and they’re becoming a big deal. Millions of

people play online games, many of them bright and a lot of them poor; the

large online worlds have a turnover larger than some small countries; and

thousands of people make a living from online games, from the developers

who create and maintain them to Chinese gold farmers. So the motives for

cheating exist; and as games are software, and software has bugs, the means

for cheating exist too. Yet if cheating becomes pervasive it spoils the fun. People

don’t enjoy an unfair ﬁght — and in the long run even successful cheaters get

bored (unless cheating and counter-cheating become the new game). Even the

perception of cheating destroys players’ enjoyment, so they go elsewhere and

the game vendor loses a ton of money. So vendors make a serious effort to stop

it. All told, online games provide a ﬁrst-class social laboratory for the study of

hacking, and game security has become the subject of serious study.

Computer games are also big business, as they have been for decades.

They drove the home-computer boom of the 1970s that in turn spawned the

PC industry; games consoles have been a huge market for microprocessors

and memory chips; and gaming — whether on consoles or PCs — has largely

driven the development of computer graphics [1367]. By 2001, game sales

in the USA hit $9.4 billion, outperforming movie box-ofﬁce sales of $8.35

billion. Comparing the two industries isn’t straightforward, as movie stars

have other sources of income too, and the industries are getting entangled

with more and more movies being made with computer graphics effects. But

in order-of-magnitude terms, computer games are probably of comparable

economic importance to movies. Certainly a blockbuster online game grosses

much more nowadays than a blockbuster movie; as games go online, you’re

selling subscriptions, not just one-off tickets [203].

‘Security’ in games has meant different things down through the years.

The early arcade games of the 1970s concentrated on protecting the coin

box against robbers. When Nintendo moved console games into the home,

they subsidised the consoles from later sales of software cartridges and other

add-ons, so a lot of effort was put into controlling which accessories could

be used, as I discussed in section 22.6; their later competitors from Sega to

Sony and Microsoft ended up ﬁghting both legal and technical battles against

reverse-engineers. Copy-protection of game software for PCs has also been a

big deal, and there have been pre-release leaks of standalone games, just like

prerelease leaks of movies. However the move to online computer games has

trimmed the concerns. As a critical part of the game logic runs on a server, the

client software can be given away, and the residual issue is whether players

can get an unfair advantage. That’s what I’ll focus on now.

730 Chapter 23 ■The Bleeding Edge

23.2.1 Types of Cheating

There are basically three types of cheating.

The ﬁrst is where the original game has a known vulnerability that goes

across into the online world and may be made worse. For example, a hand

of contract bridge starts with players taking turns to bid how many tricks

they think they can take. Each hand has four players, in two teams, and

during the bidding no-one may communicate any information about the cards

in their hand to their partner other than their public bids. In competitive play,

a screen is placed diagonally across the table during bidding so that no-one

can see their partner. Yet there are still occasional suspicions that some covert

communication has taken place, for example in the players’ tone of voice.

In the real world, allegations of cheating are heard by a jury of experienced

players, who take a view on whether the outcome was better than could have

been expected in honest play. Even so, some decisions remain controversial

for years: players may be exceptionally skilful, or lucky, and partners who’ve

played together for years may communicate subconsciously without trying to.

Bridge is an example of two much more general problems, namely exploiting

game rules, and cheating by collusion, both of which existed long before

computers. Moving to online play creates both an opportunity and a problem.

The opportunity is that if players are honest, all the bids can be mediated

through the system: there’s no tone of voice to give rise to a dispute. The

problem is that if four people are playing online bridge together from their own

homes, then there’s nothing to stop a pair setting up a separate communications

channel — a single text message of the cards in a hand is all it takes. Can

technology help? Well, online bridge means online records, so you can mine

the records of large numbers of games to determine whether a suspect pair

do better over the long run than they should. It also makes it easier to run

tournaments where each match is played simultaneously by many players

using the same deal of cards — which makes a cheat easier to spot. Finally,

it facilitates new ways of organising play: if you have an online game server

available 24 by 7, people can turn up singly to play and start a game whenever

four have arrived. So people play with many partners rather than just one; both

the motive to cheat and the means are reduced, while the risks are increased.

Where’s there’s a single forum, such as a single dominant server, you

can also construct global controls. For example, a problem in some games is

escaping, where someone who’s losing simply drops the connection. With a

global service, you can remove the incentive for this by recording an escape as

a loss (but only so long as your service is reliable — some game servers end

a quarter of sessions in crashes, and this strategy wouldn’t be popular there).

Other exploits that are also found in real-world games include pretending to

be less skilled than you are, losing a few games, but then winning once the

other player gets conﬁdent and plays for more money; pool-room and poker

23.2 Computer Games 731

sharks have used such strategies for ages, and there are many team variants

too. In some games, you can get an advantage by having multiple players, and

get some to kill off others to accumulate points. The susceptibility of online

games to this sort of rigging depends to a great extent on whether identity

is durable. If people can easily create, or cheaply buy, new identities, then

rigging games using multiple front identities becomes simpler. (This is known

as a Sybil attack and is also a problem in peer-to-peer systems [396].) One way

to deal with this is a reputation system — a topic to which I’ll return when we

discuss auctions. In others, rather than having many characters operated by

one human player, you use the reverse trick of having one character operated

by shifts of successive human operators; this is typically done in online

mulitplayer games where the goal is to accumulate online time and points.

The second type of cheating is where known computer-security issues apply

straight off to the world of gaming. Five percent of the badware measured

by Symantec in the ﬁrst half of 2007 was aimed at online games, with the

two most common items being Trojans designed to steal account information

from players of Gampass and Lineage [1239]. There’s a great variety of other

attacks, from straightforward phishing to denial-of-service attacks that push

up the network latency of your opponent so you can beat him at blitz chess. A

lot of the material in this book applies one way or another to gaming: cheaters

hack servers, eavesdrop on communications, and tamper with client memory

to make walls invisible (there’s a survey and taxonomy at [1368]). Policy issues

such as privacy turn out to matter here too: a lot of people in Second Life

were annoyed when an enterprising company built a search engine that went

through their homes and indexed their stuff. And just as in other applications,

a lot of exploits arise because of the chance discovery of bugs — such as in

one game where you could drive up to a wall in a jeep, hit the ‘disembark’

button, and appear instantly on the other side of the wall [203]. Many games

have glitches of this kind, and knowledge of them spreads quickly.

The third type are the new cheating tactics that emerge because of the nature

of computer games, and the online variety in particular. In tactical shooters,

for example, success should depend on the player’s tactics and shooting skill,

not on the game mechanics. Yet there are always shortcomings in the game’s

physics model, often introduced by network latency and by the optimisations

game designers use to deal with it. In effect, the developers try to deceive you

into believing that their world is consistent with itself and with Newton’s laws,

when it isn’t really. For example, you’d normally expect that in a shooting

duel, you’d have an advantage if you have the lowest network latency, or if

you move ﬁrst. Yet the prediction algorithms used in many game clients can

twist this or into an exclusive-or: a high-latency player has an advantage if he

moves ﬁrst. This is because clients cache information about nearby players,

so if you leap round a corner, see your enemy and shoot, then the slower

732 Chapter 23 ■The Bleeding Edge

your network connection is, the longer it will take before he can see you and

respond. (There’s a wide range of such tactics: see [203] for a discussion.)

There are many interesting borderline cases where an activity can be skill

or cheating depending on the circumstances. Mike Bond coined the term ‘neo-

tactic’ to refer to players subliminally exploiting network effects, such as the

latency advantage for ﬁrst movers. Are neotactics genius, or cheating? As in

Bridge, the one can easily be mistaken for the other. But most people would

say it’s deﬁnitely cheating if you use mechanical assistance, such as a proxy

server that slows down your packet stream just before you launch an attack.

23.2.2 Aimbots and Other Unauthorized Software

That brings us on to one of the classic game cheats, namely bots. One of the

persistent cheating strategies is to have code of your own to provide you with

automation and support. People have written a huge variety of tools, from

simple routines that repeatedly click a ﬁre button (to hack the games where the

rate at which you can physically ﬁre is a factor) through proxies that intercept

the incoming network packets, identify the bad guys, examine your outgoing

shots, and optimise their aim. These aimbots come with different levels of

sophistication, from code that does all the target acquisition and shooting, to

human-controlled versions that merely improve your aim. They can hook into

the packet stream as proxies, into the graphics card, or even into the client

code. Another variant on the same theme is the wall hack, where a player

modiﬁes his software to see through walls — for example, by changing the

graphics software to make them translucent rather than opaque.

Game companies who sell ﬁrst-person shooters reckon that aimbots seri-

ously spoil other players’ fun, so they use encryption and authentication

mechanisms to protect the packet stream. (These are usually proprietary and

hackable but that will no doubt get ﬁxed in time.) They also use guard software,

such as Punkbuster, that uses anti-virus techniques to detect attempts to hook

into game code or the drivers on which it relies. A recent innovation, found in

Call of Duty 4, is to offer action replays of kills, seen from the viewpoint of the

player who makes the kill: this enables the killed player to see whether he was

killed ‘fairly’ or not. This may not only reduce cheating, but also the perception

of cheating — which is almost as damaging to the game operator [204].

Inappropriate software can also be run on game servers. A common hack is

to get an object such as a gun to run multiple copies of a script in parallel — a

trick that could have been prevented by proper resource accounting at the

server. However, servers face critical real-time demands and their designers

try to keep them as simple as possible. Self-replicating objects are used

to run service-denial attacks, or to create temporary buildings that escape

the resource controls on permanent structures. And people program magic

swords to do unexpected tricks.

23.2 Computer Games 733

It must be said, though, that the relatively uncontrolled game scripting

languages which make this possible have also been turned to creative use.

People have realised that game engines can be used to render whole movies

in 3-d; the quality may not be as good as on Pixar’s rendering farms, but it

works, and it’s essentially free. This has led to the growth of a whole new art

form of machinima (machine cinema). As they say, it’s a rare wind that blows

nobody any good.

23.2.3 Virtual Worlds, Virtual Economies

Bots are also used in farming, where entrepreneurs do the boring legwork

of accumulating game objects such as gold coins or magic swords for sale to

impatient players. However, most of the farming is done by real people in low-

wage countries from Romania to China. ‘Gold farming’ is now a signiﬁcant

export that’s creating new economic opportunities for young people in remote

villages that have few other employers [384]. The economy of a large online

community, such as World of Warcraft — with 8 million subscribers, of whom

half a million are online at any time — is larger than that of some countries.

This means in turn that macroeconomic effects, such as exchange rates and

rents, start to matter. Second Life, for example, is essentially a 3-d chat room

run by Linden Labs, which rents out space to third parties, who can then cus-

tomise their property as they want. It has a local currency of Linden dollars,

that can be bought for U.S. dollars using a credit card, either through Linden

Labs or via third-party brokers. The currency enables in-game entrepreneurs

to sell value-added items such as clothes, artwork, pornography and services.

After the FBI cracked down on online casinos, there was a surge of interest in

gambling in Second Life; so in April 2007 the Feds visited Linden Labs [1006].

Just before the visit, 26% of announcements were about gambling; after it,

commercial rents fell. And the world of anti-money-laundering controls made

its appearance when Linden Labs started discriminating between ‘veriﬁed’

and ‘unveriﬁed’ accounts (the former being those where the player had used

a credit card to subscribe)1.

Markets for game goods are also getting better organised. For several years,

magic swords and gold coins were traded in grey markets on eBay, but starting

with Sony’s ‘Station Exchange’ in 2005, game operators began running proper

auction sites where players can trade game goods for real money. In 2006, we

had reports of the ﬁrst serious fraud: crooks used stolen identities to set up

hundreds of thousands of accounts on the South Korean game Lineage, with

1The Financial Action Task Force — an international body that bullies countries into ask-

ing people who open bank accounts to provide government-issue photo-ID and two utility

bills — wants payment systems that don’t participate in their ’identity circus’ to impose limits

on payment amounts and velocity. That’s why accounts at PayPal or even African mobile-

phone payment systems restrict what you can do if you’re unveriﬁed.

734 Chapter 23 ■The Bleeding Edge

allegedly some inside help, and cashed out by selling some $15 m in game

goods [758]. This all helps to raise the stakes in gaming, and to make stability

more important. It also brings us naturally to eBay and to other ‘real-world’

web applications.

23.3 Web Applications

While online computer games are partly implemented in servers and partly

in client software, an increasing number of services are entirely online and

accessed via a standard web browser. They range from auction services like

eBay through search engines such as Google and Yahoo, online mail services

like Hotmail and AOL, online word processors such as Google documents,

and e-commerce sites selling all kinds of good things (and bad things). Some

industries — travel, entertainment, insurance and bookselling — have moved

much of their sales online. And the recent trend to social networking brings in

all sorts of new angles.

There are many problems common to all manner of web sites. One is

that web servers are often insufﬁciently careful about the input they accept

from users, leading for example to the SQL insertion attacks I discussed in

section 4.4.2. Another increasingly common vulnerability is cross-site scripting

(XSS). Scripting languages such as javascript are supposed to observe a same

origin policy in that scripts will only act on data from the same domain; you

don’t want a script from a Maﬁa-run porn site acting on your electronic banking

data. But this policy has been repeatedly circumvented, exploiting everything

from carelessly-written web services that let users upload html containing

scripts that other users can then read and execute, to errors in the design of

virtual machines — such as the Firefox javascript bug discussed in Chapter 18.

Web services as diverse as Hotmail, Orkut, Myspace and even PayPal have

been hacked using XSS exploits, and removing them completely from your

site involves very careful processing of all user-supplied html code. However,

even if your own site is clean, your customers may still be vulnerable. The latest

tricks involve using web services to launder origin: for example, the attacker

makes a mash-up of the target site plus some evil scripts of his own, and then

gets the victim to view it through a proxy such as Google Translate [1239].

The problems are compounded when a single ﬁrm provides a wide range of

services. Google, for example, offers everything from search through maps to

mail and word-processing, and other large service companies also have broad

offerings. Where many services live at the same domain, the same origin policy

doesn’t do much work, and there have indeed been a number of XSS-type

vulnerabilities between applications at Google and elsewhere. There are also

privacy consequences of service aggregation, which I’ll come to later.

23.3 Web Applications 735

A further bundle of problems with web services is that their structure is

usually at least partially open to inspection. A user is passed from one page

to another as he goes through a process such as browsing, search, product

selection and payment; the attacker can read the html and javascript source,

observe how parameters are passed, and look for nuggets such as SQL queries

that can be manipulated [78]. He can also look to see whether input choices are

screened by javascript, and try bypassing this to see if interesting things can

be done (this needn’t mean buffer overﬂows, but even just such simple hacks

as ordering stuff at a discount). He can also monkey with hidden ﬁelds to see

if they’re used to pass interesting data. A prudent developer will assume that

clients are malicious — but most developers don’t; the rush online by millions

of businesses has totally outpaced the available security skills (and tools). As

a result, many services are not only buggy but open to manipulation.

So much personal information is now stored on web-based applications that

a successful attacker can make off with large amounts of exploitable data. In

November 2007, Salesforce.com admitted that it had lost the contact lists of

a number of its customers after an employee got phished; for example, its

customer SunTrust had 40,000 of its own customers compromised of whom

500 complained of bad emails that seemed to come from SunTrust [743].

These emails tried to install malware on their machines. In an earlier incident,

Monster.com’s resume database was breached, compromising 1.3 million job

seekers. (These incidents raise a question about the adequacy of current breach

disclosure laws, many of which don’t consider someone’s email address and

the name of one of their business contacts to be ‘personal information’ whose

loss is notiﬁable — but clearly such losses should be notiﬁed.)

So much for general vulnerabilities. Let’s now look at the speciﬁc problems

of some common web services.

23.3.1 eBay

The leading auction site, together with its payment service company PayPal,

are signiﬁcant to the security engineer for quite a number of reasons. For

starters, they’re the phishermens’ largest target by far, as well as being the

platform for lots of old-fashioned fraud. Phishing attacks against PayPal

directly are not much different from the attacks against banks that I discussed

in Chapter 2 (except in that as PayPal isn’t a bank, its customers don’t have

the protection of banking law, such as the U.S. Regulation E, and rely on

PayPal’s good will to make good their losses). Many other frauds are variants

of old commercial scams. For example, hucksters email the underbidders in an

auction, offering goods similar to those that were on sale — but which turn out

to be shoddy or nonexistent. And one of the biggest scams on eBay was run

by a trader in Salt Lake City who sold laptops online. First he traded honestly,

selling 750 laptops legitimately and accumulating a good reputation; then he

736 Chapter 23 ■The Bleeding Edge

took money for 1000 more that he didn’t deliver. That sort of trading strategy

has been around as long as commerce has.

But the auction site adds another twist. It provides a reputation service

whereby honest traders accumulate good references from their trading part-

ners, and many small-time occasional sellers have acquired high trust ratings.

So an increasingly common attack is to hijack one of their accounts — whether

by password guessing, phishing or something more technical — and use this

for fraud. Account takeovers have been reported to be growing rapidly from

the start of 2007 [542].

The easy way to exploit a hijacked account is to sell nonexistent goods, take

the money and run, but there are many variants. A trick that’s growing in

popularity in 2007 is the fake escrow site. The bad guy offers a car for sale; you

win the auction; he then suggests that you use an escrow service to which he’ll

ship the car and you’ll pay the money. Real escrow services do actually exist,

such as escrow.com, but so do many dodgy services set up by fraud gangs [57];

if you wire them the money that’s the last you’ll see of it.

Escrow scams are an example of reputation theft, which brings us to Google.

23.3.2 Google

Google’s security manager looks set to have one of the most interesting jobs

in the business over the next ﬁve years, just as her counterparts at Microsoft

have had over the last ﬁve. The sheer scale of Google’s activities make it both

a target and a conduit for all sorts of wickedness. Again, some of it’s as old as

sin, but other attacks are quite novel.

A good example is Google hacking, where people use a search engine to look

for vulnerable machines. The online Google Hacking Database has hundreds

of examples of search strings that turn up everything from unpatched servers

to ﬁles containing passwords [810]. Suitable searches can also be used against

human targets: these can be searches for anyone who’s exposed themselves

in some speciﬁc way, such as by leaving their social security number visible

online, or searches for usable data on some speciﬁc person. If you’re a possible

target, it’s a good idea to do the search ﬁrst. For example, I was a target for a

while of an animal-rights terror group2, so I used the main search engines to

ﬁnd out where my home address could be found. Companies and governments

regularly search for their own secrets too. Search has simply changed the world

since Altavista burst on the world a little over a decade ago; inquiries that

previously would have taken the resources of a major government can now be

conducted from anyone’s laptop or mobile phone in seconds. And although

the beneﬁts of this revolution greatly outweigh the costs, the costs aren’t zero.

2I was an elected member of the governing body of Cambridge University, which was thinking

of building a monkey house.

23.3 Web Applications 737

The two main innovations that enabled Google to pull away from the other

search engines were the Pagerank algorithm, which ranks a page based on

the number of other pages that link to it, and the idea of linking search to

targeted advertising, rather than the banner ads used by earlier engines like

Altavista. A number of small text ads are placed on the search result page,

and advertisers bid against each other for speciﬁc search terms in a continuous

auction. Advertisers are charged when a user clicks on one of their ads.

Google also lets publishers put links on their web pages in which it serves

ads relevant to the page content, and pays them a percentage of the click-

through revenue. This has turned out to be hugely popular and proﬁtable, and

has revolutionised classiﬁed advertising.

Yet it’s brought wave after wave of attacks. The big problem in 2006 was

click-fraud; your ﬁrm’s competitors click repeatedly on your ads, thereby

burning up your ad budget. Click-fraud can also be done by publishers who

want to maximise their commissions. Google added various algorithms to

try to detect click fraud: repeated clicks from one IP address, for example,

are discounted when it comes to billing. Attackers then ﬁgured out that

the order in which ads appeared depends on the click-through rate, as

Google optimises its own revenue by ranking popular ads higher. This led

to a further service-denial attack, impression spam, in which your competitor

repeatedly calls up the results pages in which your ads appear but doesn’t

click on them. This causes your click-through rate to plummet, so your ads get

downgraded.

In 2007, one of the big problems was Google arbitrage. A publisher buys

cheap ads to drive trafﬁc to his site, where he writes about topics that attract

more expensive ads. If customers who arrive at his site cheaply leave via

a more expensive route, he makes a proﬁt. Attitudes to this are mixed. Buying

ads in order to sell ads has a long enough history; your local paper probably

lives from classiﬁed advertising, and may also have posters all over town.

Some advertisers think it’s fraud: they pay for clicks from people fresh off

searches, and instead get second-hand trafﬁc from people escaping boring

web pages that they didn’t really want to go to. Google acts from time to time

against the arbitrageurs, whose proﬁts were dwindling by year end.

But this is just a small part of a larger problem, namely ‘Made for Adsense’

(MFA) sites. One pattern we’ve detected is the fake institution: the scamster

copies an existing charitable or educational website with only minor changes to

the content and uses the knock-off to host something with a high cost-per-click

such as job ads. The idea is that where websites are highly ranked, copies of

them will be too: and some of the bogus charities even set out to exchange

links with real ones to further confuse the issue3.

3That’s how we stumbled across the network, when they offered an ad exchange with the

Foundation for Information Policy Research, which I chair.

738 Chapter 23 ■The Bleeding Edge

Another series of sites is run by a ﬁrm that buys up abandoned domain

names, writes relevant editorial content, and ﬁlls them with ads. (The content

is written for free by journalism students who look for places at which to

‘intern’ as part of their course). When I presented an analysis of such sites

at Google, the reaction was mixed. There’s a serious policy question here:

although one might not think very much of the content, this is in some sense

a new literary genre that’s emerged on the back of the Adsense model, just

as soap operas emerged once TV stations started being funded by adverts for

fast-moving consumer goods. Googlers’ view on balance was that such sites

should be left to market forces. But there are clearly some close judgment calls

between what’s abuse and what’s merely tiresome.

There are many interesting research problems here, such as how one goes

about identifying and mapping bogus communities, where links have been

manufactured to create a semblance of real social or economic activity in

order to fool the Pagerank algorithm, and hidden communities, such as the

network of sites based on abandoned domain names. The latter, at least, is

similar to the problems faced by police and intelligence agencies searching for

insurgent groups, while distinguishing bogus communities from genuine ones

may also come to depend on increasingly sophisticated trafﬁc analysis and

social-network analysis of the sort discussed in sections 19.3.1, 21.5 and 24.3.2.

This brings us inevitably to the issue that most observers consider to be

Google’s Achilles heel: privacy. Privacy concerns operate at many levels.

First, there’s unauthorized access to data. When a ﬁrm with tens of thou-

sands of employees holds personal data on hundreds of millions

of people, there’s a clear risk that information will leak — perhaps via

a disgruntled insider, or perhaps via an exploit of some kind. These are

basically the issues I discussed in Chapter 9, although on a much larger

scale than ever before.

Second, there’s privacy law. For example, the European Commission is

in dispute about how long clickstream data should be kept: Google’s

agreed to ‘de-identify’ clickstreams after 18 months, but this doesn’t

really satisfy the Commission. In section 9.3.1 I discussed how AOL

released anonymised search histories for ‘research’, and some users were

promptly identiﬁed from their search patterns; from the technical point

of view, achieving privacy via anonymity requires frequent changes of

pseudonym. (I’ll return to privacy later when I deal with policy.)

Third, there’s lawful access to authorised data, as when the FBI (or a

divorce lawyer) turns up in Mountain View with a subpoena.

Various people, for various reasons, will want to limit the possible damage

resulting from one of more of these possible types of privacy exposure.

23.3 Web Applications 739

And the tensions look set to become steadily more serious as more information

about us becomes available and searchable.

But before we go on to examine privacy technology, there’s a third type of

web service that can collect even more intimate and pervasive information:

the social-networking site.

23.3.3 Social Networking Sites

Social networking sites such as MySpace and Facebook have taken off rapidly

since about 2004, and are now used by large numbers of young people

to organise their social lives. Other sites aim at organising professionals.

The ﬁeld is developing rapidly, but as I write in January 2008, the main users

are still the young, and the typical site offers not just a place to store a home

page but ways to link to your friends’ pages too. The key idea is that the

site mirrors the underlying social network: the added value is that it enhances

your social network by helping you communicate with friends more conve-

niently and by making new friends. The access-control mechanisms typically

let your friends see more of your stuff; there are messaging services such as

forums, chat rooms and instant messenger, to support social interaction; and

there are various structured methods of getting to know people. On some sites,

you have to be introduced by mutual acquaintances; on others, you can search

for people who meet given criteria such as age, location, musical tastes, hobbies,

sex and sexual orientation. Society always had such mechanisms, of course,

in the form of clubs, and some people see social networking merely as a kind

of online cocktail party. However, the social-networking revolution enables

rapid innovation of the mechanisms that people use to form friendships, look

for partners and set up professional and business relationships.

The putative business model is, ﬁrst, that the huge amount of information

subscribers make available to the operators of these sites will enable even better

targeted advertisement than on Google; and second, that friendship networks

can create massive lockin, which is the source of value in the information

industries. It’s hard not to have a page on Facebook if all your friends do, and

having a hundred million people keeping their address books and message

archives on your servers rather than on their own PCs may seem like a license

to print money4.

So what problems should you anticipate when designing a social-networking

site? Much government advice centres on the fearmongering aspects: young

people reveal a lot about themselves on social sites, and can attract sex preda-

tors. It’s certainly true that the Internet has had an effect on sex crimes, while

4Don’t forget fashion though: in England everyone seems to have had a MySpace page in 2006,

and most people have a Facebook page now in 2007 — but Brazilians use Orkut, and who can

tell what will be cool in 2012?

740 Chapter 23 ■The Bleeding Edge

no signiﬁcant impact has been detected on any other category of offense. Its

uptake across U.S. states and other countries was associated with a small rise

in ‘runaways’, that is, under-18s leaving home without their parents’ permis-

sion. Some runaways were no doubt kids escaping unsatisfactory homes or

simply heading off to seek their fortunes; the key question is how many of

them were abused. The ﬁgures show that Internet uptake was correlated with

a drop in reported cases of rape, and that there was no increase in the murder

rate [709]. One might worry about whether runaway teens turn to sex work,

but prostitution also fell as the Internet spread. It might still be argued that a

small increase in sexual abuse of runaway teens was masked by a larger fall in

sex crimes overall, caused in turn by the greater availability of pornography;

but the drop in sex crimes was signiﬁcant only among male offenders aged

15–24 and there was no corresponding increase of older offenders. And young

people I’ve talked to take the view that fending off unwanted advances from

older men is just part of life anyway, whether ofﬂine or online.

The view you take of all this, if you’re building such a system, may depend

on whether your site is aimed at distance networking — as with photographers

from different continents trading pictures and tips on Flickr — or at networks

involving physical relationships. In the second case, the next question is

whether you restrict membership to teens and above, as Facebook does, or let

anyone join, as MySpace does. There’s a reasonable discussion of the policy

and technical issues from ENISA [615]. As for younger children, it’s clearly

prudent for parents to keep an eye on online activities; the junior members of

our family get to use the computer in the kitchen, which also helps get across

the message that the Internet is public space rather than private space. ENISA

also recommends that schools should encourage rather than prohibit social

network use, so that bullying can be reported and dealt with. Bullying has

always been a low-grade problem for schools, erupting into very occasional

tragedy with suicides or killings. Before the Internet, bullied children could

escape for long periods of time at home and with friends; nowadays the

taunting can follow them to their bedrooms. This can make it all the more

miserable if their Internet use is secret and furtive. The cure is to bring things

into the open.

These are, of course, broad issues that apply to Internet use generally, not

just to social networking sites. And on the face of it, you might expect social

networking sites to be less dangerous than random online chat rooms, as the

service operator has an incentive to limit egregious abuse because of the asso-

ciated reputation risk. But what are the interesting security engineering issues?

One emerging property of social networking systems is the sheer complexity

of security policy. In Chapter 4, I discussed how access controls are simple

close to the hardware, but get more complex as you move up the stack through

the operating system and middleware to the application. Social networking

applications attempt to encapsulate a signiﬁcant subset of human behaviour

23.3 Web Applications 741

in groups; the result is ever-more complicated sets of rules, which are very

difﬁcult for users to manage.

For example, setting privacy policy on Facebook in October 2007 means

wading through no less than six screens of access options — essentially a set

of access control lists to different parts of your proﬁle. And the controls aren’t

linear; photos, for example, have a policy of opt-in and opt-out. If I recognise

you in a photo on someone’s page, I can tag that photo with your name, but

the tag won’t be publicly visible until the photo owner agrees (the opt-in). If

you’re a Facebook member you’ll be notiﬁed, and you can remove the tag if you

want (the opt-out) [450]. However you might not notice the tag, and this could

have consequences if the photo were embarrassing — say, a drunken party.

For example, on New Year’s day 2008, following the assassination of Benazir

Bhutto in Pakistan, the UK press published a photo of her son and political heir

Bilawal Bhutto, dressed up in a devil’s costume with red horns for a Halloween

party, which was found on the Facebook site of one of his student friends.

Many people just don’t understand what the terms used in the access con-

trols mean. For example, if you make photos available to the ‘community’,

that means by default anyone with an email address within your institu-

tion — which has led to campus police having discussions with people who’ve

uploaded photos of assorted kinds of rulebreaking activities [463]. Facebook

also doesn’t deal with multiple personae; for example, I’m Ross the computer

scientist, Ross the technology-policy guy, Ross the musician, Ross the family

man, and so on — and as I can’t separate them, I’ve ‘friended’ only people I

know from the ﬁrst two communities on Facebook.

There are also some quite subtle policy issues: for example, you’d think

it was alright to publish information that was already public? Wrong! There

was a huge row when Facebook added a news feed feature that notiﬁed

all your status changes to all your friends. Previously, if you’d broken up with

your girlfriend, this would be publicly visible anyway (assuming you made

partnership data public). Suddenly, such a change was automatically and

instantly broadcast to your entire social circle — which upset a lot of people.

It’s not just that the site had automated some of the social aspects of gossip;

it suddenly made social faux pas very visible [216]. This feature can now be

turned off, but the extra complexity just makes it even harder for people to

manage their privacy.

Another example is search. Most web services ﬁrms are tempted to use

private data in public searches in order to make them more precise. Back in the

early days (2004), it turned out that search on Friendster leaked private data:

a chain of suitably-chosen searches could infer a user’s surname and zip code

even where this data had been set as private [902]. The moral was that public

searches should never be allowed to return results based on private data.

Another lesson that should have been more widely learned is that once the

social network is known, inference control becomes much harder. Friendster

742 Chapter 23 ■The Bleeding Edge

duly ﬁxed its systems. Yet the bug was rediscovered in Facebook in 2007:

Facebook had simply left it to users to decide whether they wanted to turn

off search on private terms, and essentially none of them had done so [1197].

That’s now ﬁxed, but the issue arose yet again when Facebook made available

stripped-down versions of user proﬁles to Google. Once more, it had been

left to users to become aware of this risk and turn off the relevant feature;

once more, almost nobody did so [694]. Facebook appears to have a strategy

of dumping all the really hard security decisions on the users — so they can

respond to criticism by blaming users for not turning off features X and Y.

Searchability by default may be in their short-term ﬁnancial interest, but the

end result can too easily be unusable security plus unsafe defaults.

Another tension between corporate growth and security usability was a

recent decision to allow third-party application developers access to proﬁle

data. When someone builds an app that allows people to export photos (say)

from Flickr to Facebook, then how on earth are we to evaluate that? Even

if the two systems are secure in isolation, there’s no guarantee that this

will compose — especially where systems have complex and ever-changing

APIs, and complex hard-to-use privacy policies. Then, in late 2007, Facebook

faced a revolt of its users after it introduced ‘Beacon’, an advertising system

that told users’ friends about what they’d just purchased on other websites,

and made the feature opt-out rather than opt-in. Mark Zuckerberg, founder

and chief executive, apologized to the site’s users for handling the matter

badly. (It remains to be seen whether future marketing ideas will be opt-in.)

There are both ‘soft’ and ‘hard’ issues bundled up here. At the soft end,

people present different personae at different sites; for example, by placing

different kinds of photos on Flickr and Facebook [1276]. At the nastier end,

not all applications are written by large, respectable companies. Yet once

you authorise a third-party application to access your proﬁle, it can do

most anything you can — including spamming your friends and selling their

personal information to third parties. There’s a sense in which making a

‘friend’ on Facebook is the digital equivalent of unprotected sex — you’re

exposed to everything they’ve got.

All the usual tussles between free speech and censorship pop up too. For

example, in Flickr, you’re not supposed to upload photos you wouldn’t show

your mum unless you tag them as ‘restricted’ (i.e. adult). You’re not allowed

to view such material if you’re in Germany, or search for it in Singapore. Yet

a colleague who uploaded art nudes had his account blacklisted as ‘unsafe’,

even though he’s quite happy to show his mum his work. And as far as

data protection law is concerned, Facebook tends to reveal the data subject’s

race, sex life, health, religion, political beliefs and whether he belongs to a

trade union — precisely those ‘sensitive’ types of data that get special protec-

tion under European privacy law [235].

23.3 Web Applications 743

There’s a further bunch of problems at the interface between technical and

social mechanisms. For example, you make conﬁdential information available

to your friends, one of whom gets his account compromised, and your data

ends up public. When this was ﬁrst reported (in 2003), pioneers expected

that social pressures would make users more careful [961], but this hasn’t

happened. The underlying reasons may be familiar: in a world of strong

network externalities, systems start off insecure in order to grow as quickly as

possible. But while Windows and Symbian were insecure in order to appeal

to complementers while building a dominant position in a two-sided market,

social-network site operators bias their algorithms and their presentation to

get people to enrol as many friends as possible. This undermines any possible

social discipline.

Other socio-technical attacks include cross-site scripting vulnerabilities, of

which there have been plenty [902]. Spam is rising fast, and a particularly

ominous problem may be phishing. A recent experiment at Indiana University

sent phish to a sample of students, asking them to check out an off-campus

website that solicited entry of their university password. The success rate with

the control group was 16% but a group targeted using data harvested from

social networks were much more vulnerable — 72% of them were hooked

by the phish [653]. Already there’s a signiﬁcant amount of phishing being

reported on MySpace [796].

I’ll ﬁnish up this section by making two more general points. The ﬁrst is

that, as the social-networking sites learn rapidly from experience and clean up

their act, the largest medium-term problems may well be, ﬁrst, the migration

online of real-world social problems such as bullying; and second, that many

teens put stuff online that they’ll later regret, such as boasts of experiments

with drink, drugs and sex that get dug up when they apply for jobs. In

Seoul, a girl was asked to pick up some poo left by her dog, and refused;

a bystander ﬁlmed this, she became known as ‘dog poo girl’, and she was

hounded from university [1201]. Although that’s an extreme case, the principle

is not really new: people who posted immoderately on the old network news

system sometimes found themselves haunted by ‘the Ghost of Usenet Postings

Past’ [507]; and there are tales going back centuries of social faux pas that

ruined lives, families and fortunes. But while in olden times it would most

likely be a lapse of manners at court that got you in bad trouble, now it can be

a lapse of manners on the subway.

The world is steadily becoming more documented — more like the villages

most of us lived in until the eighteenth century, where everyone’s doings

were known. Back then, the village gossips would serve up a m`

elange of

assorted factoids about anyone local — some true, and some false — which

people would use when forming opinions. Nowadays, Google has taken over

that role, and it’s much less susceptible to social pressure to correct errors,

744 Chapter 23 ■The Bleeding Edge

or forgive the occasional blunders of otherwise decent people. Also, much of

the anonymity that people got from moving into towns during the industrial

revolution is being lost. The effect of persistent social memory on social

problems will be mixed. Bullying may be mitigated because of the record

left behind, while the embarrassment problem may be resolved by some

combination of a more relaxed attitude to youthful indiscretions, plus greater

discretion on the part of youth. We’ll have to wait and see which dominates,

but early signs are that people are becoming more relaxed: the Pew Internet &

American Life Project found that 60% of Americans are unconcerned in 2007

about the ‘digital footprint’ they leave behind, while in a survey in 2000, 84%

were worried. So we’re learning to cope [1229]. (Discretion is part of coping,

and that may involve the use of a pseudonym or nickname that isn’t too easy

for outsiders to link to your real person, but I’ll discuss all that in the next

section.)

Second, social network systems have the potential to do an awful lot of

good. The Harvard sociologist Robert Putnam documented, in his inﬂuential

book ‘Bowling Alone’, how social networks in America and elsewhere were

damaged by the advent of television, the move to the suburbs and even the

move by women into work (though TV did by far the most damage) [1052].

The baby-boom generation, who were the ﬁrst to be raised with TV, are much

less likely to join clubs, know our neighbours, meet frequently with friends or

participate in team sports than our parents did, and the succeeding ‘generation

X’ are less likely still. Now it seems that sociability is ticking upwards once

more. What TV and mass consumer culture took away, the PC and the

mobile phone may be giving back. Easier communication not only makes

people communicate more but in different ways; the old communities based

on geography are being supplemented by communities of shared interest.

We academics were among the ﬁrst to beneﬁt; the communities of people

interested in cryptography, or protocols, or number theory, have been held

together as much by the Internet as by the conference circuit for many years.

Now these beneﬁts are spreading to everybody, and that’s great.

Social-networking sites also provide a platform for rapid experimentation

and innovation in new ways of making and maintaining friendships. And they

may be brilliant for the geeky, the shy, the ugly, and people with borderline

Asperger’s. But to the extent that they try to encapsulate more and more of the

complexity of real social life, their policies will become ever more complex.

And just as we’re limited in our ability to build large software systems by

technical complexity, so social-networking systems may explore a new space in

which policy complexity — security usability, in a new guise — may provide

one of the ultimate limits to growth. It will be interesting to watch.

23.4 Privacy Technology 745

23.4 Privacy Technology

As business moves online, vast amounts of information start to get collected.

In the old days, you walked into a record store and bought an album for

cash; now you download a track from a server, which downloads a license

to your PC. The central license server knows exactly who bought access to

what, and when. Marketers think this is magniﬁcent; privacy advocates are

appalled [410]. The move to pervasive computing is also greatly increasing

the amount of information held on us by others — for example, if people start

using applications in their mobile phones to track their social networks and

help them manage their time better [407]. There will no doubt be all sorts of

‘must have’ applications in the future that collect data about us, which means

growing uncertainty about what will be available to whom.

Technology is already putting some social conventions under strain. In

pre-technological societies, two people could walk a short distance away from

everyone else and have a conversation that left no hard evidence of what was

said. If Alice claimed that Bob had tried to recruit her for an insurrection, then

Bob could always claim the converse — that it was Alice who’d proposed to

overthrow the king and he who’d refused out of loyalty. In other words, many

communications were deniable. Plausible deniability remains an important

feature of some communications today, from everyday life up to the highest

reaches of intelligence and diplomacy. It can sometimes be ﬁxed by convention:

for example, a litigant in England can write a letter marked ‘without prejudice’

to another proposing a settlement, and this letter cannot be used in evidence.

But most circumstances lack such clear and convenient rules, and the electronic

nature of communication often means that ‘just stepping outside for a minute’

isn’t an option. What then?

Another issue is anonymity. Until the industrial revolution, most people

lived in small villages, and it was a relief — in fact a revolution — to move

into a town. You could change your religion, or vote for a land-reform

candidate, without your landlord throwing you off your farm. Nowadays,

the phrase ‘electronic village’ not only captures the way in which electronic

communications have shrunk distance, but also the fear that they will shrink

our freedom too.

Can technology do anything to help? Let’s consider some ‘users’ — some

people with speciﬁc privacy problems.

1. Andrew is a missionary in Texas whose website has attracted a num-

ber of converts in Saudi Arabia. That country executes citizens who

change their religion. He suspects that some of the people who’ve

746 Chapter 23 ■The Bleeding Edge

contacted him aren’t real converts, but religious policemen hunting for

apostates. He can’t tell policemen apart from real converts. What sort

of technology should he use to communicate privately with them?

2. Betty is your ten-year-old daughter, who’s been warned by her teacher

to remain anonymous online. What sort of training and tools should

you give her to help her manage this?

3. Charles is an engineer at a Silicon Valley startup that’s still in stealth

mode, and he’s running a blog — in contravention of his company’s

rules. How can he avoid getting caught and ﬁred?

4. Dai is a human-rights worker in Vietnam, in contact with people try-

ing to set up independent trade unions, microﬁnance cooperatives and

the like. The police harass her frequently. How should she commu-

nicate with co-workers?

5. Elizabeth works as an analyst for an investment bank that’s advising

on a merger. She wants ways of investigating a takeover target with-

out letting the target get wind of her interest — or even learn that

anybody at all is interested.

6. Firoz is a gay man who lives in Teheran, where being gay is a capital

offence. He’d like some way to download porn without getting hanged.

7. Graziano is a magistrate in Palermo setting up a hotline to let people

tip off the authorities about Maﬁa activity. He knows that some of the

cops who staff the ofﬁce in future will be in the Maﬁa’s pay — and that

potential informants know this too. How does he limit the damage

that future corrupt cops can do?

This helps to illustrate that privacy isn’t just about encrypting phone calls

or web trafﬁc. For example, if Andrew tells his converts to download and

use a particular cryptography program, then so will the police spies; and the

national ﬁrewall will be set to detect anyone who sends or receives messages

using that program. Andrew has to make his trafﬁc look innocuous — so that

the religious police can’t spot converts even when they have full knowledge

of what apostate trafﬁc looks like.

And while suitable technical measures may solve part of Andrew’s problem,

they won’t be much use with Betty’s. The risk to her is largely that she will

give out information carelessly that might come back to haunt her. Filtering

software can help — if she’s not allowed to give out her home address over

the Internet, a ﬁlter can look for it, and beep if she gets careless — but most

of the effort will have to go into training her.

There’s also wide variation in the level at which the protection is provided.

Betty’s protection has to be provided mostly at the application layer, as the

main problem is unintentional leaks via content; the same applies to Charles.

23.4 Privacy Technology 747

However, Charles might face more sophisticated analysis, perhaps at the hands

of someone like Elizabeth: she might trawl through his postings looking for

metadata from camera serial numbers in the images to names of workgroups

or even printers embedded in documents, so that she can ﬁgure out who he’s

working with on his secret project.

The intensity of attacks will also vary. Charles and Firoz might face only

sporadic interest, while Dai is subjected to constant surveillance. She’ll use

anonymous communications not so much to protect herself, but to protect

others who haven’t yet come to the police’s attention. There are huge differ-

ences in protection incentives: Andrew may go to a lot of trouble to make his

website as harmless as possible to its visitors (for example, by hosting it on

the same machine as many innocuous services), while the sites in which Firoz

is interested don’t care much about his safety. Andrew, Dai and Graziano all

have to think hard about dishonest insiders. Different probability thresholds

mark the difference between success and failure; plausible deniability of an

association might be enough to get Charles off the hook, while mere suspicion

would frustrate Elizabeth’s plans. And there are different costs of failure:

Elizabeth may lose some money if she’s caught, while Firoz could lose his life.

We’ve come across anonymity mechanisms before, when we discussed how

people who don’t want their phone calls traced buy prepaid mobile phones, use

them for a while, and throw them away. Even that’s hard; and even Al-Qaida

couldn’t do it right. So what are the prospects for hard privacy online?

23.4.1 Anonymous Email – The Dining Cryptographers

and Mixes

As we remarked in several contexts, the opponent often gets most of his

information from trafﬁc analysis. Even if the communications between Alice

and Bob are encrypted and the ciphertext hidden in MP3 ﬁles, and even if on

inspection neither Alice’s laptop nor Bob’s contains any suspicious material,

the mere fact that Alice communicated with Bob may give the game away.

This is why criminals set much more store by anonymous communication

(such as using prepaid mobile phones) than by encryption. There are many

legitimate uses too, from the folks on our list above through anonymous

helplines for abuse victims; corporate whisteblowers; protest groups who

wish to dig an elephant trap for the government; anonymous student feedback

on professors; anonymous refereeing of conference paper submissions, and

anonymous HIV tests where you get the results online using a one-time

password that came with a test kit you bought for cash. You may want to apply

for a job without your current employer ﬁnding out, to exchange private email

with people who don’t use encryption, or ﬁght a harmful and vengeful cult.

There are two basic mechanisms, both invented by David Chaum in the

1980’s. The ﬁrst is the dining cryptographers problem, inspired by the ‘dining

748 Chapter 23 ■The Bleeding Edge

philosophers’ problem discussed in section 6.2.4. Several cryptographers are

gathered around a table for dinner, and the waiter informs them that the meal

has already been paid for by an anonymous benefactor, who could be one of

the participants or the NSA. The cryptographers would like to know which.

So pairs of principals share one time pads, after which each principal outputs

a function of her ‘I paid/I didn’t pay’ bit and everyone can later work out the

total parity of all such bits. As long as not more than one of the cryptogra-

phers says ‘I paid’, even parity means that the NSA paid, while odd parity

means that one of the diners paid, even if nobody can ﬁgure out who [286]. This

mechanism can be considered the anonymity equivalent of the one-time pad; it

gives ‘unconditional anonymity’, albeit at the cost of a laborious protocol and

a lot of key material. Various extensions have been proposed, including one in

which ‘dining drug dealers’ can auction a consignment of cocaine without the

bidders’ identities being revealed to the other bidders or to the seller. Nobody

except buyer and seller know who won the auction; and even the seller is not

able to ﬁnd out the identity of the highest bidder before committing to the

sale [1219].

However, for practical anonymity applications, the pioneering innovation

was another idea of Chaum’s, the mix or anonymous remailer [284]. This accepts

encrypted messages, strips off the encryption, and then remails them to the

address that it ﬁnds inside. In its simplest form, if Alice wants to send

anonymous email to Bob via Charlie and David, she sends them the message:

A−→ C:{D,{B,{M}KB}KD}KC

Charlie now strips off the outer wrapper, ﬁnds David’s address plus a cipher-

text, and sends the ciphertext to David. David decrypts it and ﬁnds Bob’s

address plus a ciphertext, so he sends the ciphertext to Bob. Bob decrypts this

and gets the message M.

Anonymous remailers came into use in the 1990s. To start off with, people

used single remailers, but, as I mentioned in section 22.3.3, an early remailer

was closed down following court action by the Scientologists, after it was used

to post material critical of them. A lot of people still rely on services such as

Hotmail and Hushmail that provide simple, low-cost anonymity, but if you

might be subjected to legal compulsion (or sophisticated technical attack) it’s

wise not to have a single point of failure5. Chainable remailers were initially

developed by the cypherpunks; they not only did nested encryption of outgo-

ing trafﬁc but also supported a reply block — a set of nested public keys and

email addresses that lets the recipient reply to you. There are also nymservers

that will store reply blocks and handle anonymous return mail automatically.

The most common design at present is the Mixmaster remailer, which also

5‘Wired’ was surprised in November 2007 when it turned out that Hushmail responded to

warrants [1177] — which itself is surprising.

23.4 Privacy Technology 749

protects against basic trafﬁc analysis by padding messages and subjecting

them to random delays [899].

A common application is anonymous posting to mailing lists with sensitive

content — applications range from reporting security vulnerabilities through

abuse support to anonymous political speech. Of course, an anonymous

remailer could be an attractive honey trap for an intelligence agency to

operate, and so it’s common to send messages through a number of successive

remailers and arrange things so that most of them would have to conspire

to break the trafﬁc. Even so, selective service-denial attacks are possible; if

the NSA runs remailers X and Y, and you try a path through X and Z, they

can cause that to not work; so you then try X and Y, which ‘works’, and

you’re happy (as are they). Remailer operators can also be subjected to all

sorts of attacks, ranging from subpoenas and litigation to spam ﬂoods that

aim get the operator blacklisted; David Mazi`

eres and Frans Kaashoek have

a paper on their experiences running such a service [849]. The technology is

still evolving, with the latest proposals incorporating not just more robust

mechanisms for ﬁxed-length packets and single-use reply blocks, but also

directory and reputation services that will allow users to monitor selective

service-denial attacks [344].

23.4.2 Anonymous Web Browsing – Tor

Anonymous connections aren’t limited to email, but can include any commu-

nications service. As the web has come to dominate online applications, The

Onion Router (Tor) has become the most widely-used anonymous communi-

cation system, with over 200,000 users. Tor began its life as an experimental US

Navy Labs system, called Onion Routing because the messages are nested like

the layers of an onion [1062]. The Navy opened it up to the world, presumably

because you can usually only be anonymous in a crowd. If Tor had been

restricted to the U.S. intelligence community, then any website getting Tor

trafﬁc could draw an obvious conclusion. U.S. Naval personnel in the Middle

East use Tor to connect back to their servers in Maryland. They don’t think

of it as an anonymity system but as a personal safety system: they don’t want

anyone watching the house they’re in to learn their afﬁliation, and they don’t

want anyone watching the servers in Maryland to learn where they are. In

effect, they hide among local (Iraqi and Maryland) men looking for porn; and

porn trafﬁc also conceals human-rights workers in the third world. Tor may

be a part of the solution adopted by several of our representative privacy users

(Charles, Dai, Elizabeth, Firoz and maybe even Graziano), so I’ll now discuss

its design and its limitations6.

6By way of declaration of interest, I hold a grant from the Tor Project that pays one of my postdocs

to help develop their software. There are also commercial services, such as Anonymizer [79], that

let you browse the web anonymously, but they’re routinely blocked by repressive governments.

750 Chapter 23 ■The Bleeding Edge

The Tor software consists of the Tor client, which forwards TCP trafﬁc,

a web proxy through which it talks to your browser, and optionally a ‘Tor

Button’ that acts as an extension to the Firefox browser and lets you switch

rapidly between normal and anonymous browsing. In the latter mode, the

Tor button disables cookies, javascript and all other plugins. Volunteers with

high-bandwidth connections enable the Tor client to also act as a server, of

which there may be a few thousand active at any time. When you turn on a Tor

client, it opens a circuit by ﬁnding three Tor servers through which it connects

to the outside world. It negotiates an encrypted session with the ﬁrst server,

then through this it negotiates another encrypted session to the second server,

through which it then sets up a third encrypted session to the exit node. Your

web browser trafﬁc ﬂows in the clear from the exit node to your destination.

This brings us immediately to one widely-publicised Tor vulnerability — the

malicious exit node. In September 2007, someone set up ﬁve Tor exit nodes,

monitored the trafﬁc that went through them, and published the interesting

stuff [917]. This included logons and passwords for a number of webmail

accounts used by embassies, including missions from Iran, India, Japan and

Russia. (This gave an insight into password robustness policy: Uzbekistan

came top with passwords like ‘s1e7u0l7c’ while Tunisia just used ‘Tunisia’

and an Indian embassy ‘1234’.) Yet the Tor documentation and website make

clear that exit trafﬁc can be read, so clueful people would have either used a

webmail service that supports TLS encryption, like Gmail, or else used email

encryption software such as PGP (which I’ll mention later).

The second problem with anonymous web browsing is the many side-

channels by which modern content calls home. This is why the proxy

distributed with the Tor client kills off cookies and javascript, but that’s

just the beginning. If Firoz downloads a porn movie, and his Windows Media

Player then calls the porn server directly to get a license, the packet trafﬁc

from his IP address to a ‘known Satanic’ site may be a giveaway; but then,

if he blocks the license request, he won’t be able to watch the ﬁlm. ActiveX

controls, Flash and other browser add-ons can also open connections outside

the proxy. For surveys of ways in which websites can defeat anonymising

services, see [1091, 1194].

Third, while the Mixmaster and later remailers can make trafﬁc analy-

sis harder by dicing, padding and delaying trafﬁc, this introduces latency

that would not be acceptable in most web applications. Low-latency, high-

bandwidth systems such as Tor are intrinsically more exposed to trafﬁc

analysis. A global adversary such as the NSA, that taps trafﬁc at many points

in the Internet, can certainly recover information about some Tor circuits by

correlating their activity; in fact, they only need to tap a small number of

key exchange points to get a good sample of the trafﬁc [920] (so if the U.S.

government ﬁgures in your threat model, it may be prudent to set up new Tor

circuits frequently).

23.4 Privacy Technology 751

Finally, many applications get users to identify themselves explicitly, and

others get them to leak information about who they are without realising

it. In section 9.3.1 I discussed how supposedly anonymous search histories

from AOL identiﬁed many users: a combination of local searches (that tell

where you live) and special-interest searches (that reveal your hobbies) may

be enough to identify you. So if you’re using Tor to do anonymous search,

and there is even the slightest possibility that your opponent might be able to

serve a subpoena on the search engine, you had better set up new circuits, and

thus new pseudonyms, frequently.

If your opponent is less capable, then trafﬁc patterns may still give the game

away. First, suppose you want to browse a forbidden website that has a known

and stable structure; a modern commercial web page might contain some 30

objects ranging in site from a few hundred bytes to a few tens of kilobytes.

This pattern is usually unique and is clearly visible even after TLS encryption.

Even although Tor trafﬁc (as seen by a wiretap close to the user) lies under

three layes of Tor encryption, and even though cells are padded to 512 bytes,

random web pages still leak a lot of information about their identity. So if

Andrew wants his converts to view his website through Tor, and there’s a real

risk that they’ll be killed if they’re caught, he should think hard. Should he pad

his webpages so that, encrypted, they will be the same size as a popular and

innocuous site? Should be put short sermons on YouTube, of the same length

as popular music tracks? Or should he use a different technology entirely?

An opponent who can occasionally get control of the forbidden website

can play yet more tricks. Graziano, who’s worried about Maﬁa takeover of

the police’s Maﬁa tip-off site, should consider the following attack. The Maﬁa

technicians make a number of probes to all the Tor servers as the page is

loaded, and from the effects on server load they can identify the path along

which the download was made. They then go to the local ISP, which they

bribe or coerce into handing over the trafﬁc logs that show who established a

connection with the entry node at the relevant time [919]. (So from Graziano’s

point of view, at least, the recent European directive compelling ISPs to retain

trafﬁc logs may not always help the police.)

There’s no doubt that Tor is an extremely useful privacy tool, but it has to

be used with care. It’s more effective when browsing websites that try to respect

users’ privacy than when browsing sites that try to compromise them; and it’s

often used in combination with other tools. For example, human-rights workers

in less developed countries commonly use it along with Skype and PGP.

23.4.3 Conﬁdential and Anonymous Phone Calls

I discussed in Chapter 20 how criminals looking for anonymous communica-

tions often just buy prepaid mobiles, use them for a while, and throw them

away. They are a useful tool for others too; among our representative privacy

752 Chapter 23 ■The Bleeding Edge

users, Andrew might think of telling his converts to use them. But they are

not the only option, and they don’t provide protection against wiretapping. If

your opponent has the technology to do automatic keyword search or speaker

recognition on phone trafﬁc, as the NSA does, or the manpower to run a large

number of wiretaps, as a typical third-world despot does, then you might

want to consider voice over IP.

In theory, you can run VOIP communications through proxies like Tor [665];

but in practice, not many people do; and as anonymity usually means hiding in

a crowd, that brings us to Skype. Skype is not only the largest VOIP operator,

which gives safety in numbers; it’s got a peer-to-peer architecture, so your

calls go end-to-end; and the trafﬁc’s encrypted, with mechanisms that have

undergone an independent evaluation [165].

So what can go wrong? Well, if Andrew were to use Skype to talk to

his converts then he’d better not use the same username to talk to all of

them; otherwise the religious police will learn this username from their bogus

convert and search for everyone who calls it. Fortunately, you can get multiple,

throwaway Skype usernames, and provided Andrew uses a different username

for each contact Skype may be a suitable mechanism. The next problem, for

some applications at least, is that Skype being owned by a large U.S. company

is likely to respond to warrants7So if your threat model includes the U.S.

Government, you’d better assume that the call content can be decrypted once

the NSA identiﬁes your trafﬁc as of interest. You might be at risk if you’re

opposing a government, such as that of Uzbekistan, with which the USA

has intelligence-sharing agreements; and you might also be at risk if Skype’s

parent company, eBay, has an ofﬁce in the country whose police you’re trying

to hide your trafﬁc from. So if Andrew’s unsure about whether eBay would

help out the Saudi government, he might use Skype largely as an anonymity

mechanism, and use it to mask the transfer of ﬁles that are encrypted using a

product such as PGP.

Human-rights workers such as Dai do in fact use Skype plus PGP plus Tor

to protect their trafﬁc, and the attacks to which they’re subjected are the stuff

of intelligence tradecraft. The police enter their homes covertly to implant

rootkits that sniff passwords, and room bugs to listen to conversations. When

you encrypt a phone call, you have to wonder whether the secret police are

getting one side of it (or both) from a hidden microphone. Countering such

attacks requires tradecraft in turn. Some of this is just like in spy movies:

leaving telltales to detect covert entry, keeping your laptop with you at all

times, and holding sensitive conversations in places that are hard to bug. Other

aspects of it are different: as human-rights workers (like journalists but unlike

7Skype itself is actually a Luxembourg company, and its ofﬁcers who respond to law enforcement

are based there: so an FBI National Security Letter may not be effective, but a judicial warrant

should be.

23.4 Privacy Technology 753

spies) are known to the host government, they need to avoid breaking the law

and they need to nurture support structures, including overt support from

overseas NGOs and governments. They also need — while under intermittent

observation — to make covert contact with people who aren’t themselves

under suspicion.

23.4.4 Email Encryption

During the ‘Crypto Wars’ on the 1990s, cyber-activists fought their govern-

ments for the right to encrypt email, while governments pushed for laws

restricting encryption. I’ll discuss the politics in the next chapter. However

one focus of that struggle, the encryption product Pretty Good Privacy (PGP),

along with compatible free products such as GPG, have become fairly widely

used among geeks. A typical use is by Computer Emergency Response Teams

(CERTs) who encrypt information about attacks and vulnerabilities when they

share it with each other. Many private individuals also have PGP encryption

keys and some encrypt trafﬁc to each other by default.

PGP has a number of features but in its most basic form, each user generates

a private/public keypair. To protect a message, you sign a hash of it using

your private key, encrypt both the message and the signature with a randomly

chosen session key, and then encrypt the session key using the public key

of each of the intended recipients. Thus, if Alice wants to send an encrypted

email to Bob and Charlie, she forms the message

{KS}KB,{KS}KC,{M,{h(M)}−1

KA}KS

The management of keys is left to the user, the rationale being that a single

centralized certiﬁcation authority would become such an attractive target that

it would likely be cracked or come under legal coercion. So the intended mode

of operation is that each user collects the public keys of people she intends

to correspond with and bundles them into a public keyring that she keeps on

her system. The public keys can be authenticated by any convenient method

such as by printing them on her business card; to make this easier, PGP

supports a key ﬁngerprint which is a one-way hash of the public key, presented

as a hexadecimal string. Another mechanism is for users to sign each others’

keys. This may simply be used as an integrity-protection mechanism on their

public keyrings, but becomes more interesting if the signatures are exported.

The set of publicly visible PGP signatures makes up the web of trust,which

may be thought of as an early form of social network of people interested

in cryptography. Yet another mechanism was to establish key servers; yet as

anyone could upload any key, we ended up with keys for addresses such

as president@whitehouse.gov not controlled by the people you might

think. Colleagues and I also published a book of important public keys [67].

754 Chapter 23 ■The Bleeding Edge

Many things were learned from the deployment and use of PGP during the

1990s. One of the most signiﬁcant was usability. In a seminal paper, Alma

Whitten and Doug Tygar did a cognitive walkthrough analysis of PGP 5.0

followed by a lab test, to assess whether motivated but cryptologically unso-

phisticated users could understand what was going on well enough to drive

the program safely — to understand the need to generate a public/private

keypair, ﬁgure out how to do so, encrypt messages and sign keys as needed,

and not make gross errors such as accidentally failing to encrypt, or trusting

the wrong public keys. The analysis showed unsafe design decisions and

defaults, such as downloading keys from the MIT server without making clear

that this was happening. The actual test threw up much worse horrors. Only

four of twelve subjects were able to correctly send encrypted email to the other

subjects, and only three of them expressed any doubt about keys from the

key server. Every subject made at least one signiﬁcant error [1342]. The moral

is that if you’re going to get people without degrees in math or computer

science to use encryption, you have to bury it transparently in another product

(such as an online computer game) or you have to train them — and test them

afterwards. So PGP and similar products can be an option for human-rights

workers (and are used by them); but for lonely converts in a hostile country,

encryption alone is questionable.

There may be other reasons why encrypting email is only part of the

solution. In some countries, including Russia, Zimbabwe and the UK, the police

have the power to require you to decrypt ciphertext they seize, or even hand

over the key. This power is also available to the civil courts in many more

countries, and to many tax authorities. Other situations in which coercion may

be a problem include where soldiers or intelligence agents could be captured;

where police power is abused, for example to seize a key on the grounds of

a supposed criminal investigation but where in reality they’ve been bribed

to obtain commercially conﬁdential information; and even in private homes

(kids can be abused by parents, as I noted in the chapter on medical privacy,

and householders are sometimes tortured by robbers to get bank card PINs

and to open safes [1326]).

Making encryption resistant to rubber hose cryptanalysis, as it’s called, is hard,

but it’s possible at least to block access to old messages. For example, the U.S.

Defense Messaging System supports the use of public encryption keys only

once. Each user has a key server that will provide a fresh public encryption key

on demand, signed by the user’s signing key, and once the user receives and

decrypts the message he destroys the decryption key. This forward secrecy

property is also found in Skype; beating someone’s passphrase out of them

doesn’t let you decipher old conversations. As for stored data, making that

coercion-resistant brings us to the topic of steganography.

23.4 Privacy Technology 755

23.4.5 Steganography and Forensics Countermeasures

When your threat model includes coercion, simply destroying old keys may

not always be enough, as the very existence of protected material can be

sufﬁcient to cause harm. In such circumstances, more complete plausible

deniability can be provided by the use of steganography, which is about hiding

data in other data. As an example, Fabien Petitcolas wrote a program called

MP3stego, which will take some information you want to hide (such as an

encrypted message) and hide it in audio: it takes an audio track and compresses

it using the MP3 algorithm, and wherever it can make a random choice about

the compression it uses this to hide the next bit of message. And the CIA

is reported to have had a camera that hid data in the least signiﬁcant bits

of randomly-selected pixels [1020]. There are many steganography programs

ﬂoating around on the net, but most of them are easy to break: they simply hide

your message in the least-signiﬁcant bits of an audio or video ﬁle, and that’s

easy to detect. Recall our discussion of steganography theory in section 22.4:

the two participants, Alice and Bob, have to communicate via a warden, Willie,

who wins the game if he can detect even the existence of a hidden message.

The classic use of steganography is hiding sensitive data (such as ciphertext,

where that arouses suspicion) in innocuous communications, though increas-

ingly nowadays people worry about protecting stored data. Most customs

authorities have the power to require travellers to decrypt any material found

on the hard disk of their laptop in order to check for subversive material,

pornography and the like. There are many crude ways to hide the existence of

ﬁles; at most borders it’s still enough to have an Apple laptop, or a separate

Linux partition on your hard disk which runs Linux, as the normal customs

tools don’t deal with these. But that problem will be ﬁxed eventually, and

against a capable opponent such low-level tricks are likely to be ineffective.

Files can be hidden using steganography tools in larger multimedia ﬁles, but

this is inefﬁcient.

Adi Shamir, Roger Needham and I invented the steganographic ﬁle system,

which has the property that a user may provide it with the name of an

object, such as a ﬁle or directory, together with a password; and if these are

correct for an object in the system, access to it will be provided. However, an

attacker who does not have the matching object name and password, and lacks

the computational power to guess it, can get no information about whether the

named object even exists. This is an even stronger property than Bell-LaPadula;

Low cannot even demonstrate the existence of High. In our initial design, the

whole disk was encrypted, and fragments of the ﬁles are scattered through it

at places that depend on the password, with some redundancy to recover from

cases where High data is accidentally overwritten by a Low user [75, 856].

756 Chapter 23 ■The Bleeding Edge

In recent years, ﬁle-encryption programs such as TrueCrypt have adopted

this idea although in TrueCrypt’s case the implementation is simpler: each

encrypted volume has its free space overwritten with random data, and there

may or may not be a hidden volume in there that can be revealed to a user

with the right password.

Now TrueCrypt is one of the tools commonly used by human-rights workers;

would it be sensible for Firoz to use it too? The answer is, as usual, ‘it depends’.

If the Iranian religious police normally only ﬁnd TrueCrypt installed by

human-rights workers there, he’s likely to be treated as one of them if he’s

raided and it’s found. In general, if the existence of a product is in itself

incriminating, he might want to hide that too.

The fundamental problem facing forensic investigators, as I’ll discuss in

detail later in section 24.5, is the sheer volume of data found when searching a

house nowadays: there can be terabytes of data scattered over laptops, mobile

phones, cameras, iPods and memory sticks. If you don’t want a needle to be

found, build a larger haystack. So Firoz might have a lot of electronic junk

scattered around his apartment, as cover for the memory stick that actually

contains the forbidden pictures stashed in a hidden TrueCrypt container. He

might even have some straight porn in the ordinary encrypted volume, so

he’s got something to give the police if they torture him. And there are many

ad-hoc ways in which content can be made inaccessible to the casual searcher;

he might damage the memory stick in some repairable way. If he had a

forbidden movie in WMV format, he might delete its license from the WMP

store — so the license store had to be restored from backup before the movie

could be played. (A movie for which a license is no longer available is a much

less suspicious kind of ciphertext than a TrueCrypt volume.)

In short, as the world adapts to the digital age, people adopt ways of doing

things, and these procedures in turn have weak points, which leads us back

to tradecraft. What works will vary from one place and time to another,

as it depends on what the local opponents actually do. But there are some

principles. For example, anonymity loves company; it’s much easier to hide in

a crowd than in the middle of a desert. And in some applications, deniability

may be enough: Crowds was a system in which users group together and do

web page forwarding for each other, so that if one of them downloaded a

subversive web page, the secret police have several hundred suspects to deal

with [1067]. A similar scheme was devised by a well-known company CEO

who, each morning, used to borrow at random one of the mobile phones of

his managers, and have his switchboard forward his calls.

Forensics are subtly different: cops tend only to have tools for the most

popular products. They can usually search for ‘obvious’ wickedness in Win-

dows PCs but often can’t search Macs at all; they have tools to extract address

books from the three or four most popular mobile phones, but not obscure

makes; they can wiretap the large ISPs but often not the mom-and-pop outﬁts.

23.4 Privacy Technology 757

They’re also usually a bit behind the curve. They may know how to deal with

Facebook now, but they probably didn’t in 2004. Cool kids and gadget freaks

may always be a few steps ahead.

23.4.6 Putting It All Together

Returning now to our list of typical privacy technology users, what can we say?

1. The missionary, Andrew, has one of the hardest secure communication

tasks. He can’t meet his converts to train them to use Tor and PGP prop-

erly, and religious factors might prevent them communicating covertly

by joining an online computer game in which they played the roles of

dragons, wizards and so on. Perhaps the simplest solution for him is

Skype.

2. In the case of your daughter Betty, all the evidence is that parental con-

cerns over the Internet are grossly over-inﬂated. Rather than trying to

get her to surf the net using Tor (which she’d just consider to be creepy

if her friends don’t use it too), you’d be better to make scams, phishing

and other abuses into a regular but not obsessive topic of conversa-

tion round the dinner table. (Once she gets the conﬁdence to join in the

conversation, she may turn out to have better tales than you do.)

3. The corporate engineer, Charles, may ﬁnd his main risk is that if he posts

from a work machine, then even if he’s using a throwaway webmail

address, he might inadvertently include material in documents such as

local printer or workgroup names or a camera serial number that the

corporate security guy then ﬁnds on Google. The simplest solution is to

use home equipment that isn’t cross-contaminated with work material.

Essentially this is a multilevel policy of the sort discussed in Chapter 8.

4. The human-rights activist Dai has one of the hardest jobs of all, but as

she’s being regularly shaken down by the police and is in contact with

a network of other activists with whom she can share experiences, she

at least has an opportunity to evolve good tradecraft over time.

5. The M&A analyst Elizabeth may well ﬁnd that Tor does pretty well

what she needs. Her main problem will be using it properly (even I once

found that I’d misconﬁgured my system so that I thought I was brows-

ing through Tor when I wasn’t — and I’m supposed to be a security

expert).

6. Firoz is in a pretty bad way, and quite frankly were I in his situation I’d

emigrate. If that’s not possible then he should not just use Tor, but get a

Mac or Linux box so he’s less exposed to porn-site malware. Some com-

bination of cryptographic hiding, camouﬂage and deception may save

758 Chapter 23 ■The Bleeding Edge

his life if he gets raided by the police; and perhaps he should join the

Revolutionary Guard so the police won’t dare raid him in the ﬁrst place.

7. Graziano also has an extremely hard job. It’s bad enough defending a

covert network against one or two traitors at the client end (as Andrew

must); defending against occasional treachery at the server side is even

harder. Were I designing such a system I’d establish clear public poli-

cies and expectations on how informers’ identity would be protected,

so that any attempt by a future corrupt webmaster to subvert the pro-

cedures would be visible. I’d also test the system regularly by having

undercover policemen call in as informers, and track their revelations, to

spot bent cops who lose information. Where informers did identify

themselves — deliberately or accidentally — I’d ensure that only one

handler and his supervisor know this.

Wicked people use anonymity too, of course, and the many tales of how

they fail underline the difﬁculty of ﬁnding true anonymity in the modern

world. In a child-custody case in Taunton, England, the wife’s lawyer emailed

a bogus legal judgment to the father, pretending the email was from a fathers’

rights charity. When the father read this out in court, the lawyer stood up

and accused him of forgery. Outraged, the father tracked the email to a shop

in London’s Tottenham Court Road, where the staff remembered someone

coming in to use their service and dug out still images from their CCTV

camera which identiﬁed the lawyer, Bruce Hyman [397]. Mr Hyman was sent

to prison for twelve months at Bristol Crown Court. He was an expert on

evidence — and the ﬁrst British barrister to be imprisoned in modern times

for perverting the course of justice.

Richard Clayton wrote a thesis on anonymity and traceability in cyberspace,

which Mr Hyman should perhaps have read [300]. There are many ways

in which people who try to be anonymous, fail; and there are also many

ways in which even people who made no particular effort to hide themselves

end up not being traceable. It’s hard to establish responsibility when abusive

trafﬁc comes from a phone line in a multi-occupied student house, or when

someone accesses a dial-up account on a free ISP from a phone whose calling

line ID has been blocked. ISPs also often keep quite inadequate logs and

can’t trace abusive trafﬁc afterwards. So in practice, as opposed to theory,

anonymity is already pretty widespread. This may gradually contract over

time, because pressure over peer-to-peer trafﬁc, spam and phishing may make

ISPs manage things more tightly and respond better to complaints. The view

of UK ISPs, for example, is that ‘Anonymity should be explicitly supported by

relevant tools, rather than being present as a blanket status quo, open to use

and misuse’ [307].

23.5 Elections 759

As privacy technology evolves, it may modify the shape of the trade-off

between privacy and surveillance that is conditioned by the much larger-scale

development of online technology in general. Privacy technology will be driven

to some extent by the desire to evade copyright, by various political liberation

agendas, and by criminal innovation. Tools invented to protect the privacy of

the law-abiding, and of foreign lawbreakers whose subversion we support, will

be used occasionally by criminals in our countries too. So far, there’s little sign

of it, but it’s bound to happen eventually. For this reason a number of people

have proposed identity escrow schemes in which net users have pseudonyms

which normally provide identity protection but which can be removed by order

of a court [255]. But such systems would put most of the privacy users we dis-

cussed in this section directly in harm’s way. What’s more, escrow mechanisms

tend to be expensive and fragile, and cause unforeseen side-effects [4].

In the next chapter I’ll describe the ‘Crypto Wars’ — the long struggle

through the 1990s by governments to control cryptography, by demanding that

keys be escrowed. Eventually they gave up on that; and the same arguments

apply to anonymity systems. I believe we just have to accept that providing

privacy to people we approve of means that occasionally some people we don’t

approve of will use it too. As Whit Difﬁe, the inventor of digital signatures

and a leading crypto warrior, put it: ‘If you campaign for liberty you’re likely

to ﬁnd yourself drinking in bad company at the wrong end of the bar’.

23.5 Elections

One application of which all democracies by deﬁnition approve, and in which

almost all mandate anonymity, is the election. However, the move to electronic

voting has been highly controversial. In the USA, Congress voted almost four

billion dollars to upgrade states’ election systems following the Florida ﬁasco

in 2000, and a lot of this money’s been wasted on voting machines that turned

out to be insecure. There have been similar scandals elsewhere, including the

UK and the Netherlands.

Research into electronic election mechanisms goes back to the early 1980s,

when David Chaum invented digital cash — a payment medium that is anony-

mous, untraceable and unlinkable [285, 287]. In section 5.7.3 I described the

mechanism: a customer presents a banknote to a bank, blinded by a random

factor so that the bank can’t see the serial number; the bank signs the note;

the customer then unblinds it; and she now has an electronic banknote with a

serial number the bank doesn’t know. There are a few more details you have

to ﬁx to get a viable system, such as arranging that the customer’s anonymity

fails if she spends the banknote twice [287]. Digital cash hasn’t succeeded,

760 Chapter 23 ■The Bleeding Edge

as it’s not really compatible either with the anti-money-laundering regime or

the banks’ business models8. However the application on which a number of

research teams are still working is the digital election. The voter can be given

a ballot paper manufactured using the same general techniques as a digital

coin; she can spend it with the candidate of his choice; and she can get caught

if she cheats by double-spending.

There are a number of further requirements on electronic voting systems,

of which perhaps the two most difﬁcult to satisfy simultaneously are that

the voter should be able to satisfy herself that her vote has been properly

counted and that she should not be able to prove to anybody else how she

voted. If she can, then the doors are opened to vote-buying and intimidation.

Getting the anonymity and auditability right simultaneously depends on a

good combination of physical security and computer-security mechanisms.

Digital elections remained something of an academic backwater until 2000,

when the outcome of the U.S. Presidential election turned on a handful of

disputed votes in Florida. At the time, I was attending the Applications

Security conference in New Orleans, and we organised a debate; it rapidly

became clear that, even though politicians thought that mechanical or paper

voting systems should be replaced with electronic ones as quickly as possible,

security experts didn’t agree. A large majority of the attendees — including

many NSA and defense contractor staff — voted (on an old-fashioned show

of hands) they didn’t trust electronic elections9. Nonetheless Congress went

ahead and passed the Help America Vote Act in 2002, which provided $3.8

billion for states to update their voting equipment.

By the following year, this particular barrel of pork had degenerated into

a national scandal. Many problems were reported in the 2002 elections [551];

then, the following summer, the leading voting-machine supplier Diebold left

its voting system ﬁles on an open web site, a stunning security lapse. Avi

Rubin and colleagues at Johns Hopkins trawled through them found that

the equipment was far below even minimal standards of security expected

in other contexts. Voters could cast unlimited votes, insiders could identify

voters, and outsiders could also hack the system [731]. Almost on cue, Diebold

CEO Walden O’Dell, who was active in the campaign to re-elect President

Bush, wrote ‘I am committed to helping Ohio deliver its electoral votes to the

president next year’ [1320]. This led to uproar.

Electronic equipment had actually been used for some time to count ballots

in a number of U.S. districts, but there are a number of different ways to do

8A variant may be used for pseudonymous credentials in Trusted Computing [220].

9One of the strongest arguments was a simple question: do you know how to clear the Internet

Explorer cache? As the hotel didn’t have an Internet connection, we all had to check our email

at a caf´

e in Bourbon Street that had two PCs, one with Netscape and the other with IE. The

attendees preferred Netscape as it was easy to stop the next user retrieving your password from

the cache.

23.5 Elections 761

this. One option is optical scanning, where paper ballots are used but fed into a

system that recognises votes, gets an operator to adjudicate difﬁcult cases, and

outputs the tally. This has the advantage that, if the count is challenged, ofﬁcials

(or a court) can send for the original ballots and count them by hand. Another

alternative is the ballotmarking machine: the voter makes her choices on a touch

screen, after which the machine prints out a voting form that she can inspect

visually and drop into a ballot box. Many (but not all) of the problems arose

from ‘Direct-recording electronic’ (DRE) voting systems, in which the voter’s

choice is entered directly into a terminal that tallies the votes and outputs the

result at the end of the day. If the software in such a device is buggy, or open to

manipulation, it can give the wrong result; and unless the result is wildly out of

kilter with common sense, there’s simply no way to tell. The only veriﬁcation

procedure available on many models was to press the ‘count’ button again to

get it to print out the tally again. Even although voting machines are certiﬁed by

the Federal Election Commission (FEC), the FEC rules don’t require that a tally

be independently auditable. This is wrong, according to the majority of experts,

who now believe that all voting systems should have a voter-veriﬁable audit

trail. This happens automatically with scanning systems; Rebecca Mercuri

advocates that DRE equipment should display the voter’s choice on a paper

roll behind a window and get them to validate it prior to casting. (That was in

1992, and was reiterated in her thesis on electronic voting in 2000 [875, 876].)

The latest round in the U.S. voting saga comes from California, Florida

and Ohio. The Californian Secretary of State Debra Bowen authorized and

paid for a large team of computer scientists, led by University of California

professors David Wagner and Matt Bishop, to do a top-to-bottom evaluation

of the state’s voting systems, including source code reviews and red-team

attacks, in order to decide what equipment could be used in the 2008 elections.

The reports, published in May 2007, make depressing reading [215]. All

of the voting systems examined contained serious design ﬂaws that led

directly to speciﬁc vulnerabilities that attackers could exploit to affect election

outcomes. All of the previously approved voting machines — by Diebold, Hart

and Sequoia — had their certiﬁcation withdrawn, and were informed they

would need to undertake substantial remediation before recertiﬁcation. A late-

submitted system from ES&S was also decertiﬁed. California could still take

such radical action, as perhaps three-quarters of the 9 million people who voted

in 2004 did so using a paper or optical-scan ballot. As this book went to press in

December 2007, Ms Bowen had just said that electronic voting systems were

still not good enough to be trusted with state elections. ‘When the government

ﬁnds a car is unsafe, it orders a recall’, she said. ‘Here we’re talking about

systems used to cast and tally votes, the most basic tool of democracy’. [1343].

A similar inspection of Florida equipment was carried out by scientists at

Florida State University; they reported a bundle of new vulnerabilities in the

Diebold equipment in July 2007 [514]. Ohio followed suit; their evaluation

762 Chapter 23 ■The Bleeding Edge

of election equipment and standards came to similar conclusions. All the

evaluated equipment had serious security failings: data that should have

been encrypted wasn’t; encryption done badly (for example, the key stored

in the clear next to the ciphertext); buffer overﬂows; useless (and misapplied)

physical security; SQL injection; audit logs that could be tampered with; and

undocumented back doors [855]. Interestingly, the Florida and Ohio teams

found plenty of new vulnerabilities that the California team missed, and all

were working quickly; this raises interesting questions about the total number

of security ﬂaws in these systems.

Our experience in the UK is broadly similar, although the detail is different.

Tony Blair’s government progressively expanded the use of postal and other

absentee forms of ballot, which was criticised by opposition parties as it made

vote-buying and intimidation easier. Party workers (of which Blair’s Labour

party had more) could pressure voters into opting for a postal ballot, then

collect their ballot forms, ﬁll them out, and submit them. Plans to extend voting

from the post to email and text were criticised for making this existing low-

grade abuse easier and potentially open to automation. Finally, in the May 2007

local government elections, electronic voting pilots were held in eleven areas

around the UK. Two of my postdocs acted as scrutineers in the Bedford election,

and observed the same kind of shambles that had been reported at various U.S.

elections. The counting was slower than with paper; the system (optical-scan

software bought from Spain) had a high error rate, resulting in many more

ballots than expected being sent to human adjudicators for decision. (This was

because the printers had changed the ink halfway through the print run, and

half the ballot papers were the wrong shade of black.) Even worse, the software

sometimes sent the same ballot paper to multiple adjudicators, and it wasn’t

clear which of their decisions were counted. In the end, so that everyone could

go home, the returning ofﬁcer accepted a letter of assurance (written on the spot

by the vendor) saying that no vote would have been miscounted as a result.

Yet the exercise left the representatives from the various parties with serious

misgivings. The Open Rights Group, which organised the volunteers, reported

that it could not express conﬁdence in the results for the areas observed [987].

There was an interesting twist in the Netherlands. DRE voting machines

had been introduced progressively during the 1990s, and cyber-rights activists

were worried about the possibility of tampering and fraud along the lines

observed in the USA. They discovered that the machines from the leading

vendor, Nedap, were vulnerable to a Tempest attack: using simple equipment,

an observer sitting outside the polling station could see what party a voter had

selected [541]. From the security engineer’s perspective this was great stuff, as

it led to the declassiﬁcation by the German intelligence folks of a lot of Cold

War tempest material, as I discussed in section 17.4.2 (the Nedap machines are

also used in Germany). The activists also got a result: on October 1 2007 the

District Court in Amsterdam decertiﬁed all the Nedap machines.

23.5 Elections 763

As for other countries, the picture is mixed. The OpenNet Initiative (of

which I’ve been a member since 2006) monitors election abuses in the third

world. We have found that in some less-developed country elections, the state

has systematically censored opposition parties’ websites and run denial-of-

service attacks; in others (typically the most backward), elections are rigged

by more traditional methods such as kidnapping and murdering opposition

candidates. The evidence of electronic cheating is less clear-cut but is often

suspected. Take for example Russia. I wrote in the ﬁrst edition in 2001:

‘I sincerely hope that the election of Vladimir Putin as the president of Russia

had nothing to do with the fact that the national electoral reporting system

is run by FAPSI, a Russian signals intelligence agency formed in 1991 as the

successor to the KGB’s 8th and 16th directorates. Its head, General Starovoitov,

was reported to be an old KGB type; his agency reported directly to President

Yeltsin, who chose Putin as his successor’ [509, 678]. Yet by the time Putin’s

party was re-elected in 2007, the cheating had become so blatant — with gross

media bias and state employees ordered to vote for the ruling party — that the

international community would not accept the result as free and fair.

Wherever you go, electronic abuses at election time, and abuses of electronic

election equipment, are just one of many tools used by the powerful to hang

on to power. It’s worth remembering that in Florida in 2000, more voters

were disenfranchised as a result of registration abuses than there were ballots

disputed because of hanging chads. And just as the government can bias an

election by making it harder to vote if you haven’t got a car, it could conceivably

make it harder to vote if you haven’t got a computer. It’s not unknown for

the ballot to be made so complex as to disenfranchise the less educated.

And large-scale abuses can defeat even technical ballot privacy; for example,

in a number of less-developed countries, districts that elected the ‘wrong’

candidate have been punished. (And although we shake our heads in sorrow

when that happens in Zimbabwe, we just shrug when a British government

channels extra money to schools and hospitals in marginal constituences.) In

fact, it has struck me that if an incumbent wants to change not 1% of the votes,

but 10% — say to turn a 40–60 defeat into a 60–40 victory — then bribing

or bullying voters may provide a more robust result than tinkering with the

vote-tallying computer. Voters who’ve been bribed or bullied are less likely to

riot than voters who’ve been cheated. The bullied voters in Russia didn’t riot;

the cheated voters in Kenya did.

So high-technology cheating shouldn’t get all, or even most, of an election

monitor’s attention. But it needs to get some. And it behoves citizens to be

more sceptical than usual about the latest wizzo technology when it’s being

sold to us by politicians who hope to get reelected using it. Finally, even where

politicians have comfortable majorities and aren’t consciously trying to cheat,

they are often vulnerable to computer salesmen, as they’re scared of being

accused of technophobia. It takes geeks to have the conﬁdence to say stop!

764 Chapter 23 ■The Bleeding Edge

23.6 Summary

Some of the most challenging security engineering problems at the beginning

of the twenty-ﬁrst century have to do with the new online applications that are

sweeping the world, from online games through search and auctions to social

networking. This chapter was really just a helicopter tour of new services and

the new cheating strategies they’ve spawned.

Much of what goes wrong with online services, as with anonymity services

and digital elections, is just the same as we’ve seen elsewhere — the usual sad

litany of bugs and blunders, of protection requirements ignored in the rush to

market or just not understood by the developers of the early systems. Elections

in particular provide a sobering case history of proprietary systems developed

for an application that was known to be sensitive, and by developers who

made all sorts of security claims; yet once their products were exposed to fresh

air and sunlight, they turned out to be terrible.

What’s also starting to become clear is that as more and more of human life

moves online, so the criticality and the complexity of online applications grow

at the same time. Many of the familiar problems come back again and again,

in ever less tractable forms. Enforcing privacy is difﬁcult enough in a large

hospital, but just about doable. How do you enforce privacy in something

as big as Google, or as complex as Facebook? And how do you do security

architecture when ever more functionality is provided to ever more people

by ever more code written by constantly growing armies of inexperienced

programmers? Traditional software engineering tools helped developers get

ever further up the complexity mountain before they fell off. How do you see

to it that you don’t fall off, or if you do, you don’t fall too hard?

Research Problems

This leads me to suggest that one of the critical research problems between

now and the third edition of this book (if there is one) will be how protection

mechanisms scale.

The hard mechanism-design problem may be how one goes about evolving

‘security’ (or any other emergent property) in a socio-technical system with

billions of users. In the simple, million-user government applications of

yesteryear, a central authority could impose some elementary rules — a ‘Secret’

document had to be locked in a ﬁling cabinet when you went to the toilet,

and a ‘Secret’ computer system needed an Orange book evaluation. But such

rules were never natural, and people always had to break them to get their

Further Reading

The standard reference on game security at present is Greg Hoglund and

Gary McGraw’s book [617]. For the history of computer games, and cheat-

ing, read Jeff Yan and Brian Randell in [1367]; Jeff’s thesis discusses online

bridge [1364]. There’s an annual conference, NetGames, which usually has

a number of papers on cheating, and the Terra Nova blog on virtual

worlds has regular articles on cheating.

The best general work I know of on security in web services is Mike Andrews

and James Whitaker’s ‘How to Break Web Software’ [78]. There are many books

on speciﬁc services, such as John Battelle’s book on Google [125] and Adam

Cohen’s of eBay [310]; and if you need to explain the problem to management,

read (or give them) the article by Jeremy Epstein, Scott Matsumoto and Gary

McGraw about how web-service developers get software security wrong [436].

As for social networking, I don’t think the deﬁnitive book has come out yet.

As for privacy technology, the best introduction to anonymous remailers is

probably [849]. I don’t know of a good treatment of privacy and anonymity

technology in real-world contexts; the above vignettes of Andrew and others

are my own modest attempt to ﬁll that gap. To go into this subject in depth,

look through the anonymity bibliography maintained by Roger Dingledine,

Nick Matthewson and George Danezis [82], and the survey of anonymity

systems by George Danezis and Claudia Diaz [347]. For trafﬁc analysis, you

might start with the survey by Richard Clayton and George Danezis [346].

There’s now quite a literature on electronic voting. The issues are largely

the same as with voting by mail or by phone, but not quite. An early survey

of the requirements, and of the things that can go wrong, was written by Mike

766 Chapter 23 ■The Bleeding Edge

Shamos [1149], who is also a prominent defender of electronic voting [1150];

while Roy Saltzman (for many years the authority at NIST) discusses things that

have gone wrong in the USA, and various NIST recommendations, in [1103].

The leading critics of current electronic voting arrangements include David

Dill’s Veriﬁed Voting Foundation, and Rebecca Mercuri, whose 2000 thesis on

‘Electronic Vote Tabulation — Checks & Balances’ [876] might perhaps have

been heeded more, along with an early report on the feasibility of Internet

voting from the State of California [253]. Certainly, the recent evaluation

reports from California [215], Florida [514], Ohio [855] and Britain [987] lend

strong conﬁrmation to the sceptical viewpoint.

PART

III

In the ﬁnal part of the book I cover three themes:

politics, management and assurance. Given that we

now have some idea how to provide protection, the

three big questions are: what are you allowed to do?

How do you go about organizing it? And how do you

know when you’re done?

Since 9/11, we’ve seen the growth of a security-

industrial complex that has consumed billions of dol-

lars and caused great inconvenience to the public, for

often negligible gains in actual protection. Politicians

scare up the vote, and vendors help them with sys-

tems that are best described as ‘security theater’. This

gives rise to many difﬁcult political issues for the secu-

rity engineer. Are our societies vulnerable to terrorism

because we overreact, and if so, how can we avoid

becoming part of the problem rather than part of the

solution? Can we ﬁnd ways to make more rational

decisions about allocating protective resources, or at

least stop security arguments being used to bolster bad

policy? And how do these issues interact with the more

traditional security policy issues we already worried

about in the 1990s, such as surveillance, wiretapping,

the rules of digital evidence, censorship, export control

and privacy law?

Our next chapter is about management. This has

become a dirty word in the information security world;

there are endless vapid articles written in manage-

mentese which manage to say nothing at great length.

But management issues are important: organisational

768 Part III

and economic incentives often determine whether secure systems get built

or not. The growth of security economics as a discipline over the last six

years enables us to discuss these problems in a much more mature way than

previously, and the insights help us tackle the critical problem of how you go

about managing a team to develop a predictably dependable system.

Assurance is a huge political can of worms. On the face of it, it’s just an

engineering issue: how do you go about ﬁnding convincing answers to the

questions: are we building the right system? and, are we building it right?

These questions are familiar from software engineering (which can teach us

a lot), but they acquire new meaning when systems are exposed to hostile

attack. Also, most of the organisational structures within which assurance

claims can be made, or certiﬁed, are poisoned one way or another. Claims

about system security properties are often thinly veiled assertions of power and

control, so it should surprise no-one if the results of evaluation by equipment

makers, insurers’ laboratories, military agencies and academic attackers are

very different. So it’s really important for the security engineer to set out at the

start of a project not just what the objective is, but the criteria by which it will

be judged a success or a failure.

CHAPTER

Terror, Justice and Freedom

Al-Qaida spent $500,000 on the event, while America, in the incident and its

aftermath, lost — according to the lowest estimate — more than $500 billion.

— Osama bin Laden

Experience should teach us to be most on our guard to protect liberty when the

government’s purposes are beneﬁcient ... The greatest dangers to liberty lurk in

insidious encroachment by men of zeal, well meaning but without understanding.

— Supreme Court Justice Louis Brandeis

They that can give up essential liberty to obtain a little temporary safety deserve

neither liberty nor safety.

— Benjamin Franklin

24.1 Introduction

The attacks of September 11, 2001, on New York and Washington have had a

huge impact on the world of security engineering, and this impact has been

deepened by the later attacks on Madrid, London and elsewhere. As everyone

has surely realised by now — and as the quote from Osama bin Laden bluntly

spells out — modern terrorism works largely by provoking overreaction.

There are many thorny issues. First, there’s the political question: are

Western societies uniquely vulnerable — because we’re open societies with

democracy and a free press, whose interaction facilitates fearmongering — and

if so what (if anything) should we do about it? The attacks challenged our core

values — expressed in the USA as the Constitution, and in Europe as the Con-

vention on Human Rights. Our common heritage of democracy and the rule

of law, built slowly and painfully since the eighteenth century, might have

769

770 Chapter 24 ■Terror, Justice and Freedom

been thought well entrenched, especially after we defended it successfully in

the Cold War. Yet the aftermath of 9/11 saw one government after another

introducing authoritarian measures ranging from ﬁngerprinting at airports

through ID cards and large-scale surveillance to detention without trial and

even torture. Scant heed has been given to whether these measures would

actually be effective: we saw in Chapter 15 that the US-VISIT ﬁngerprinting

program didn’t work, and that given the false alarm rate of the underlying

technology it could never reasonably have been expected to work. We’ve not

merely compromised our principles; we’ve wasted billions on bad engineering,

and damaged whole industries. Can’t we ﬁnd better ways to defend freedom?

Second, there’s the economic question: why are such vast amounts of

money spent on security measures of little or no value? America alone has

spent over $14 bn on screening airline passengers without catching a single

terrorist — and it’s rather doubtful that the 9/11 tactics would ever work

again, as neither ﬂight crew nor passengers will ever be as passive again

(indeed, on 9/11, the tactics only worked for the ﬁrst 71 minutes). As I noted

in Chapter 1, well-known vulnerabilities in screening ground staff, reinforcing

cockpit doors and guarding aircraft on the ground overnight have been

ignored by the political leadership. Never mind that they could be ﬁxed for a

fraction of the cost of passenger screening: invisible measures don’t have the

political impact and can’t compete for budget dollars. So we spend a fortune

on measures that annoy passengers but make ﬂying no safer, and according

to a Harvard study don’t even meet the basic quality standards for other,

less-political, screening programs [801]. Is there any way — short of waiting

for more attacks — to establish protection priorities more sensibly?

Third, there are the effects on our industry. President Eisenhower warned

in his valedictory speech that ‘we must guard against the acquisition of

unwarranted inﬂuence, whether sought or unsought, by the military indus-

trial complex. The potential for the disastrous rise of misplaced power exists

and will persist’. In the wake of 9/11, we saw a frantic stampede by security

vendors, consultancy companies, and intelligence agencies hustling for pub-

licity, money and power. We’re seeing the emergence of a security-industrial

complex that’s capturing policy in the same ways that the defense industry

did at the start of the Cold War. One might have thought that technological

progress would have a positive effect on trade-offs between freedom and

security; that better sensors and smarter processing would shift the ROC curve

towards greater precision. Yet the real-world outcome seems often to be the

reverse. How is the civic-minded engineer to deal with this?

Fourth, technical security arguments are often used to bolster the case for

bad policy. All through the Irish terrorist campaign, the British police had to

charge arrested terrorist suspects within four days. But after 9/11, this was

quickly raised to 28 days; then the government said it needed 90 days, claiming

they might have difﬁculty decrypting data on PCs seized from suspects. That

24.2 Terrorism 771

argument turned out to be misleading: the real problem was police inefﬁciency

at managing forensics. Now if the police had just said ‘we need to hold suspects

for 90 days because we don’t have enough Somali interpreters’ then common

sense could have kicked in; Parliament might well have told them to use staff

from commercial translation agencies. But security technology arguments are

repeatedly used to bamboozle legislators, and engineers who work for ﬁrms

with lucrative government contracts may ﬁnd it difﬁcult to speak out.

Finally, there is the spillover into public policy on topics such as wiretapping,

surveillance and export control, that affect security engineers directly, and the

corresponding neglect of the more ‘civilian’ policy issues such as consumer

protection and liability. Even before 9/11, governments were struggling to

ﬁnd a role in cyberspace, and not doing a particularly good job of it. The attacks

and their aftermath have skewed their efforts in ways that raise pervasive and

sometimes quite difﬁcult issues of freedom and justice. Authoritarian be-

haviour by Western governments also provides an excuse for rules in places

from Burma to Zimbabwe to censor communications and spy on their citizens.

Now the falling costs of storage may have made increased surveillance

inevitable; but the ‘war on terror’ is exacerbating this and may be catalysing

deeper and faster changes that we’d have seen otherwise.

In this chapter, I’m going to look at terrorism, then discuss the directly

related questions of surveillance and control, before discussing some other IT

policy matters and trying to put the whole in context.

24.2 Terrorism

Political violence is nothing new; anthropologists have found that tribal

warfare was endemic among early humans, as indeed it is among chim-

panzees [777]. Terror has long been used to cow subject populations — by

the Maya, by the Inca, by William the Conqueror. Terrorism of the ‘modern’

sort goes back centuries: Guy Fawkes tried to blow up Britain’s Houses of

Parliament in 1605; his successors, the Irish Republican Army, ran a number

of campaigns against the UK. In the latest, from 1970–94, some three thousand

people died, and the IRA even blew up a hotel where Margaret Thatcher was

staying for a party conference, killing several of her colleagues. During the

Cold War the Russians supported not just the IRA but the Baader Meinhof

Gang in Germany and many others; the West armed and supported partisans

from France in World War 2, and jihadists ﬁghting the Soviets in Afghanistan

in the 1980s. Some terrorists, like Baader and Meinhof, ended up in jail, while

others — such as the IRA leaders Gerry Adams and Martin McGuinness, the

Irgun leader Menachim Begin, the French resistance leader Charles de Gaulle

and the African anti-colonial leaders Jomo Kenyatta, Robert Mugabe and

Nelson Mandela — ended up in ofﬁce.

772 Chapter 24 ■Terror, Justice and Freedom

What general lessons can be drawn from this history? Well, there’s good

news and bad news.

24.2.1 Causes of Political Violence

The ﬁrst piece of good news is that the trend in terrorist violence has been

steadily downward [909]. There were many insurgencies in the 1960s and 70s,

some ethnic, some anti-colonial, and some ideological. Many were ﬁnanced

by the Soviet Union or its allies as proxy conﬂicts in the Cold War, although a

handful (notably the Nicaraguan Contras and the resistance to the Soviets in

Afghanistan) were ﬁnanced by the West. The end of the Cold War removed

the motive and the money.

The second (and related) point is that the causes of civil conﬂict are mostly

economic. An inﬂuential study by Paul Collier and Anke Hoefﬂer for the World

Bank looked at wars from 1960-1999 to see whether they were caused largely

by grievances (such as high inequality, a lack of political rights, or ethnic and

religious divisions), or by greed (some rebellions are more economically viable

than others) [315]. The data show convincingly that grievances play little role;

the incidence of rebellion was largely determined by whether it could be

sustained. (Indeed, Cicero said two thousand years ago that ‘Endless money

forms the sinews of war’.) Thus the IRA campaign continued largely because

of support from the Soviet bloc and from Irish-Americans; when the former

vanished and the latter decided that terror was no longer acceptable, the guns

were put beyond use. Similarly, the conﬂict in Sierra Leone was driven by

conﬂict diamonds, the Tamil revolt in Sri Lanka by funds from ethnic Tamils

in the USA and India, and Al-Qaida was ﬁnanced by rich donors in the Gulf

states. So the economic analysis gives clear advice on how to deal with an

insurgency: cut off their money supply.

24.2.2 The Psychology of Political Violence

Less encouraging ﬁndings come from scholars of psychology, politics and

the media. I mentioned the affect heuristic in section 2.3.2: where people rely

on affect, or emotion, calculations of probability tend to be disregarded. The

prospect of a happy event, such as winning the lottery, will blind most people

to the long odds and the low expected return; similarly, a dreadful event, such

as a terrorist attack, will make most people disregard the fact that such events

are exceedingly rare [1189]. Most of the Americans who died as a result of

9/11 did so since then in car crashes, after deciding to drive rather than ﬂy.

There are other effects, too, at the border between psychology and culture.

A study of the psychology of terror by Tom Pyszczynski, Sheldon Solomon

and Jeff Greenberg looked at how people cope with the fear of death. They got

22 municipal court judges in Tucson, Arizona, to participate in an experiment

24.2 Terrorism 773

in which they were asked to set bail for a drug-addicted prostitute [1053].

They were all given a personality questionnaire ﬁrst, in which half were asked

questions such as ‘Please brieﬂy describe the emotions that the thought of

your own death arouses in you’ to remind them that we all die one day. The

judges for whom mortality had become salient set an average bail of $455

while the control group set an average bond of $50 — a huge effect for such

an experiment. Further experiments showed that the mortality-salience group

had not become mean: they were prepared to give larger rewards to citizens

who performed some public act. It turns out that the fear of death makes

people adhere more strongly to cultural norms and defend their worldview

much more vigorously.

Thinkers have long known that, given the transience of human existence

in such a large and strange cosmos, people search for meaning and perma-

nence through religion, through their children, through their creative works,

through their tribe and their nation. Different generations of philosophers

theorised about this in different languages, from the medieval ‘memento mori’

through psychonalysis to more recent writings on our need to assuage exis-

tential anxiety. Pyszczynski and his colleagues now provide an experimental

methodology to study this; it turns out, for example, that mortality salience

intensiﬁes altruism and the need for heroes. The 9/11 attacks brought mortal-

ity to the forefront of people’s minds, and were also an assault on symbols of

national and cultural pride. It was natural that the response included religion

(the highest level of church attendance since the 1950s), patriotism (in the form

of a high approval rating for the President), and intensiﬁed bigotry. It was also

natural that, as the memory of the attacks receded, society would repolarise

because of divergent core values. The analysis can also support constructive

suggestions: for example, a future national leader trying to keep a country

together following an attack could do well to constantly remind people what

they’re ﬁghting for. It turns out that, when they’re reminded that they’ll die

one day, both conservatives and liberals take a more polarised view of an

anti-American essay written by a foreign student — except in experiments

where they are ﬁrst reminded of the Constitution [1053].

Some countries have taken a bipartisan approach to terrorism — as when

Germany faced the Baader-Meinhof Gang, and Britain the IRA. In other

countries, politicians have given in to the temptation to use fearmongering to

get re-elected. The American political scientist John Mueller has documented

the Bush administration’s hyping of the terror threat in the campaign against

John Kerry [909]; here in the UK we saw Tony Blair proposing the introduction

of ID cards, in a move that brilliantly split the opposition Conservative party in

the run-up to the 2005 election (the authoritarian Tory leader Michael Howard

was in favour, but the libertarian majority in his shadow cabinet forced a

U-turn). How can we make sense of a world in which critical decisions, with

huge costs and impact, are made on such a basis?

774 Chapter 24 ■Terror, Justice and Freedom

24.2.3 The Role of Political Institutions

In fact, there’s a whole academic subject — public-choice economics — devoted

to explaining why governments act the way they do, and for which its founder

James Buchanan won the Nobel Prize in 1986. As he put it in his prize lecture,

‘Economists should cease proffering policy advice as if they were employed

by a benevolent despot, and they should look to the structure within which

political decisions are made’. Much government behaviour is easily explained

by the incentives facing individual public-sector decision makers. It’s natural

for ofﬁcials to build empires as they are ranked by their span of control

rather than, as in industry, by the proﬁts they generate. Similarly, politicians

maximise their chances of reelection rather than the abstract welfare of the

public. Understanding their decisions requires methodological individual-

ism — considering the incentives facing individual presidents, congressmen,

generals, police chiefs and newspaper editors, rather than the potential gains

or losses of a nation. We know it’s prudent to design institutions so that their

leaders’ incentives are aligned with its goals — we give company managers

stock options to make them act like shareholders. But this is harder in a polity.

How is the national interest to be deﬁned?

Public-choice scholars argue that both markets and politics are instruments

of exchange. In the former we seek to optimise our utility individually, while

in the latter we do the same but using collective actions to achieve goals

that we cannot attain in markets because of externalities or other failures.

The political process in turn is thus prone to speciﬁc types of failure, such as

deﬁcit ﬁnancing. Intergenerational bargaining is hard: it’s easy for politicians

to borrow money to buy votes now, and leave the bill with the next generation.

But then why do some countries have much worse public debt than others?

The short answer is that institutions matter. Political results depend critically

on the rules that constrain political action.

Although public-choice economics emerged in response to problems in

public ﬁnance in the 1960s, it has some clear lessons. Constitutions matter, as

they set the ground rules of the political game. So do administrative structures,

as ofﬁcials are self-interested agents too. In the UK, for example, the initial

response to 9/11 was to increase the budget for the security service; but this

hundred million dollars or so didn’t offer real pork to the security-industrial

complex. So all the pet projects got dusted off, and the political beauty contest

was won by a national ID card, a grandiose project that in its original form

would have cost £20 billion ($40 billion [809]). Observers of the Washington

scene have remarked that a similar dynamic may have been involved in the

decision to invade Iraq: although the 2001 invasion of Afghanistan had been

successful, it had not given much of a role to the Pentagon barons who’d spent

careers assembling ﬂeets of tanks, capital ships and ﬁghter-bombers. Cynics

remarked that it didn’t give much of a payoff to the defense industry either.

24.2 Terrorism 775

Similar things were said in the aftermath of World War 1, which was blamed

on the ‘merchants of death’. I suppose we will have to wait for the verdict of

the historians.

24.2.4 The Role of the Press

The third locus of concern must surely be the press. ‘If it bleeds, it leads’, as

the saying goes; bad news sells more papers than good. Editors want to sell

more papers, so they listen to the scariest versions of the story of the day.

For example, in 1997, I got some news coverage when I remarked that British

Telecom was spending hundreds of millions more on bug-ﬁxing than its

Korean counterpart: was BT wasting money, I asked, or was the infrastructure

in middle-income countries at risk? In 1999, after we’d checked out all the

university systems, I concluded that although some stuff would break, none

of it really mattered. I wrote up a paper and got the University to send out

a press release telling people not to worry. There was an almost total lack of

interest. There were doomsayers on TV right up till the last midnight of 1999;

but ‘We’re not all going to die’ just isn’t a story.

The self-interest of media owners combines with that of politicians who

want to get re-elected, ofﬁcials who want to build empires, and vendors

who want to sell security stuff. They pick up on, and amplify, the temporary

blip in patriotism and the need for heroes that terrorist attacks naturally instil.

Fearmongering gets politicians on the front page and helps them control the

agenda so that their opponents are always off-balance and following along

behind.

24.2.5 The Democratic Response

Is this a counsel of despair? I don’t think so: people learn over time. On

the 7th July 2005, four suicide bombers killed 52 people on London’s public

transport and injured about 700. The initial response of the public was one

of gritty resignation: ‘Oh, well, we knew something like this was going to

happen — bad luck if you were there, but life goes on.’1The psychological

effect on the population was much less than that of the 9/11 bombings on

America — which must surely be due to a quarter-century of IRA bombings.

Both bombers and fearmongers lose their impact over time.

And as populations learn, so will political elites. John Mueller has written

a history of the attitudes to terrorism of successive U.S. administrations [909].

Presidents Kennedy, Johnson, Nixon and Ford ignored terrorism. President

1One curious thing was that the press went along with this for a couple of days: then there was

an explosion of fearmongering. It seems that ministers needed a day or two of meetings to sort

out their shopping lists and decide what they would try to shake out of Parliament.

776 Chapter 24 ■Terror, Justice and Freedom

Carter made a big deal of the Iran hostage crisis, and like 9/11 it gave him a

huge boost in the polls at the beginning, but later it ended his presidency. His

Secretary of State Cyrus Vance later admitted they should have played down

the crisis rather than giving undeserved credibility to the ‘students’ who’d

kidnapped U.S. diplomats. President Reagan mostly ignored provocations,

but succumbed to temptation over the Lebanese hostages and shipped arms

to Iran to secure their release. However, once he’d distanced himself from this

error, his ratings recovered quickly. Now President Bush’s fear-based policies

have led to serious problems round the world and tumbling popularity; the

contenders for the 2008 election all propose policy changes of various kinds.

Much the same has happened in the U.K., where Tony Blair’s departure from

ofﬁce was met with a great sigh of relief and a rebound in the polls for the

ruling Labour Party. His successor Gordon Brown has forbidden ministers to

use the phrase ‘war on terror’. The message is getting through: fearmongering

can bring spectacular short-term political gains, but the voters eventually see

through it. And just as this book went to press, in early January 2008, the

voters of Iowa selected a Republican candidate who says ‘The quickest way to

get out of Iraq is to win’, and a Democract who promises to end the war in Iraq

and be a President ‘who understands that 9/11 is not a way to scare up votes

but a challenge that should unite America and the world against the common

threats of the 21st century’. So it looks like the voters will get their say.

24.3 Surveillance

One of the side-effects of 9/11 has been a huge increase in technical surveil-

lance, both by wiretapping and through the mining of commercial data sources

by government agencies. Recent disclosures of unlawful surveillance in a num-

ber of countries, together with differing U.S. and European views on privacy,

have politicised matters. Wiretapping was already an issue in the 1990s, and

millions of words have been written about it. In this section, all I can reasonably

try to provide is a helicopter tour: to place the debate in context, sketch what’s

going on, and provide pointers to primary sources.

24.3.1 The History of Government Wiretapping

Rulers have always tried to control communications. In classical times, couriers

were checked at customs posts, and from the Middle Ages, many kings either

operated a postal monopoly or granted it to a crony. The letter opening and

codebreaking facilities of early modern states, the so-called Black Chambers, are

described in David Kahn’s history [676].

The invention of electronic communications brought forth a defensive

response. In most of Europe, the telegraph service was set up as part of the

24.3 Surveillance 777

Post Ofﬁce and was always owned by the government. Even where it wasn’t,

regulation was usually so tight that the industry’s growth was severely

hampered, leaving America with a clear competitive advantage. A profusion

of national rules, which sometimes clashed with each other, so exasperated

everyone that the International Telegraph Union (ITU) was set up in 1865 [1215].

This didn’t satisfy everyone. In Britain, the telegraph industry was nationalized

by Gladstone in 1869.

The invention of the telephone further increased both government interest

in surveillance and resistance to it, both legal and technical. In the USA, the

Supreme Court ruled in 1928 in Olmstead vs United States that wiretapping

didn’t violate the fourth amendment provisions on search and seizure as there

was no physical breach of a dwelling; Judge Brandeis famously dissented.

In 1967, the Court reversed itself in Katz vs United States, ruling that the

amendment protects people, not places. The following year, Congress legalized

Federal wiretapping (in ‘title III’ of the Omnibus Crime Control and Safe Streets

Act) following testimony on the scale of organized crime. In 1978, following

an investigation into the Nixon administration’s abuses, Congress passed

the Federal Intelligence Surveillance Act (FISA), which controls wiretapping

for national security. In 1986, the Electronic Communications Protection Act

(ECPA) relaxed the Title III warrant provisions. By the early 1990s, the spread

of deregulated services from mobile phones to call forwarding had started

to undermine the authorities’ ability to implement wiretaps, as did technical

developments such as out-of-band signaling and adaptive echo cancellation in

modems.

So in 1994 the Communications Assistance for Law Enforcement Act

(CALEA) required all communications companies to make their networks

tappable in ways approved by the FBI. By 1999, over 2,450,000 telephone

conversations were legally tapped following 1,350 court orders [434, 851]. The

relevant law is 18 USC (U.S. Code) 2510–2521 for telco services, while FISA’s

regulation of foreign intelligence gathering is now codiﬁed in U.S. law as 50

USC 1801–1811 [1272].

Even before 9/11, some serious analysts believed that there were at least

as many unauthorized wiretaps as authorized ones [387]. First, there’s phone

company collusion: while a phone company must give the police access if

they present a warrant, in many countries they are also allowed to give

access otherwise — and there have been many reports over the years of phone

companies being cosy with the government. Second, there’s intelligence-

agency collusion: if the NSA wants to wiretap an American citizen without

a warrant they can get an ally to do it, and return the favour later (it’s said

that Margaret Thatcher used the Canadian intelligence services to wiretap

ministers who were suspected of disloyalty) [496]. Third, in some countries,

wiretapping is uncontrolled if one of the subscribers consents — so that calls

from phone boxes are free to tap (the owner of the phone box is considered to

778 Chapter 24 ■Terror, Justice and Freedom

be the legal subscriber). Finally, in many countries, the police get hold of email

and other stored communications by subpoena rather than warrant (they used

to do this in America too before a court stopped the practice in June 2007 [795]).

But even if the ofﬁcial ﬁgures had to be doubled or tripled, democratic

regimes used wiretapping very much less than authoritarian ones. For

example, lawful wiretapping amounted to 63,243 line-days in the USA in

1999, or an average of just over 173 taps in operation on an average day.

The former East Germany had some 25,000 telephone taps in place, despite

having a fraction of the U.S. population [474]. There was also extensive use

of technical surveillance measures such as room bugs and body wires. (It’s

hardly surprising that nudist resorts became extremely popular in that sad

country.)

The incidence of wiretapping was also highly variable within and between

democracies. In the USA, for example, only about half the states used it, and

for many years the bulk of the taps were in the ‘Maﬁa’ states of New York, New

Jersey and Florida (though recently, Pennsylvania and California have caught

up) [582]. There is similar variation in Europe. Wiretaps are very common

in the Netherlands, despite Dutch liberalism on other issues [248]: they have

up to 1,000 taps on the go at once with a tenth of America’s population.

In a homicide investigation, for example, it’s routine to tap everyone in the

victim’s address book for a week to monitor how they react to the news of

the death. The developed country with the most wiretaps is Italy, thanks to its

tradition of organised crime [794]. In the UK, wiretaps are supposed to need

a ministerial warrant, and are rarer; but police use room bugs and similar

techniques (including computer exploits) quite a lot in serious cases. To some

extent, the technologies are interchangeable: if you can mount a rootkit in

a gangster’s laptop you can record, and mail home, everything said nearby,

whether it’s said to someone in the same room, or over a phone.

The cost of wiretapping is an issue. Before CALEA was introduced, in

1993, U.S. police agencies spent only $51.7 million on wiretaps — perhaps a

good estimate of their value before the issue became politicised [582]. The

implementation of CALEA has supposedly cost over $500 m, even though

it doesn’t cover ISPs. The FCC has recently (2006–7) extended the CALEA

rules to VOIP, which has provoked much grumbling from the industry about

the added costs of compliance, loss of some of the ﬂexibility which IP-based

services offer, loss of opportunities to innovate, and potential security problems

with VOIP services. Certainly it’s a lot harder to wiretap VOIP calls: as the

critics point out, ‘The paradigm of VoIP intercept difﬁculty is a call between

two road warriors who constantly change locations and who, for example,

may call from a cafe in Boston to a hotel room in Paris and an hour later

from an ofﬁce in Cambridge to a giftshop at the Louvre’ [156]. So how can

policymarkers ﬁgure out whether it’s worth it? If the agencies had to face the

full economic costs of wiretapping, would they cut back and spend the money

24.3 Surveillance 779

on more gumshoes instead? Once you start molding an infrastructure to meet

requirements other than cost and efﬁciency, someone has to pay: and as the

infrastructure gets more complex, the bills keep on mounting. If other people

have to pay them, the incentives are perverted and inefﬁciency can become

structural.

Since 9/11, though, economic arguments about surveillance have been more

or less suspended. 43 days after the attacks, Congress passed the Patriot Act,

which facilitated electronic surveillance in a number of ways; for example,

it allowed increased access by law enforcement to stored records (including

ﬁnancial, medical and government records), ‘sneak-and-peek’ searches of

homes and businesses without the owner’s knowledge, and the use by the

FBI of National Security Letters to get access to ﬁnancial, email and telephone

records. While access to email is often wiretapping, access to call records is

really trafﬁc analysis, which I’ll deal with in the next section, and may account

for most of the actual volume of interception.

The result has been a steady increase in wiretapping rather than a step

change. The 1350 wiretaps authorized by State and Federal courts in 1999 fell

to 1190 in 2000, rose to 1491 in 2001, fell to 1358 in 2002, and then rose to 1,442

in 2003, 1,710 in 2004 and 1,773 in 2005. There has been a sharper increase in

FISA warrants, from 934 in 2001 to 1228 in 2002, 1724 in 2003 and eventually

2176 in 2006 [435]. This reﬂects the greater interest in foreign nationals and the

Patriot Act’s provision that FISA warrants could be used in national-security

cases. (These used to be extremely rare: in 1998, for example, only 45 of the

FBI’s 12,730 convictions involved what the Justice Department classiﬁed as

internal security or terrorism matters [1259]).

In December 2005, the New York Times revealed that the NSA had been

illegally wiretapping people in the U.S. without a warrant. The Administration

proposed a rewrite of FISA to legalise this activity, the result was the recently

enacted ‘Protect America Act’, which amends the FISA and sunsets early in

2008. Under this Act, the NSA no longer needs even a FISA warrant to tap a call

if one party’s believed to be outside the USA or a non-U.S. person. This in effect

allowed warrantless surveillance of large numbers of U.S. calls. However, due

to the very visible dispute between the President and the Congress over U.S.

wiretap law, it’s not clear whether and how Congress will revise this when

it comes up for renewal. The current action (October 2007) is about granting

retrospective immunity to phone companies who cooperated with unlawful

wiretapping activities.

24.3.2 The Growing Controversy about Trafﬁc Analysis

However the great bulk of police communications intelligence in developed

countries does not come from the surveillance of content, but the analysis

of telephone toll records and other communications data. I examined in the

780 Chapter 24 ■Terror, Justice and Freedom

chapter on telecomms security how criminals go to great lengths to bury their

signals in innocuous trafﬁc using techniques such as pre-paid mobile phones

and PBX hacking; and the techniques used by the police to trace networks of

criminal contacts nonetheless.

Again, this is nothing new. Rulers have long used their control over postal

services to track the correspondents of suspects, even when the letters weren’t

opened. The introduction of postage stamps in 1840 was an advance for privacy

as it made it much easier to send a letter anonymously. Some countries got so

worried about the threat of sedition and libel that they passed laws requiring

a return address to be written on the back of the envelope. The development

of the telegraph, on the other hand, was an advance for surveillance; as

messages were logged by sender, receiver and word count, trafﬁc totals

could be compiled and were found to be an effective indicator of economic

activity [1215]. The First World War brought home to the combatants the value

of the intelligence that could be gleaned from listening to the volume of enemy

radio trafﬁc, even when it couldn’t conveniently be deciphered [676, 923].

Later conﬂicts reinforced this.

Trafﬁc analysis continues to provide the bulk of police communications

intelligence. For example, in the USA, there were 1,329 wiretap applications

approved in 1998 (the last year for which comparable statistics were available

when I wrote the ﬁrst edition of this book) while there were 4886 subpoenas

(plus 4621 extensions) for pen registers (devices which record all the numbers

dialed from a particular phone line) and 2437 subpoenas (plus 2770 extensions)

for trap-and-trace devices (which record the calling line ID of incoming calls,

even if the caller tries to block it). In other words, there were more than

ten times as many warrants for communications data as for content. What’s

more, these data were even more incomplete than for wiretapping. The trend

in recent years — even before 9/11 — was to switch to using subpoenas for

the call-detail records in the phone companies’ databases, rather than getting

pen-register data directly from the switch (a move facilitated by CALEA).

Bell Atlantic, for example, indicated that for the years 1989 through 1992, it

had responded to 25,453 subpoenas or court orders for toll billing records of

213,821 of its customers, while NYNEX reported that it had processed 25,510

subpoenas covering an unrecorded number of customers in 1992 alone [279].

Scaled up across the country, this suggests perhaps half a million customers

are having their records seized every year, and that trafﬁc data are collected

on perhaps a hundred times as many people as are subjected to wiretapping.

Statistics have become much more patchy and sporadic since 9/11, but there’s

no reason to believe that trafﬁc data have become less important: they have

been more important for years, and across many countries. (Indeed, recently

we’re getting indications of further qualitative increases in trafﬁc surveillance,

which I’ll discuss below.) Why should this be?

24.3 Surveillance 781

Wiretaps are so expensive to listen to and transcribe that most police forces

use them only as a weapon of last resort. In contrast, the numbers a suspect

calls, and that call him, give a rapid overview of his pattern of contacts.

Also, while wiretaps usually have fairly strict warrantry requirements, most

countries impose little or no restrictions on the police use of communications

data. In the USA, no paperwork was required until ECPA. Even after that,

they have been easy to get: under 18 USC 3123 [1272], the investigative ofﬁcer

merely had to certify to a magistrate ‘that the information likely to be obtained

by such installation and use is relevant to an ongoing criminal investigation’.

This can be any crime — felony or misdemeanour — and under either Federal

or State law. Unlike with wiretaps, the court has no power to deny a subpoena

once a formally correct application has been made, and there is no court

supervision once the order has been granted. Since the passage of CALEA,

warrants are still required for such communications data as the addresses to

which a subscriber has sent e-mail messages, but basic toll records could be

obtained under subpoena — and the subscriber need not be notiﬁed.

The most controversial current issue may be access to multiple generations

of call data and indeed to whole phone-company databases. In section 19.3.1,

I described the snowball search, in which the investigator not only looks at

who the target calls, but who they call, and so on recursively, accumulating

n-th generation contacts like a snowball rolling downhill, and then eventually

sifting the snowball for known suspects or suspicious patterns. If a pen-

search, the administrative costs mount up. There were thus rumours in many

countries for years that the phone companies simply give the intelligence

services access to (or even copies of) their databases.

24.3.3 Unlawful Surveillance

In 2006, it emerged that the rumours were correct. AT&T had indeed given the

NSA the call records of millions of Americans; the agency’s goal is ‘to create a

database of every call ever made within the nation’s borders’ so it can map the

entire U.S. social network for the War on Terror [277]. Apparently this data has

now been collected for the 200 m customers of AT&T, Verizon and BellSouth,

the nation’s three biggest phone companies. The program started just after

9/11. Qwest did not cooperate, because its CEO at the time, Joe Nacchio, did

not believe the NSA’s claim that Qwest didn’t need a court order (or approval

under FISA). The NSA put pressure on Qwest by threatening to withhold

classiﬁed contracts, and Qwest’s lawyers asked NSA to take its proposal to

the FISA court. The NSA refused, saying the court might not agree with them.

It’s since emerged that the NSA had put pressure on Qwest to hand over data

even before 9/11 [528]. In October 2007, further conﬁrmation was obtained by

Democrat senators when Verizon admitted to them that it had given the FBI

782 Chapter 24 ■Terror, Justice and Freedom

second-generation call data on its customers against national security letters

on 720 occasions since 2005 [925]; and in November 2007, the Washington Post

revealed that the NSA had tapped a lot of purely domestic phone calls and

trafﬁc data, and had also tapped AT&T’s peering centre in San Francisco to

get access to Internet trafﬁc as well [926].

Both phone and computer service records can be provided to bodies other

than law enforcement agencies under 18 USC 2703(c); thus, for example, we

ﬁnd Virginia and Maryland planning to use mobile phone tracking data to

monitor congestion on the Capital Beltway [1179]. Toll data use for marketing

purposes was also expressly envisioned by Congress when this law was

passed. However, the growing availability of mobile phone records has now

made them available to criminals too, enabling gangsters to track targets and

ﬁnd out if any of their colleagues have been calling the police [830].

In the UK, ﬁles of telephone toll tickets were provided to the police without

any control whatsoever until European law forced the government to regulate

the practice in the Regulation of Investigatory Powers Act in 2000. It was long

rumoured that the large phone companies gave the spooks copies of their

itemised billing databases as a matter of course. Since then, communications

data requires only a notice from a senior police ofﬁcer to the phone company

or ISP, not a warrant; and data can be provided to a wide range of public-sector

bodies, just as in the USA. (There was a public outcry when the Government

published regulations under the Act, which made clear that your mobile

phone records could be seized by anyone from your parish council to the Egg

Marketing Board.)

24.3.4 Access to Search Terms and Location Data

One problem is that communications data and content are becoming inter-

mixed, as what’s content at one level of abstraction is often communications

data at the next. People might think of a URL is just the address of a page to be

fetched, but a URL such as http://www.google.com/search?q=marijuana+

cultivation+UK contains the terms entered into a search engine as well as the

search engine’s name. Clearly some policemen would like a list of everyone

who submitted such an enquiry. Equally clearly, giving this sort of data to the

police on a large scale would have a chilling effect on online discourse.

In the USA, the Department of Justice issued a subpoena to a number of

search engines to hand over two full months’ worth of search queries, as well

as all the URLs in their index, claiming it needed the data to bolster its claims

that the Child Online Protection Act did not violate the constitution and that

ﬁltering could be effective against child pornography. (Recall we discussed in

section 9.3.1 how when AOL released some search histories, a number of them

were easily identiﬁable to individuals.) AOL, Microsoft and Yahoo quietly

complied, but Google resisted. A judge ruled that the Department would get

24.3 Surveillance 783

no search queries, and only a random sample of 50,000 of the URLs it had

originally sought [1353].

In the UK, the government tried to deﬁne URLs as trafﬁc data when it

was pushing the RIP bill through parliament, and the news that the police

would have unrestricted access to the URLs each user enters — their click-

stream — caused a public outcry against ‘Big Browser’, and the deﬁnition of

communications data was trimmed. For general Internet trafﬁc, it means IP

addresses, but it also includes email addresses. All this can be demanded with

only a notice from a senior policeman.

More subtleties arise with the phone system. In Britain, all information about

the location of mobile phones counts as trafﬁc data, and ofﬁcials get it easily;

but in the USA, the Court of Appeals ruled in 2000 that when the police get a

warrant for the location of a mobile, the cell in which it is active is sufﬁcient,

and that to require triangulation on the device (an interpretation the police

had wanted) would invade privacy [1273]. Also, even cell-granularity location

information would not be available under the lower standards applied to

pen-register subpoenas. Subpoenas were also found insufﬁcient for post-cut-

through dialed digits as there is no way to distinguish in advance from digits

dialed to route calls and digits dialed to access or give information. What this

means in practice is that if a target goes down a 7–11 store and buys a phone

card for a few dollars, the police can’t get a list of who he calls without a full

wiretap warrant. All they can get by subpoena are the digits he dials to contact

the phone card operator, not the digits he dials afterwards to be connected.

24.3.5 Data Mining

The analysis of call data is only one aspect of a much wider issue: law

enforcement data matching, namely the processing of data from numerous

sources. The earliest serious use of multiple source data appears to have

been in Germany in the late 1970s to track down safe houses used by the

Baader Meinhof terrorist group. Investigators looked for rented apartments

with irregular peaks in utility usage, and for which the rent and electricity bills

were paid by remote credit transfer from a series of different locations. This

worked: it yielded a list of several hundred apartments among which were

several safe houses. The tools to do this kind of analysis are now shipped with

a number of the products used for trafﬁc analysis and for the management

of major police investigations. The extent to which they’re used depends on

the local regulatory climate; there have been rows in the UK over police

access to databases of the prescriptions ﬁlled by pharmacists, while in the USA

doctors are alarmed at the frequency with which personal health information is

subpoenaed from health insurance companies by investigators. There are also

practical limits imposed by the cost of understanding the many proprietary

data formats used by commercial and government data processors. But it’s

784 Chapter 24 ■Terror, Justice and Freedom

common for police to have access at least to utility data, such as electricity bills

which get trawled to ﬁnd marijuana growers.

However, there are many indications that the combination of more aggres-

sive searches and mounting data volumes are making data-mining operations

since 9/11 less productive. Terrorists are just so rare as a percentage of the

population that any tests you use to ‘detect’ them would require extraordinary

sensitivity if you’re not to drown in false positives. Adding more data doesn’t

necessarily help; as I explained in section 15.9, combining multiple sensors is

hard and you’re unlikely to improve both the false positive and false negative

error rates at the same time. Simply put, if you’re looking for a needle in a

haystack, the last thing you need to do is to build a bigger haystack. As Jeff

Jonas, the chief scientist at IBM’s data-mining operation, put it, ‘techniques

that look at people’s behavior to predict terrorist intent are so far from reaching

the level of accuracy that’s necessary that I see them as nothing but civil liberty

infringement engines’ [519].

Finally, policemen (and even divorce lawyers) are increasingly using sub-

poenas to get hold of email from service providers once the recipient has

read it. The legal reasoning is that whereas it takes an interception warrant to

get the postman to hand over physical mail, a simple search warrant will do

once the letter lands on your doormat; and so although a proper warrant is

needed to seize email on its way through an ISP to you, once it’s sitting in your

mail folder at AOL or Google it’s just stored data. You might think it prudent

to use a mail service provider that deletes mail once you’ve read it; but in the

UK at least, a court found that police who ordered an ISP to preserve email that

they’d normally overwritten were acting lawfully [974], and in March 2006

the European Union adopted a Directive compelling Member States to enact

laws compelling communication services to retain trafﬁc data for between

six months and two years. It’s unclear how many ISPs will go to the trouble

of deleting email contents; if they have to retain the headers anyway, they

might as well keep the lot. And in the long term, absolutely anything that gets

monitored and logged is potentially liable to be subpoenaed.

24.3.6 Surveillance via ISPs – Carnivore

and its Offspring

One big recent development is intrusive surveillance at Internet Service

Providers (ISPs). Tapping data trafﬁc is harder than voice used to be; there

are many obstacles, such as transient IP addresses given to most customers

and the increasingly distributed nature of trafﬁc. In the old days (say 2002), an

ISP might have had modem racks, and a LAN where a wiretap device could

be located; nowadays many customers come in via DSL, and providers use

switched networks that often don’t have any obvious place to put a tap.

24.3 Surveillance 785

Many countries have laws requiring ISPs to facilitate wiretapping, and the

usual way to do it at a large ISP is to have equipment already installed

that will split a target network so that copies of packets of interest go to

a separate classiﬁed network with wiretap equipment. Small ISPs tend not

to have such facilities. In the late 1990s, the FBI developed a system called

Carnivore that they could lug around to smaller ISPs when a wiretap was

needed; it was a PC with software that could be conﬁgured to record a suspect’s

email, or web browsing, or whatever trafﬁc a warrant or subpoena speciﬁed.

It became controversial in 2000 when an ISP challenged it in court; it was

being used to record email headers as trafﬁc data, without a wiretap warrant.

Congress legalized this practice in the Patriot Act in 2001, and in about 2002

Carnivore was retired in favour of more modern equipment. We have recent

FOI revelations about the FBI’s current wiretapping network, DCSNet, which

is very slick –allowing agents remote and near-instantaneous access to trafﬁc

and content from participating phone companies [1178].

Access by the police and intelligence services to ISPs is patchy for a number

of reasons. No-one bothers about small ISPs, but they can grow quickly; large

ISPs’ systems can be hard to integrate with law-enforcement kit, and the

project can remain stuck in the development backlog for years as it brings no

revenue; ISPs coming into contact with the world of surveillance for the ﬁrst

time usually don’t have cleared staff to operate government equipment; and

the wiretap equipment is very often poorly engineered [1151]. As a result, it’s

often not practical for the police to tap particular ISPs for months or even years

on end, and the information about which providers are wiretap-resistant is

rather closely held. (Smart bad guys still use small ISPs.) In addition, it is often

difﬁcult for the authorities to get IP trafﬁc data without a full wiretap warrant;

for assorted technical reasons, all the trafﬁc data that it’s usually convenient

to provide against a subpoena are extracts from those logs that the ISP keeps

anyway. And things often go wrong because the police don’t understand ISPs;

they subpoena the wrong things, or provide inaccurate timestamps so that the

wrong user is associated with an IP address. For an analysis of failure modes,

see Clayton [300].

24.3.7 Communications Intelligence on Foreign Targets

I discussed the technical aspects of signals intelligence in Chapter 19; now let’s

look brieﬂy at the political and organizational aspects.

The bulk of communications intelligence, whether involving wiretaps, trafﬁc

analysis or other techniques, is not conducted for law enforcement purposes

but for foreign intelligence. In the U.S. the main agency responsible for this is

the National Security Agency, which has huge facilities and tens of thousands

of employees. While law enforcement agencies have 150–200 active wiretaps

at any one time, the NSA utterly dwarfs this. The situation is similar in other

786 Chapter 24 ■Terror, Justice and Freedom

countries; Britain’s Government Communications Headquarters (GCHQ) has

thousands of employees and a budget of about a billion dollars, while for

many years one single police ofﬁcer at New Scotland Yard handled the

administration of all the police wiretaps in London (and did other things too).

Information has steadily trickled out about the scale and effectiveness

of modern signals intelligence operations. David Kahn’s inﬂuential history of

cryptography laid the groundwork by describing much of what happened

up till the start of World War Two [676]; an anonymous former NSA analyst,

later identiﬁed as Perry Fellwock, revealed the scale of NSA operations in

1972 [462]. ‘Information gathering by NSA is complete’, he wrote. ‘It covers

what foreign governments are doing, planning to do, have done in the

past: what armies are moving where and against whom; what air forces are

moving where, and what their capabilities are. There really aren’t any limits

on NSA. Its mission goes all the way from calling in the B-52s in Vietnam to

monitoring every aspect of the Soviet space program’.

While Fellwock’s motive was opposition to Vietnam, the next major whistle-

blower was a British wartime codebreaker, Frederick Winterbotham, who

wanted to write a memoir of his wartime achievements and, as he was dying,

was not bothered about prosecution. In 1974, he revealed the Allies’ success in

breaking German and Japanese cipher systems during that war [1350], which

led to many further books on World War 2 sigint [296, 677, 1336]. Thereafter

there was a slow drip of revelations by investigative journalists, quite of few

of whose sources were concerned about corruption or abuse of the facilities by

ofﬁcials monitoring targets they should not have, such as domestic political

groups. For example, whistleblower Peg Newsham revealed that the NSA had

illegally tapped a phone call made by Senator Strom Thurmond [258, 259].

James Bamford pieced together a fair amount of information on the NSA from

open sources and by talking to former employees [112], while the most substan-

tial recent source on the organization and methods of U.S. and allied signals

intelligence was put together by New Zealand journalist Nicky Hager [576]

following the New Zealand intelligence community’s failure to obey an order

from their Prime Minister to downgrade intelligence cooperation with the USA.

The end of the Cold War forced the agencies to ﬁnd new reasons to justify

their budgets, and a common theme was developing economic intelligence

operations against competitor countries. This accelerated the ﬂow of informa-

tion about sources and methods. The most high-proﬁle expos´

e of US economic

espionage was made in a 1999 report to the European parliament [443], which

was concerned that after the collapse of the USSR, European Union member

nations were becoming the NSA’s main targets [262].

The picture that emerged from these sources was of a worldwide signals

intelligence collection system, Echelon, run jointly by the USA, the UK, Canada,

Australia and New Zealand. Data, faxes and phone calls get collected at a large

number of nodes which range from international communications cables that

24.3 Surveillance 787

land in member countries (or are tapped clandestinely underwater), through

observation of trafﬁc to and from commercial communications satellites and

special sigint satellites that collect trafﬁc over hostile countries, to listening

posts in member states’ embassies [443]. The collected trafﬁc is searched in

real time by computers known as dictionaries according to criteria such as

the phone numbers or IP addresses of the sender or receiver, plus keyword

search on the contents of email. These search criteria are entered by member

countries’ intelligence analysts; the dictionaries then collect trafﬁc satisfying

them and ship them back to the analyst. Echelon appears to work very much

like Google, except that instead of searching web pages it searches through the

world’s phone and data network trafﬁc in real time.

24.3.8 Intelligence Strengths and Weaknesses

Echelon seems impressive — if scary. But several points are worth bearing in

mind.

First, the network built up by the NSA and its allies was mainly aimed at

the old USSR, where human intelligence was difﬁcult, and hoovering up vast

quantities of phone calls gave at least some idea of what was going on. But the

resulting political and economic intelligence turned out to be poor; the West

thought that Russia’s economy was about twice as large as it actually was,

and was surprised by its collapse post-1989. (The agencies’ incentives to talk

up the threat are clear.) In any case, much of the effort was military, aimed

at understanding Soviet radar and communications, and at gaining a decisive

advantage in location, jamming and deception. Without an ability to conduct

electronic warfare, a modern state is not competitive in air or naval warfare or

in tank battles on the ground. So it’s not surprising that most of the personnel

at NSA are military, and its director has always been a serving general. There

is still a lot of effort put into understanding the signals of potential adversaries.

Second, there have been some successes against terrorists — notably the

arrest of the alleged 9/11 mastermind Khalid Shaikh Mohammed after he

used a mobile phone SIM from a batch bought by a known terrorist in

Switzerland. But electronic warfare against insurgents in Iraq has proved to

be unproductive, as I discussed in Chapter 19. And it’s long been clear that

much more effort should have been put into human intelligence. In an article

published just before 9/11, an analyst wrote ‘The CIA probably doesn’t have

a single truly qualiﬁed Arabic-speaking ofﬁcer of Middle Eastern background

who can play a believable Muslim fundamentalist who would volunteer to

spend years of his life with shitty food and no women in the mountains

of Afghanistan. For Christ’s sake, most case ofﬁcers live in the suburbs of

Virginia. We don’t do that kind of thing’. Another put it even more bluntly:

‘Operations that include diarrhea as a way of life don’t happen’ [521]. The

combination of stand-off technical intelligence plus massive ﬁrepower suits

788 Chapter 24 ■Terror, Justice and Freedom

the private interests of bureaucrats and equipment vendors, but it makes

allied forces ineffective at counterinsurgency, where the enemy blends with

the civilian population. Similar perverse incentives hamper the military. For

example, Britain is spending billions on two new aircraft carriers and on

modernising the nuclear deterrent, but even six years after 9/11 we haven’t

trained enough soldiers to carry a basic conversation in Arabic. The big debate

now brewing in the Pentagon is not just about intelligence, but how to evolve

a smarter approach to counterinsurgency across the board [414].

Third, while the proliferation of mobile phones, wireless LANs and online

services presents the agencies with a cornucopia of new information sources,

the volume is a huge problem. Even with a budget of billions of dollars a year

and tens of thousands of staff, not even the NSA can collect all the electronic

communications everywhere in the world. The days in which they could

record all transatlantic phone calls with a rack of 16-track tape recorders are

no more. Equipment for tapping high-speed backbone links does exist [167],

but it’s expensive. Sprint’s budget is bigger than the NSA’s, and is spent on

low-cost commercial products rather than high-cost classiﬁed ones, so they

can put in lines much faster than the NSA can tap them. Data volumes force

most trafﬁc selection to be done locally, and in real time [770]. Suppose, for

example, that the NSA got interested in the UK university system — let’s call it

a hundred institutions at 2 Gbit/sec each. They couldn’t ship all the bits across

the Atlantic to Fort Meade as there just isn’t enough transatlantic bandwidth.

Tapping all the data streams of all the corporations in Japan would be an order

of magnitude harder.

Fourth, although other countries may complain about U.S. sigint collec-

tion, for them to moralize about it is hypocritical. Other countries also run

intelligence operations, and are often much more aggressive in conducting

economic and other non-military espionage. The real difference between the

WASP countries and the others is that no-one else has built the Echelon

‘system-of-systems’. Indeed, there may be network effects at work in sigint

as elsewhere: the value of a network grows faster than its size, and the more

you tap, the cheaper it gets. There have thus been moves to construct a ‘Euro-

pean Echelon’ involving the police and intelligence agencies of continental

European countries [430, 445].

The mature view, I think, is that signals intelligence is necessary for a

nation’s survival but potentially dangerous — just like the armed forces it

serves. An army can be a good servant but is likely to be an intolerable

master. The issue is not whether such resources should exist, but how they are

held accountable. In the USA, hearings by Senator Church in 1975 detailed a

number of abuses such as the illegal monitoring of U.S. citizens [292]. Foreign

intelligence gathering is now regulated by U.S. law in the form of 50 USC

1801–1811 [1272], which codiﬁes FISA. This isn’t perfect; as already noted, it’s

the subject of ﬁerce tussles between the executive and the legislature about

24.3 Surveillance 789

the recent provision that the NSA can wiretap U.S. calls so long as one of the

parties is believed not to be a U.S. person. Even before this, the number of FISA

warrants has risen steadily since 9/11 to exceed the number of ordinary (title

III) wiretap warrants. But at least Congress has got interested. And the USA is

lucky: in most countries, the oversight of intelligence isn’t even discussed.

Finally, poor accountability costs more than just erosion of liberty and

occasional political abuse. There is also a real operational cost in the prolifer-

ation of intelligence bureaucracies that turn out to be largely useless once the

shooting starts. In Washington during the Cold War, the agencies hated each

other much more than they hated the Russians. In the UK, one of the most

vicious intelligence battles was not against the IRA, but between the police

and MI5 over who would be the lead in the ﬁght against the IRA. There are

numerous accounts of intelligence inefﬁciency and inﬁghting by well-placed

insiders, such as R.V. Jones [671]. It is in this context of bureaucratic turf wars

that I’ll now describe the ‘Crypto Wars’ of the 1990s, which were a formative

experience for many governments (and NGOs) on issues of surveillance and

technology policy.

24.3.9 The Crypto Wars

Technology policy during the 1990s was dominated by acrimonious debates

about key escrow — the doctrine that anyone who encrypted data should give

the government a copy of the key, so that the civilian use of cryptography

would not interfere with intelligence gathering by the NSA and others.

Although some restrictions on cryptography had existed for years and

irritated both academic researchers and civilian users, they shot to the headlines

in 1993 when President Clinton astonished the IT industry with the Escrowed

Encryption Standard, more popularly known as the Clipper chip. This was

a proposed replacement for DES, with a built-in back door key so that

government agencies could decipher any trafﬁc. The NSA had tried to sell

the program to the cabinet of President Bush senior and failed; but the new

administration was happy to help.

American opinion polarized. The government argued that since cryptogra-

phy is about keeping messages secret, it could be used by criminals to prevent

the police gathering evidence from wiretaps; the IT industry (with a few

exceptions) took the conﬂicting view that cryptography was the only means of

protecting electronic commerce and was thus vital to the future development

of the net. Civil liberties groups lined up with the industry, and claimed that

cryptography would be the critical technology for privacy. By 1994, the NSA

had concluded that they faced a war with Microsoft that Bill would win, so

they handed off the policy lead to the FBI while continuing to direct matters

from behind the scenes.

790 Chapter 24 ■Terror, Justice and Freedom

The debate got rapidly tangled up with export controls on weapons, the

means by which cryptography was traditionally controlled. U.S. software

ﬁrms were not allowed to export products containing cryptography which

was too hard to break (usually meaning a keylength of over 40 bits). A

U.S. software author, Phil Zimmermann, was hauled up before a grand jury

for arms trafﬁcking after a program he wrote — PGP — ‘escaped’ on to the

Internet. He immediately became a folk hero and made a fortune as his product

grabbed market leadership. The conﬂict became international: the U.S. State

Department tried hard to persuade other countries to control cryptography too.

It became one of the personal missions of Vice-President Gore (a reason why

many in Redmond and the Valley contributed to the Bush campaign in 2000).

The results were mixed. Some countries with memories of oppressive

regimes, such as Germany and Japan, resisted American blandishments. Oth-

ers, such as Russia, seized the excuse to pass harsh crypto control laws. France

thumbed its nose by relaxing a traditional prohibition on non-government use

of crypto; Britain obediently changed from a liberal, laissez-faire policy under

John Major in the mid 1990s to a draconian law under Tony Blair. The Regula-

tion of Investigatory Powers (RIP) Act of 2000 enables the police to demand that I

hand over a key or password in my possession, and the Export Control Act of

2002 instructs me to get an export license if I send any cryptographic software

outside Europe that uses keys longer than 56 bits. Oh, and the government

has also taken powers to vet foreign research students studying dangerous

subjects like computer science, and to refuse visas to those they consider a

proliferation risk.

I was involved in all this as one of the academics whose research and

teaching was under threat from the proposed controls, and in 1998 I was one

of the people who set up the Foundation for Information Policy Research, the

UK’s leading internet-policy think-tank, which wrestled with crypto policy,

export policy, copyright and related issues. In the next few sections I’ll lay out

a brief background to the crypto wars, and then describe the consequences for

export controls today, and for what we can learn about the way governments

have failed to get to grips with the Internet.

24.3.9.1 The Back Story to Crypto Policy

Many countries made laws in the mid-19th century banning the use of cryp-

tography in telegraph messages, and some even forbade the use of languages

other than those on an approved list. Prussia went as far as to require telegraph

operators to keep copies of the plaintext of all messages [1215]. Sometimes the

excuse was law enforcement — preventing people obtaining horse race results

or stock prices in advance of the ‘ofﬁcial’ transmissions — but the real concern

24.3 Surveillance 791

was national security. This pattern was to repeat itself again in the twentieth

century.

After the immense success that the Allies had during World War 2 with

cryptanalysis and signals intelligence in general, the UK and US governments

agreed in 1957 to continue intelligence cooperation. This is known as the

UKUSA agreement, although Canada, Australia and New Zealand quickly

joined. The member nations operated a crypto policy whose main goal was

to prevent the proliferation of cryptographic equipment and know-how. Until

the 1980s, about the only makers of cryptographic equipment were companies

selling into government markets. They could mostly be trusted not to sell

anything overseas which would upset their major customers at home. This

was reinforced by export controls which were operated ‘in as covert a way

as possible, with the minimum of open guidance to anyone wanting, for

example, an export licence. Most things were done in behind-the-scenes

negotiation between the ofﬁcials and a trusted representative of the would-be

exporter’. [142]

In these negotiations, the authorities would try to steer applicants towards

using weak cryptography where possible, and where confronted with a more

sophisticated user would try to see to it that systems had a ‘back door’ (known

in the trade as a red thread) which would give access to trafﬁc. Anyone who

tried to sell decent crypto domestically could be dissuaded by various means. If

they were a large company, they would be threatened with loss of government

contracts; if a small one, they could be strangled with red tape as they tried to

get telecomms and other product approvals.

The ‘nonproliferation’ controls were much wider than cryptography, as

computers also fell within their scope. By the mid-1980s, the home computers

kids had in their bedrooms were considered to be munitions, and manufac-

turers ended up doing lots of paperwork for export orders. This pleased the

bureaucrats as it gave them jobs and power. The power was often abused: in

one case, an export order for a large number of British-made home computers

to the school system in Yugoslavia was blocked at the insistence of the U.S.

authorities, on the grounds that it contained a U.S. microprocessor; a U.S. ﬁrm

was promptly granted a license to export into this market. Although incidents

like this brought the system into disrepute, it persists to this day.

Crypto policy was run in these years along the same lines as controls on

missile technology exports: to let just enough out to prevent companies in

other countries developing viable markets. Whenever crypto controls got so

onerous that banks in somewhere like Brazil or South Africa started having

crypto equipment custom built by local electronics ﬁrms, export licensing

would ease up until the threat had passed. And, as I described in the chapter

on API security, the hardware security modules sold to banks throughout this

792 Chapter 24 ■Terror, Justice and Freedom

period had such poor interface designs that compromising them was trivial

anyway.

Vulnerabilities in bank crypto merely increased the risk of fraud slightly,

but bad crypto elsewhere exposed its users to surveillance. The Swedish gov-

ernment got upset when they learned that the ‘export version’ of Lotus Notes

which they used widely in public service had its cryptography deliberately

weakened to allow NSA access; and at least one (U.S. export approved) cipher

machine has broadcast its plaintext in the clear in the VHF band. But the most

notorious example was the B ¨

uhler case.

Hans B ¨

uhler worked as a salesman for the Swiss ﬁrm Crypto AG, which

was a leading supplier of cryptographic equipment to governments without

the technical capability to build their own. He was arrested in 1992 in Iran

and the authorities accused him of selling them cipher machines which had

been tampered with so that the NSA could get at the plaintext. After he

had spent some time in prison, Crypto AG paid 1.44 billion Rials — about

a million U.S. dollars — to bail him, but then ﬁred him once he got back to

Switzerland. B ¨

uhler then alleged on Swiss radio and TV that the ﬁrm was

secretly controlled by the German intelligence services and that it had been

involved in intelligence work for years [238]. The interpretation commonly

put on this was that ultimate control resided with the NSA (the founder of

Crypto, Boris Hagelin, had been a lifelong friend of William Friedman, the

NSA’s chief scientist) and that equipment was routinely red threaded [824]. A

competing interpretation is that these allegations were concocted by the NSA

to undermine the company, as it was one of the third world’s few sources of

cryptographic equipment. B ¨

uhler’s story is told in [1228].

24.3.9.2 DES and Crypto Research

Despite the very poor implementation quality of early banking cryptosystems,

the NSA still worried in the seventies that the banking sector might evolve

good algorithms that would escape into the wild. Many countries were still

using rotor machines or other equipment that could be broken using the

techniques developed in World War 2. How could the banking industry’s

thirst for a respectable cipher be slaked, not just in the U.S. but overseas,

without this cipher being adopted by foreign governments and thus adding to

the costs of intelligence collection?

The solution was the Data Encryption Standard (DES). At the time, as I

mentioned in section 5.4.3.2, there was controversy about whether 56 bits

were enough. We now know that this was deliberate. The NSA did not

at the time have the machinery to do DES keysearch; that came later. But

by giving the impression that they did, they managed to stop most foreign

governments adopting it. The rotor machines continued in service, in many

cases reimplemented using microcontrollers, and the trafﬁc continued to be

24.3 Surveillance 793

harvested. Foreigners who encrypted their important data with such ciphers

merely solved the NSA’s trafﬁc selection problem.

A second initiative was to undermine academic research in cryptology. In

the 1970s this was done directly by harassing the people involved; by the 1980s

it had evolved into the subtler strategy of claiming that published research

work was all old hat. The agencies opposed crypto research funding by saying

‘we did all that stuff thirty years ago; why should the taxpayer pay for it twice?’

The insinuation that DES may have had a ‘trapdoor’ inserted into it ﬁtted well

with this play. A side effect we still live with is that the crypto and computer

security communities got separated from each other in the early 1980s as the

NSA worked to suppress one and build up the other.

By the mid 1990s this line had become exhausted. Agency blunders in

the design of various key escrow systems showed that they have no special

expertise in cryptology compared with the open research community, and as

attempts to inﬂuence the direction of academic research by interfering with

funding have become less effective they have become much less common.

24.3.9.3 The Clipper Chip

Crypto policy came into the open in 1993 with the launch of the Clipper

chip. The immediate stimulus was the proposed introduction by AT&T to

the U.S. domestic market of a high-grade encrypting telephone that would

have used Difﬁe-Hellman key exchange and triple-DES to protect trafﬁc. The

NSA thought that the government could use its huge buying power to ensure

the success of a different standard in which spare keys would be available

to the agencies to decrypt trafﬁc. This led to a public outcry; an AT&T computer

scientist, Matt Blaze, found a protocol vulnerability in Clipper [183] and the

proposal was withdrawn.

Several more attempts were made to promote the use of cryptography

with government access to keys in various guises. Key escrow acquired

various new names, such as key recovery; certiﬁcation authorities which kept

copies of their clients’ private decryption keys became known as Trusted

Third Parties (TTPs) — somewhat emphasising the NSA deﬁnition of a trusted

component as one which can break security. In the UK, a key escrow protocol

was introduced for the public sector, and this was used to try to get the

private sector to adopt it to; but a number of vulnerabilities were found in it

too [76].

Much of the real policy leverage had to do with export licensing. As the

typical U.S. software ﬁrm exports most of its product, and as maintaining a

separate product line for export is expensive, many ﬁrms could be dissuaded

from offering strong cryptography by prohibiting its export. Products with

‘approved’ key escrow functionality were then granted preferential U.S. export

license treatment. The history of this struggle is still to be fully written, but

794 Chapter 24 ■Terror, Justice and Freedom

a ﬁrst draft is available from Difﬁe and Landau [387] and many of the U.S.

source documents, obtained under FOIA, have been published in [1135].

One of the engineering lessons from this whole process is that doing key

escrow properly is hard. Making two-party security protocols into three-party

protocols increases the complexity and the risk of serious design errors, and

centralizing the escrow databases creates huge targets [4]. Where escrow is

required it’s usually better done with simple local mechanisms. In one army,

the elegant solution is that every ofﬁcer must write down his passphrase on

a piece of paper, put it into an envelope, stamp it ‘Secret’ and hand it to

his commanding ofﬁcer, who puts it in his ofﬁce safe. That way the keys

are kept in the same place as the documents whose electronic versions they

protect, and there’s no central database for an airplane to bomb or a spy to

steal.

24.3.10 Did the Crypto Wars Matter?

When the key escrow debate got going in the UK in 1994–5, I took a line

that was unpopular at the time with both the pro-escrow and the anti-escrow

lobbies. The pro-escrow people said that as crypto provided conﬁdentiality,

and conﬁdentiality could help criminals, there needed to be some way to

defeat it. The anti-escrow lobby said that since crypto was necessary for

privacy, there must not be a way to defeat it. I argued in [35] that essentially all

the premises behind these arguments were wrong. Most crypto applications

(in the real world, as opposed to academia) are about authentication rather

than conﬁdentiality; they help the police rather than hindering them. As

for criminals, they require unobtrusive communications — and encrypting a

phone call is a good way to bring yourself to the attention of the agencies. As

for privacy, most violations result from abuse of authorized access by insiders.

Finally, a much more severe problem for policemen investigating electronic

crimes is to ﬁnd acceptable evidence, for which decent authentication can be

helpful.

This is not to say that the police have no use for wiretaps. Although many

police forces get by quite happily without them, and many of the ﬁgures

put forward by the pro-wiretap lobby are dishonest [387], there are some

occasions where wiretapping can be economic as an investigative tool. The

Walsh report — by a senior Australian intelligence ofﬁcer — gives a reasonably

balanced examination of the issues [1311]. Walsh compared the operational

merits of wiretaps, bugs and physical surveillance, and pointed out that

wiretaps were either the cheapest or the only investigative technique in some

circumstances. He nonetheless found that there is ‘no compelling reason or

virtue to move early on regulation or legislation concerning cryptography’,

but he did recommend that police and intelligence agencies be allowed to

24.3 Surveillance 795

hack into target computers to obtain access or evidence2. It took the view

that although there will be some policing costs associated with technological

advances, there will also be opportunities: for example, to infect a suspect’s

computer with software that will turn it into a listening device. This hit the

nail on the head. The police — like the intelligence services — are reaping a

rich harvest from modern technology.

We all knew, of course, that the police forces who argued in favour of

key escrow did so under orders and as a front for the spooks3.Nowthe

aims and objectives of policemen and spies are not quite identical, and

confusing them has clouded matters. It is perhaps an oversimpliﬁcation that

the former try to prevent crimes at home, while the latter try to commit

them abroad; but such aphorisms bring out some of the underlying tension.

For example, policemen want to preserve evidence while spies like to be

able to forge or repudiate documents at will. During the discussions on

a European policy toward key escrow (‘Euroclipper’) that led up to the

Electronic Signature Directive, the German government demanded that only

conﬁdentiality keys should be escrowed, not signature keys; while Britain

wanted signature keys to be escrowed as well. The British view followed the

military doctrine that deception is at least as important as eavesdropping,

while the Germans supported the police doctrine of avoiding investigative

techniques that undermine the value of any evidence subsequently seized.

The key goal of the intelligence community in the 1990s, as we later learned,

was to minimise the number of systems that used crypto by default. If a signif-

icant proportion of data trafﬁc were encrypted, then the automated keyword

searching done by systems such as Echelon would be largely frustrated. The

NSA was quite aware that many new network systems were being built rapidly

during the dotcom boom, and if cryptography wasn’t built in at the start,

it should usually be too expensive to retroﬁt it later. So each year the NSA

held the line on crypto controls meant dozens of systems open to surveil-

lance for decades in the future. In these terms, the policy was successful:

little of the world’s network trafﬁc is encrypted, the main exceptions being

DRM-protected content, Skype, the few web pages that are protected by TLS,

opportunistic TLS encryption between mail servers, SSH trafﬁc, corporate

VPNs and online computer games. Everything else is pretty much open to

interception — including masses of highly sensitive email between companies.

2The Walsh report has an interesting publishing history. Originally released in 1997 as an

unclassiﬁed document, it was withdrawn three weeks later after people asked why it wasn’t

yet on sale in the shops. It was then republished in redacted form. Then researchers found

unexpurgated copies in a number of libraries. So these were published on the web, and the

redacted parts drew attention at once to the issues the government considered sensitive. As late

as 1999, the Australian government was still trying to suppress the report [1311].

3This was admitted in an unguarded moment in 1996 by the UK representative on the European

body responsible for crypto policy [596].

796 Chapter 24 ■Terror, Justice and Freedom

In the end, the crypto wars ended in the USA because Al Gore felt he needed

to woo Silicon Valley in 2000 and gave up on the initiative (too late — many

software millionaires supported the Republicans that year), and in Europe

because the European Commission felt that it was getting in the way of

building conﬁdence in online banking and commerce — so they passed an

Electronic Signature Directive that said in effect that signature keys couldn’t

be escrowed or they would lose their legal effectiveness. The Germans had

won the argument. As for whether it mattered, U.S. government reports of

Title III wiretaps since then disclose only one case in which cryptography

prevented the authorities from recovering the plaintext [435].

24.3.11 Export Control

The main spillover from the crypto wars was the imposition of much more

stringent export controls than before, particularly in Europe. There is a survey

of cryptography law at [736]; here’s a quick summary.

International arms control agreements (COCOM and Wassenaar) bind most

governments to implement export controls on cryptographic equipment,

and the latter is implemented in the European Union by an EU regula-

tion compelling Member States to control and license the export of dual-use

goods — goods which have both civilian and military uses. Cryptanalytic prod-

ucts fall under the military regime, whereas the great bulk of software that just

uses cryptography for protection falls under dual-use.

But national implementations vary. UK law didn’t control the export of

intangibles until 2002, so crypto software could be exported electronically;

the Belgian government grants licences for almost anything; and Switzerland

remains a large exporter of crypto equipment. Domestic controls also varied.

The French government started off from a position of prohibiting almost all

civilian cryptography and moved to almost complete liberalisation, while

Britain went the other way.

What this meant in practice during the 1990s was that European researchers

like me could write crypto software and publish it on our web pages, while

our counterparts in the USA were prevented from doing that by the U.S.

International Trafﬁcking in Arms Regulations (ITAR). Non-U.S. companies

started to get a competitive advantage because they could export software

in intangible form. The U.S. government got annoyed and in 1997, Al Gore

persuaded the incoming British Prime Minister Tony Blair to get Europe to

extend export control to intangibles. Meanwhile the USA relaxed its own

controls, so now the positions are reversed, and Europe has the ﬁercest rules.

Tens of thousands of small software companies are breaking the law without

knowing it by exporting products (or even by giving away software) containing

crypto with keys longer than 56 bits.

24.4 Censorship 797

There are several ways to deal with this. In many countries people will just

ignore the law and just pay a bribe if, by misfortune, they are targeted for

enforcement. In Northern Europe, one course of action is to try to use various

Open General Export Licenses (OGELs) that provide speciﬁc exemptions for

particular products and activities, but these require a cumbersome registration

process and will often be unsuited to an innovative company. Another is to

use the exemption in export law for material being put in the public domain;

make your software (or the relevant parts of it) free or open-source and make

your money on support and services. Another, in the UK, at least, is to use

the fact that placing something on a web server isn’t export; the exporter, in

law, is any person outside Europe who downloads it. So a developer can leave

material online for download without committing an offence. Yet another is of

course to actually apply for an export license, but the licensing system is

geared to small numbers of large companies that export military hardware

and are known to the licensing authorities. If large numbers of small software

ﬁrms were to deluge them with applications for licenses, the system would

break down. At present some ofﬁcials are trying to empire-build by ‘raising

awareness’ of export controls among academics (who ignore them); thankfully

there are no signs of the controls being marketed to the software industry.

24.4 Censorship

I wrote in the ﬁrst edition that ‘the 1990s debate on crypto policy is likely to be

a test run for an even bigger battle, which will be over anonymity, censorship

and copyright’. Although (as I discussed in Chapter 22) copyright law has

largely stabilised, there is still pressure from Hollywood for ISPs to ﬁlter out

ﬁle-sharing trafﬁc. However censorship has become a much bigger issue over

the past few years.

Censorship is done for a variety of motives. China blocks not just dissident

websites, but even emails mentioning forbidden movements. Some countries

switch censorship on during elections, or after crises; Burma imposed curfews

after suppressing a wave of demonstrations. The live debate in the USA is about

whether ISPs who are also phone companies should be able to block VOIP,

and whether ISPs who also run cable channels should be able to block P2P:

the principle of net neutrality says that ISPs should treat all packets equally. Net

neutrality isn’t as much of an issue in Europe where there’s more competition

between ISPs; the issue is that different European countries ban different types

of content (France and Germany, for example, ban the sale of Nazi memorabilia,

and won’t let Amazon sell copies of Mein Kampf). Many countries have made

attempts to introduce some kind of controls on child pornography — it’s

become a standard excuse for politicians who want to ‘do something’ about

the Internet — and as I write there’s a European initiative to ban radical

798 Chapter 24 ■Terror, Justice and Freedom

Islamist websites. Finally, censorship is sometimes imposed by courts in the

context of civil disputes, such as the ban on publishing the DeCSS code that I

mentioned in Chapter 22.

Censorship also takes a number of forms, from blocking certain types of

trafﬁc to IP address ﬁltering, DNS poisoning, content inspection, and out-of-

band mechanisms such as the punishment of individuals who downloaded

(or were alleged to have downloaded) discountenanced material. I’ll look now

at a number of cases. (Declaration of interest: I’ve been funded by the Open

Net Initiative as a result of which my students and postdocs have been busy

measuring censorship in a number of countries.)

24.4.1 Censorship by Authoritarian Regimes

Rulers have long censored books, although the invention of the printing press

made their job a whole lot harder. For example, John Wycliffe translated

the Bible into English in 1380–1, but the Lollard movement he started was

suppressed along with the Peasants’ Revolt. When William Tyndale had

another go in 1524–5, the technology now let him spread the word so quickly

that the princes and bishops could not suppress it. They had him burned at the

stake, but too late; over 50,000 copies of the New Testament had been printed,

and the Reformation got under way. After that upset, printers were closely

licensed and controlled; things only eased up in the eighteenth century.

The invention of the Internet has made the censors’ job easier in some ways

and harder in others. It’s easier for the authorities to order changes in material

that not many people care about: for example, courts that ﬁnd a newspaper

guilty of libel order the offending material to be removed, and changing

the historical record wasn’t possible when it consisted of physical copies in

libraries rather than, as now, the online archive. It’s easier for the authorities

to observe the transmission of disapproved material, as they can monitor

the content of electronic communications much more easily than physical

packages. But mostly it’s harder for them, as nowadays everyone can be a

publisher; governments can still crack down on mainstream publishers, but

have to contend with thousands of bloggers. A good reason for hope comes

from observation of countries that try hard to censor content, such as China.

China had 137 million Internet users at the end of 2006, including a quarter

of the population in the big cities. The government of Hu Jintao is committed

to control and has invested hugely in ﬁltering technology. People refer to ‘the

Great Firewall of China’ although in fact the controls in that country are a

complex socio-technical system that gives defence in depth against a range of

material, from pornography to religious material to political dissent [984].

First, there are the perimeter defences. Most of China’s Internet trafﬁc ﬂows

through routers in Shenzhen near Hong Kong which ﬁlter on IP addresses

to block access to known ‘bad’ sites like the Voice of America and the BBC;

24.4 Censorship 799

they also use DNS cache poisoning. In addition, deep packet inspection at

the TCP level is used to identify emails and web pages containing forbidden

words such as ‘Falun Gong’: TCP reset packets are sent to both ends of such

connections to tear them down. (I described the mechanisms in section 21.4.2.3

and noted there that while they can be fairly easily circumvented, anyone who

did so regularly might expect a visit from the police.) Keyword ﬁltering based

on about 1000 wicked words is also implemented in Chinese search engines

and blogs, while some individual Internet service providers also implement

their own blocking and Internet caf´

es are required to by law.

Second, there are application-level defences. Some services are blocked and

some aren’t, depending on the extent to which the service provider plays along

with the regime. There was a huge row when Google agreed to censor its search

results in China (what they actually do is to populate their China index using

spiders that search from within China, and thus only see material that’s visible

there anyway). The incentives created by China’s rapidly growing markets

enable its government to bully large international ﬁrms into compliance. One

effect is that, as more and more of the online action moves to server farms run

by transnational ﬁrms, the borders that matter are those of ﬁrms rather than

of nations [918] (this is still probably an improvement, as new companies are

easier to start than new countries).

Third, there are social defences. These range from 30,000 online police,

through trials of cyber-dissidents and laws requiring cyber-caf´

es to identify

customers, to a pair of Internet police cartoon mascots (Jingjing and Chacha)

who pop up everywhere online to remind users that they’re in social space

rather than private space.

Yet the controls appear to be falling behind. There are more than 20 million

blogs in China, and although the online police are vigorous at taking down

openly seditious material, the online discussion of local news events has led

to the emergence of a proper ‘public opinion’ that for the ﬁrst time is not in

thrall to media managers [985]. This is not just a function of email and blogs

but also the rapid growth in mobile phone use. Local events such as land

seizures by corrupt ofﬁcials can now rapidly climb the news agenda, exposing

the government to pressures from which it was previously insulated. It will

be interesting to see how things go as China hosts the Olympics in 2008 and

continues to develop beyond that.

A somewhat different example is Burma. There, a sudden increase in fuel

prices in August 2007 led to mass protests and a violent crackdown by the army

from September 26th that left perhaps several hundred people dead. During the

protests and at the start of the crackdown, Burmese citizens used the Internet

and mobile phone services to send photos, videos and other information to the

outside world, with the result that their insurrection grabbed world headlines

and the crackdown brought widespread condemnation on the ruling junta.

800 Chapter 24 ■Terror, Justice and Freedom

This happened despite the fact that Burma is one of only 30 countries in the

world with less than 1% of its population online.

Other authoritarian states — such as Belarus, Uganda and the Yemen — had

imposed Internet censorship around elections and other political events, and

initially the Burmese junta concentrated on ﬁltering political information

arriving from overseas. But the world headlines clearly caused pain, and an

Internet closedown started on September 29th, the third day of the crackdown.

This was the ﬁrst time wholesale Internet blocking was used to stop news

getting out [986]. Service was resumed patchily after October 4; from the 4th to

the 12th there was a curfew, with connectivity available only from 10pm until

4am; and in the third phase, some Internet caf´

es were allowed to reopen on

October 11th, but speeds were limited to 256 kbit/sec; others were closed down

and had equipment conﬁscated. It seems that most Burmese had been using

censorship-circumvention tools such as proxies. In fact the uprising was called

‘the g-lite revolution’ after a popular Gmail proxy, http://glite.sayni.net.

If the lesson to learn from this sad incident is that even 1% Internet use can

destabilise a dictatorship, and that even dictatorships have a hard time getting

by without the Internet, then that’s rather encouraging.

24.4.2 Network Neutrality

A number of less developed countries block voice-over-IP (VOIP) services to

make phone tapping easier, and to keep the state phone company proﬁtable.

LDC phone companies often get much of their revenue from their share of the

charges paid by foreigners to call that country, and VOIP lets expats escape

these charges.

However, most of the problems experienced by VOIP operators are in the

developed world, and particularly in America. A number of ISPs are also phone

companies, and use technical mechanisms to disrupt VOIP services — such

as introducing jitter or short outages into the packet stream. This affects not

just wireline providers but also mobile ﬁrms. As a result, a ﬁerce debate has

erupted in Washington about network neutrality. On one side, the VOIP industry

argues in favour of a law that would compel ISPs to treat all trafﬁc equally; on

the other, the phone companies retort ‘Don’t regulate the Internet’.

The issue is wider than just VOIP. Phone companies always charged widely

different rates for different types of trafﬁc: given that they have high ﬁxed costs

and low marginal costs, they have every incentive to price discriminate. Ed

Whitacre, the AT&T chairman, kicked off the debate in 2005 when he argued

that for companies like Google, Yahoo or Vonage to use ‘his’ broadband

pipes for free to make money for themselves was ‘nuts’ [1340]. This has split

Congress broadly on party lines, with Democrats favouring net neutrality and

Republicans favoring the phone companies.

24.4 Censorship 801

In Europe, net neutrality is less of an issue, as we have more competition

in the ISP market. Regulators tend to take the view that if some ISPs indulge in

trafﬁc shaping (as it’s politely called), then that doesn’t matter so long as

customers can switch to other ISPs that don’t. There are some residual issues

to do with mobile operators, as international calls from mobiles are expensive,

but regulators are trying to tackle high charges directly rather than worrying

about whether people can use Skype over GPRS.

24.4.3 Peer-to-Peer, Hate Speech and Child Porn

The three horses being ﬂogged by the advocates of Internet censorship in the

developed countries are ﬁle sharing, hate speech and child pornography.

File-sharing systems raise some of the net neutrality issues; for example,

Comcast has been disrupting BitTorrent, using the same forged-reset packet

techniques observed in the Great Firewall of China [409]. In Comcast’s case,

being a cable operator, they want their customers to watch TV on their cable

channel, rather than as downloads, to maximise their ad revenue. Other

ISPs have different incentives; many people sign up to broadband service

speciﬁcally so they can download stuff. Whether this makes a proﬁt for the ISP

or not will depend on how much trafﬁc new customers generate and whether

the backhaul costs more than their subscriptions. In general, ISPs make money

from P2P, though often they have to restrict bandwidth use.

The main players arguing for ﬁltering of peer-to-peer trafﬁc are the music

companies. Many universities have been bullied by the threat of litigation into

restricting such trafﬁc on student LANs; others have cut it simply to save

on bandwidth charges. And despite all the economic evidence I discussed

in Chapter 22, about the modest effect that ﬁle-sharing has on music sales,

the music industry believes its interests would be served by imposing this

censorship more widely. ISPs resist cenorship citing the high costs of ﬁltering.

It’s therefore going to be interesting to see whether countries introduce

mandatory ﬁltering for ‘moral’ purposes, which the music industry can then

have used for its purposes too.

There was a recent attempt in Europe to introduce a duty on ISPs to ﬁlter

hate speech, and speciﬁcally jihadist websites. Europe has a history of such

restrictions: France and Germany both prohibit the sale of Nazi memorabilia. (I

recall one German justice minister telling a policy conference that her greatest

achievement in ofﬁce was to stop Amazon selling ‘Mein Kampf’ in Germany,

and her greatest ambition was to stop them selling it in Arizona too.) I’m very

sceptical about whether such a law would make Europe any safer; banning

the writings of the militant Deobandi Muslim sect, to which perhaps a third of

Britain’s Muslims belong, is likely to aggravate community tensions more than

anything else. Furthermore, research shows that most of the hate literature

distributed inside and outside Britain’s mosques is produced or funded by

802 Chapter 24 ■Terror, Justice and Freedom

religious institutions from Saudi Arabia [857]. The response of our government

is not to stand up to King Abdullah, but to invite him for a state visit. Internet

censorship here (as elsewhere) appears to be a displacement activity; it lets

the government claim it’s doing something. It’s also likely to encourage all the

third-world despots and Asian strongmen who denounce the freedom of

speech on the Internet. Better, I’d think, to leave this material in the open, as

America does, and let the police monitor the trafﬁc to the worst of the sites,

rather than driving Muslim youth to acquire the skills of the Chinese and

Burmese at using proxies. In the end, the policy advice to the European Com-

mission was along these lines: they should train the police to use the existing

laws better [442]. And while they’re at it, let law enforcement be technology-

neutral: the cops should also monitor the young men who sell the hate tracts

in the mosques (and if they ever pluck up the courage to prosecute them for

breaking the existing laws on incitement to murder, so much the better).

The third horseman is child pornography. During the 1990s, as governments

were looking for some handle on the Internet, a view arose that explicit images

of child sex abuse were about the one thing that all states could agree should

be banned. When arguing in favour of the latest repressive measure — such as

key escrow — governments trotted out people from children’s charities who

would argue passionately that the Stalinism du jour was vital to save children

from harm [272]. Needless to say, those of us on the liberal side of the argument

would have preferred the charities to spend their money campaigning about

more serious and potentially ﬁxable child-protection problems, such as the

abuse of children in local authority care homes, and under-age prostitution;

and when a really serious problem arose at the boundary between IT policy and

child protection — a proposal to construct a national child-welfare database

that will expose the personal information of millions of children to hundreds

of thousands of public-sector workers [66] — these worthy child-protectors

remained silent.

The child-porn debate has subsided in most countries4, as terrorism has

taken the place of kiddieporn as the executive’s ace of trumps –the argument

that no-one’s supposed to gainsay. But the hysteria did have some evil effects.

They were severe in the UK where it was used to justify not only more

pervasive online surveillance, but also a National High-Tech Crime Unit. This

unit ran Operation Ore, in which some eight thousand UK citizens got raided

by the police on suspicion of purchasing child pornography. It turned out that

most of them were probably victims of card fraud. The porn squad didn’t

understand card fraud, and didn’t want to know; they were ﬁxated on getting

porn convictions, and didn’t ask their experts to even consider the possibility

4Russia’s a notable exception; Putin uses kiddieporn as the leading excuse for censorship directed

at political opponents, while his police take little action against the many pornographers and

other online criminals in that country.

24.5 Forensics and Rules of Evidence 803

of fraud. Several thousand men had their lives disrupted for months or even

years following wrongful arrest for highly stigmatised offences of which they

were innocent, and at the time of writing (2007) there’s a steady stream of

acquittals, of civil lawsuits for compensation against police forces, and calls

for public inquiries. The sad story of police bungling and cover-up is told by

Duncan Campbell in [260, 261]. For some, the revelation that the police had

screwed up came too late; over thirty men, faced with the prospect of a public

prosecution that would probably destroy their families, killed themselves. At

least one, Commodore David White, commander of British forces in Gibraltar,

appears to have been innocent [594].

The cause of all this was that operators of illegal porn sites bought up

lists of credit card numbers and then booked them through the portals that

they used to collect payment — presumably in the belief that many people

would not dare to report debits for such services to the police. And although

the police justiﬁed their operations by claiming they would reduce harm

to children, the child-porn purveyors in the Ore case escaped prosecution.

(The operator of the main portal, Thomas Reedy, did get convicted and

sentenced to over 1000 years in a Texas jail, but he was just the fall guy who

collected the credit card payments. The gangsters in Indonesia and Brazil

who organised and photographed the child abuse do not seem to have been

seriously pursued.)

America actually handled this case much better than Britain. Some 300,000

U.S. credit card numbers were found on Reedy’s servers; the police used

the names for intelligence rather than evidence, matching the names against

their databases, identifying suspects of concern — such as people working

with children — and quietly investigating them. Over a hundred convictions

for actual child abuse followed, and no wrongful convictions of which I’m

aware. As with jihadist websites, a pragmatic emphasis on good old-fashioned

policing is much preferable to fearmongering and grand political gestures.

24.5 Forensics and Rules of Evidence

This leads us naturally to the last main topic in the justice space, namely

how information can be recovered from computers, mobile phones and other

electronic devices for use in evidence. The three big changes in recent years

have been, ﬁrst, the sheer volumes of data; second, the growth of search

engines and other tools to ﬁnd relevant material; and third, that courts are

becoming gradually more relaxed and competent.

24.5.1 Forensics

When the police raid even a small-time drug dealer nowadays, they can get

well over a Terabyte of data: several laptops, half-a-dozen mobile phones, a

804 Chapter 24 ■Terror, Justice and Freedom

couple of iPods and perhaps a box of memory sticks. The suspect may also

have dozens of accounts online for webmail services, social-networking sites

and other services. He may have interesting gadgets — such as navigators that

hold his location history (much of which is also available, with less resolution,

via his mobile phone records). Security researchers have found all sorts of

clever ways of extracting information from the data — for example, you can

identify which camera took a picture from the pattern noise of the CCD

array [818], and the number of such tricks can only increase.

The use of all this material in evidence depends, in most countries, on

following certain procedures. Material has to be lawfully collected, whether

with a search warrant or equivalent powers; and the forensic ofﬁcer has to

maintain a chain of custody, which means being able to satisfy a court that

evidence wasn’t tampered with afterwards. The details can vary from one

jurisdiction to another, and I’ll describe them in the next section.

The basic procedure is to use tools that have been appropriately tested and

evaluated to make trustworthy copies of data, which may mean computing a

one-way hash of the data so as to establish its authenticity later; to document

everything that’s done; and to have means of dealing appropriately with

any private material that’s found (such as privileged attorney-client emails,

or the trade secrets of the suspect’s employer). The details can be found in

standard forensics textbooks such as Sammes and Jenkinson [1105], and much

of the technical complexity comes from the proliferation of mobile phones,

organisers, iPods and other storage devices, which the practitioner should be

able to deal with. Indeed, as time goes on, specialist ﬁrms are springing up

that deal with phones and other less common types of kit.

Computer forensics pose increasingly complex engineering problems. A

recent example is that many police forces adopted a rigid procedure of always

turning PCs off, so that hard disks could be mirrored and multiple copies

made for prosecution and defence lawyers. The Rockphish gang exploited this

by making their phishing software memory-resident. The police would arrive

at a house containing a phishing server, inform the startled householder that

his PC was being used for wicked purposes, switch the machine off — and lose

all the information that would have let them trace the real server for which the

seized machine had been acting as a proxy.

A related problem is that Windows Vista ships with Bitlocker, a disc

encryption utility that stores keys in the TPM chip on the motherboard, and

thus makes ﬁles unusable after the machine’s switched off unless you know

thepassword.WhiletheUKnowhasalawenablingcourtstojailpeoplefor

failing to supply a password, most countries don’t; so thoughtful police forces

now operate a rule whereby a decision on whether to switch the machine off

is at the ofﬁcer’s discretion. It’s a judgment call whether to risk losing data by

turning the machine off, or to image it when it’s running and risk the defence

lawyers arguing that it was tampered with. Truth to tell, however, the forensic

24.5 Forensics and Rules of Evidence 805

folks are still not discovering any great technical sophistication among normal

criminals.

Another issue is that it may be important to minimise the disruption caused

by forensic copying, especially where the machine belongs to someone other

than an arrested suspect. Even where a machine is conﬁscated from a suspect,

it can be problematic if you take too long to examine it. For example, in the

Operation Ore cases I mentioned in the last section, many people who later

turned out to be innocent had their PCs taken away and stored for months

or even years because the police didn’t have the forensic capacity to cope.

As a result they remained under a cloud of suspicion for much longer than

was reasonable; this had an adverse effect on people in regulated professions,

leading to litigation against the police.

There’s also the issue of loss of access to data, which for an individual or

small business can be catastrophic. I reckon it’s prudent practice nowadays

for a student to have seizure-proof offsite backup, for example by getting a

Gmail account and emailing copies of your thesis there regularly as you write

it. Otherwise your house might be raided and both your PC and backups

removed to the police forensic lab for months or even years. And it needn’t be

your fault; perhaps the guy on the second ﬂoor is smoking dope, or running a

supernode in a music ﬁle-sharing system. You can just never tell.

Another forensic pitfall is relying on evidence extracted from the systems of

one party to a dispute, without applying enough scepticism about claims made

for its dependability. Recall the Munden case I described in section 10.4.3. A

man was falsely accused and wrongly convicted of attempted fraud after he

complained of unauthorized withdrawals from his bank account. On appeal,

his defence team got an order from the court that the bank open its systems to

the defence expert as it had done to the prosecution. The bank refused, the bank

statements were ruled inadmissible and the case collapsed. So it’s worthwhile

when relying on forensic evidence supplied by a disputant to think in advance

about whether it will have to withstand examination by hostile experts.

In general, when designing a system you should stop and think about

the forensic aspects. You may want it not to provide evidence; an example

is the policy adopted by Microsoft after their antitrust battles with the U.S.

government, at which embarrassing emails came out. The ﬁrm reacted with a

policy that all emails should be discarded after a ﬁxed period of time unless

someone took positive action to save them. In other circumstances you may

want your system to provide evidence. Then there’s not just the matter of

whether the relevant data are preserved, and for how long (if your local statute

of limitations for civil claims is seven years, you’ll probably want to keep

business data for at least this long), but also how the data are to be extracted.

In many jurisdictions, court rules admit evidence only if it passes certain tests,

for example that it was generated in the normal course of business operations.

So we need to look at such requirements next.

806 Chapter 24 ■Terror, Justice and Freedom

24.5.2 Admissibility of Evidence

When courts were ﬁrst confronted with computer evidence in the 1960s there

were many concerns about its reliability. There was not just the engineering

issue of whether the data were accurate, but the legal issue of whether

computer-generated data were inadmissible on the grounds that they were

hearsay. Different legislatures tackled this differently. In the U.S. most of the

law is found in the Federal Rules of Evidence where computer records are

usually introduced as business records. We ﬁnd at 803(6):

Records of regularly conducted activity. A memorandum, report, record, or

data compilation, in any form, of acts, events, conditions, opinions, or diag-

noses, made at or near the time by, or from information transmitted by, a

person with knowledge, if kept in the course of a regularly conducted busi-

ness activity, and if it was the regular practice of that business activity to

make the memorandum, report, record, or data compilation, all as shown by

the testimony of the custodian or other qualiﬁed witness, unless the source of

information or the method or circumstances of preparation indicate lack

of trustworthiness. The term ‘business’ as used in this paragraph includes

business, institution, association, profession, occupation, and calling of every

kind, whether or not conducted for proﬁt.

The UK is similar: the Civil Evidence Act 1995 covers civil litigation while

the Police and Criminal Evidence Act 1984 deals with criminal matters5. The

requirement that the machine be operated in the normal course of business

can cause problems when machines have to be operated in abnormal ways

to extract information. In one case in my own experience, a woman was

accused of stealing a debit card from the mail and the police wished to

ascertain whether a torn-off corner of a PIN mailer found in her purse would

activate the stolen card. So they got the branch manager to put the card

into a statement printer in the branch, entered the PIN, and the card was

conﬁscated. The manager testiﬁed that the way the card was conﬁscated

showed that it was because the account had been closed rather than because

the PIN was wrong. However, the court ruled this evidence to be inadmissible.

The rules of electronic evidence in the common-law countries (England, the

USA, Canada, Australia, South Africa and Singapore) are analysed in detail

by Stephen Mason [838]; for a summary of relevant U.S. cases, read Orin

Kerr [714].

There are some special legal provisions for particular technologies, many of

them enacted during or shortly after the dotcom boom as legislators sought to

smooth the path for e-commerce without really understanding the problems.

5The latter used to require a certiﬁcate from the machine operator to the effect that the equipment

was working normally, but this was dropped as it caused problems with evidence from hacked

machines.

24.5 Forensics and Rules of Evidence 807

Many industry lobbyists claimed that e-commerce was held up by uncertainty

about whether electronic documents would be accepted as ‘writing’ for those

laws that required certain transactions to be written (typical examples are

real-estate transfers and patent licenses). Legislatures, starting with Utah’s,

therefore introduced laws granting special status to digital signatures. In most

cases these had no effect, as courts took the sensible view that an email is

writing just as a letter is: the essence of a signature is the signer’s intent, and

courts had long decided cases this way. For example, a farm worker who was

crushed to death by a tractor and had managed to scrawl ‘all to mum’ on the

tyre was held to have made a legal will [1358, 1359]. For surveys of digital

signature laws, see [109, 524].

However there’s one case in which eager legislators got it completely wrong,

and that’s Europe. The Electronic Signature Directive, which came into force

in 2000, compels all Member States to give special force to an advanced elec-

tronic signature, which basically means a digital signature generated with a

smartcard. Europe’s smartcard industry thought this would earn them lots

of money. However, it had the opposite effect. At present, the risk that a

paper check will be forged is borne by the relying party: if someone forges

a check on my account, then it’s not my signature, and I have not given

the bank my mandate to debit my account; so if they negligently rely on a

forged signature and do so, that’s their lookout6. However, if I were foolish

enough to ever accept an advanced electronic signature device, then there

would be a presumption of the validity of any signature that appeared to

have been made with it. All of a sudden, the risk shifts from the bank to me.

I become liable to anyone in the world for any signature that appears to have

been made by this infernal device, regardless of whether or not I actually

made it! This, coupled with the facts that smartcards don’t have a trusted

user interface and that the PCs which most people would use to provide

this interface are easily and frequently subverted, made electronic signatures

instantly unattractive.

Finally, a word on click-wrap. In 2000, the U.S. Congress enacted the

Electronic Signatures in Global and National Commerce (‘ESIGN’) Act, which

gives legal force to any ‘sound, symbol, or process’ by which a consumer

assents to something. So pressing a telephone keypad (‘press 0 to agree or

9 to terminate this transaction’), clicking a hyper-link to enter a web site,

or clicking ‘continue’ on a software installer, the consumer consents to be

bound to a contract [457]. This makes click-wrap licenses work in America.

The general view of lawyers in Europe is that they probably don’t work here,

but no-one’s eager to bring the ﬁrst case.

6Some countries, like Switzerland, let their banks shift the fraud risk to the account holder using

their terms and conditions, but Britain always prohibited this, ﬁrst by common law and then by

the Bills of Exchange Act 1886.

808 Chapter 24 ■Terror, Justice and Freedom

24.6 Privacy and Data Protection

Data protection is a term used in Europe to mean the protection of personal

information from inappropriate use. Personal information generally means

any data kept on an identiﬁable human being, or data subject,suchasbank

account details and credit card purchasing patterns. It corresponds roughly to

the U.S. term computer privacy. The difference in terminology is accompanied

by a huge difference in law and in attitudes. This is likely to remain a problem

for global business, and may get worse.

European law gives data subjects the right to inspect personal data held on

them, have them changed if inaccurate, understand how they’re processed,

and in many cases prevent them being passed on to other organizations

without their consent. There are exemptions for national security, but they

are not as complete as the spooks would like: there was a big row when it

turned out that data from SWIFT, which processes interbank payments, were

being copied to the Department of Homeland Security without the knowledge

of data subjects. European privacy authorities ruled that SWIFT had broken

European, Belgian and Swiss privacy law, and it agreed to stop processing

European data in the USA by the end of 2009 [995, 996].

Almost all commercial data are covered, and there are particularly stringent

controls on data relating to intimate matters such as health, religion, race,

sexual life and political afﬁliations. Finally, recent law prescribes that personal

data may not be sent to organizations in countries whose laws do not provide

comparable protection. In practice that means America and India, where legal

protections on privacy are fragmentary. The resolution so far is the safe harbour

agreement whereby a data processor in America or India promises to their

European customer to abide by European law. Many ﬁrms do this, pioneered

by Citibank which set up such an arrangement to process German cardholder

data in South Dakota. But this creates practical enforcement problems for EU

citizens who feel that their rights have been violated; they aren’t privy to

the contract, and may have a hard time persuading the U.S. Department of

Commerce to take action against a U.S. ﬁrm that is quite possibly obeying local

laws perfectly well. So the safe harbour provisions may well fail when tested

in court. For a discussion, see [1339]. We’ll have to wait until test cases ﬁnd

their way to the European Court.

If safe harbour fails, the cynical ﬁx may be to put the servers in a European

country with very lax enforcement, such as Britain, but even so there are

problems: the UK is currently in dispute with the European Commission,

which claims that British law falls short of European requirements on eleven

separate points [406]. Another is to insist that customers agree to their personal

data being shared before you do business with them. This works to some

24.6 Privacy and Data Protection 809

extent at present (it’s how U.S. medical insurers get away with their abuses),

but it doesn’t work for data protection as coercive consent is speciﬁcally

disallowed [66].

European privacy law didn’t spring full-formed from the brow of Zeus

though, and it may be helpful to look at its origins.

24.6.1 European Data Protection

Technofear isn’t a late twentieth century invention. As early as 1890, Jus-

tices Warren and Brandeis warned of the threat to privacy posed by ’recent

inventions and business methods’ — speciﬁcally photography and investiga-

tive journalism [1321]. Years later, after large retail businesses started using

computers in the 1950s and banks followed in the early 1960s, people started

to worry about the social implications if all a citizen’s transactions could be

collected, consolidated and analyzed. In Europe, big business escaped censure

by making the case that only government could afford enough computers to

be a serious privacy threat. Once people realised it was both economic and

rational for government to extend its grasp by using the personal data of all

citizens as a basis for prognosis, this became a human rights issue — given the

recent memory of the Gestapo in most European countries.

A patchwork of data protection laws started to appear starting with the

German state of Hesse in 1969. Because of the rate at which technology

changes, the successful laws have been technology neutral. Their common

theme was a regulator (whether at national or state level) to whom users

of personal data had to report and who could instruct them to cease and

desist from inappropriate processing. The practical effect was usually that the

general law became expressed through a plethora of domain-speciﬁc codes of

practice.

Over time, processing by multinational businesses became an issue too,

and people realised that purely local or national initiatives were likely to be

ineffective against them. Following a voluntary code of conduct promulgated

by the OECD in 1980 [991], data protection was entrenched by a Council

of Europe convention in January 1981, which entered into force in October

1985 [327]. Although strictly speaking this convention was voluntary, many

states signed up to it for fear of losing access to data processing markets. It

required signatory states to pass domestic legislation to implement at least

certain minimum safeguards. Data had to be obtained lawfully and processed

fairly, and states had to ensure that legal remedies were available when

breaches occurred.

The quality of implementation varied widely. In the UK, for example, Mar-

garet Thatcher unashamedly did the least possible to comply with European

810 Chapter 24 ■Terror, Justice and Freedom

law; a data protection body was established but starved of funds and technical

expertise, and many exemptions were provided for favored constituencies7.

In hard-line privacy countries, such as Germany, the data protection bod-

ies became serious law-enforcement agencies. Many other countries, such as

Australia, Canada, New Zealand and Switzerland passed comparable privacy

laws in the 1980s and early 1990s: some, like Switzerland, went for the German

model while others, like Iceland, followed the British one.

By the early 1990s it was clear that the difference between national laws

was creating barriers to trade. Many businesses avoided controls altogether

by moving their data processing to the USA. So data protection was ﬁnally

elevated to the status of full-blown European law in 1995 with a Data Protection

Directive [444]. This sets higher minimum standards than most countries had

required before, with particularly stringent controls on highly sensitive data

such as health, religion, race and political afﬁliation. It also prevents personal

information being shipped to ‘data havens’ such as the USA unless there are

comparable controls enforced by contract.

24.6.2 Differences between Europe and the USA

The history in the USA is surveyed in [933]; basically business managed to

persuade government to leave privacy largely to ‘self-regulation’. Although

there is a patchwork of state and federal laws, they are application-speciﬁc

and highly fragmented. In general, privacy in federal government records and

in communications is fairly heavily regulated, while business data are largely

uncontrolled. There are a few islands of regulation, such as the Fair Credit

Reporting Act of 1970, which governs disclosure of credit information and is

broadly similar to European rules; the Video Privacy Protection Act or ‘Bork

Bill’, enacted after a Washington newspaper published Judge Robert Bork’s

video rental history following his nomination to the U.S. Supreme Court; the

Drivers’ Privacy Protection Act, enacted to protect privacy of DMV records

after the actress Rebecca Schaeffer was murdered by an obsessed fan who

hired a private eye to ﬁnd her address; and the Health Insurance Portability

and Accountability Act which protects medical records and which I discussed

in Chapter 9. However U.S. privacy law also includes several torts that provide

a basis for civil action, and they cover a surprising number of circumstances;

for a survey, see Daniel Solove [1200]. There was also a landmark case in 2006,

when Choicepoint paid $10 m to settle a lawsuit brought by the FTC after it

failed to vet subscribers properly and let crooks buy the personal information

of over 160,000 Americans, leading to at least 800 cases of ‘identity theft’ [459].

7In one case where you’d expect there to be an exemption, there wasn’t; journalists who kept

notes on their laptops or PCs which identiﬁed people were formally liable to give copies of this

information to the data subjects on demand.

24.6 Privacy and Data Protection 811

That may have started to put privacy on CEOs’ radar. Yet, overall, privacy

regulation in the USA is slack compared with Europe.

Attitudes also differ. Some researchers report a growing feeling in the USA

that people have lost control of the uses to which their personal information is

put, while in some European countries privacy is seen as a fundamental human

right that requires vigorous legislative support; in Germany, it’s entrenched

in the constitution [1339]. But it must be said that there’s a persistent problem

here. As I discussed in section 7.5.4, people say that they value privacy, yet act

otherwise. The great majority of people, whether in the USA or Europe, will

trade their privacy for very small advantages. Privacy-enhancing technologies

have been offered for sale, yet most have failed in the marketplace.

There’s simply no telling how the gulf between the USA and Europe

on privacy laws will evolve over time. In recent years, Europe has been

getting less coherent: the UK in particular has been drifting towards the U.S.

model, with ever more relaxed enforcement; and the new Member States

that used to be part of the Soviet Union or Yugoslavia are not rocking the

boat. Commerce is certainly pulling in the U.S. direction. As I discussed in

section 7.5.4, technology simultaneously creates the incentive for greater price

discrimination and the means to do it by collecting ever more personal data.

Yet in other countries, courts have become more protective of citizens’ rights

post-9/11. In Germany, which generally takes the hardest line, privacy trumps

even the ‘war on terror’: the highest court found unconstitutional a 2001 police

action to create a ﬁle on over 30,000 male students or former students aged 18

to 40 from Muslim-majority countries — even though no-one was arrested as

a result. It decided that such exercises could be performed only in response to

concrete threats, not as a precautionary measure [244].

The ﬂip side of the privacy-law coin is freedom-of-information law. A

radical version of this is proposed by David Brin [227]. He reasons that the

falling costs of data acquisition, transmission and storage will make pervasive

surveillance technologies available to the authorities, so the only real question

is whether they are available to the rest of us too. He paints a choice between

two futures — one in which the citizens live in fear of an East German–style

police force and one in which ofﬁcials are held to account by public scrutiny.

The cameras will exist: will they be surveillance cams or webcams? He argues

that essentially all information should be open — including, for example, all

our bank accounts. Weaker versions of this have been tried: tax returns are

published in Iceland and in some Swiss cantons, and the practice cuts evasion,

as rich men fear the loss of social status that an artiﬁcially low declared

income would bring. Still weaker versions, such as the U.S. and U.K. Freedom

of Information Acts, still give some useful beneﬁt in ensuring that the ﬂow of

information between the citizen and the state isn’t all one-way. As technology

continues to develop, the privacy and freedom-of-information boundaries will

no doubt involve a lot of pushing and shoving.

812 Chapter 24 ■Terror, Justice and Freedom

There are some interesting engineering questions. For example, while U.S.

felony convictions remain on the record for ever, many European countries

have offender-rehabilitation laws, under which most convictions disappear

after a period of time that depends on the severity of the offence. But how

can such laws be enforced now that web search engines exist? The German

response is that if you want to cite a criminal case, you’re supposed to get an

ofﬁcially de-identiﬁed transcript from the court. In Italy, a convicted business

person got a court to order the removal from a government website of a record

of his conviction, after the conviction had expired. But if electronic newspaper

archives are searchable online, what good will this do — unless the identities

of all offenders are blocked from electronic reporting? There has recently, for

example, been much debate over the monitoring of former child sex offenders,

with laws in some states requiring that registers of offenders be publicly

available, and riots in the UK following the naming of some former offenders

by a Sunday newspaper. How can you rehabilitate offenders in a world

with Google? For example, do you tag the names of offenders in newspaper

accounts of trials with an expiration date, and pass laws compelling search

and archive services to respect them?

The upshot is that even if data is public, its use can still cause offences

under European privacy law. This causes peculiar difﬁculties in the USA,

where courts have consistently interpreted the First Amendment to mean

that you can’t stop the repetition of true statements in peacetime except in a

small number of cases8. So it’s hardly surprising that the current ﬂashpoint

between Europe and America over privacy concerns Google. The immediate

casus belli is that EU law requires personal data to be deleted once it’s no

longer needed, while Google built its systems to keep data such as clickstreams

indeﬁnitely. During 2007, the European data-protection folks brought this to

Google’s attention; the search engine has offered to de-identify clickstreams

after 18 months. Given the difﬁculty of doing inference control properly — as

I discussed in Chapter 9 — this claim will no doubt be examined closely by the

European authorities. No doubt this saga will run and run. Even in America,

there’s been a call from CDT and EFF for a ‘Do Not Track’ list, similar to the

Do Not Call list, so that people could opt out; other activists disagree, saying

this would undermine the paid-by-ads model of useful web services [1333].

In any case, less than a percent of people bother to use ad-blocking software.

We’ll have to wait and see.

24.7 Summary

Governments and public policy are entangled more and more with the work

of the security engineer. The ‘crypto wars’ were a harbinger of this, as were the

8The classic example is a regulated profession such as securities trading.

Further Reading

It’s extraordinarily easy for technopolicy arguments to get detached at one or

more corners from reality, and many of the nightmares conjured up to get

attention and money (such as ‘credit card transactions being intercepted on

the Internet’) are really the modern equivalent of the monsters that appeared

on medieval maps to cover up the cartographer’s ignorance. An engineer

who wants to build things that work and last has a duty not to get car-

ried away. For this reason, it’s particularly important to dig out primary

sources — material written by experienced insiders such as R.V. Jones [671]

and Gerard Walsh [1311].

There’s a good book on the history of wiretapping and crypto policy by

Whit Difﬁe and Susan Landau, who had a long involvement in the policy

814 Chapter 24 ■Terror, Justice and Freedom

process [387], an NRC study on cryptography policy was also inﬂuential [950];

and there’s a compilation of primary materials at [1135]. There’s also use-

ful stuff at the web sites of organizations such as EPIC [432], EFF [422],

FIPR [484], CDT [278], the Privacy Exchange [1048] and on mailing lists such

as politech [1031] and ukcrypto [1267].

There are many resources on online censorship, starting perhaps with the

OpenNet Initiative; and Reporters without Borders publish a ‘Handbook

for bloggers and cyber-dissidents’ that not only contains guides on how to

circumvent censorship, but a number of case histories of how blogging has

helped open up the media in less liberal countries [1069].

The standard work on computer forensics is by Tony Sammes and Brian

Jenkinson [1105], and there’s a nice article by Peter Sommer on the forensics

and evidential issues that arose when prosecuting some UK youngsters who

hacked the USAF Rome airbase [1202]. The Department of Justice’s ‘Guidelines

for Searching and Seizing Computers’ also bear some attention [381]. For

collections of computer crime case histories, see Peter Neumann [962], Dorothy

Denning [370] and Donn Parker [1005]. The standard work on computer

evidence in the common law countries is by Stephen Mason [838].

On the topic of data protection, there is a huge literature but no concise

guide that I know of. [1339] provides a good historical overview, with a

perspective on the coming collision between Europe and the USA. Simson

Garﬁnkel [515] and Michael Froomkin [504] survey privacy and surveillance

issues with special relevance to the USA.

CHAPTER

Managing the Development

of Secure Systems

My own experience is that developers with a clean, expressive set of speciﬁc security

requirements can build a very tight machine. They don’t have to be security gurus,

but they have to understand what they’re trying to build and how it should work.

— Rick Smith

One of the most important problems we face today, as techniques and systems

become more and more pervasive, is the risk of missing that ﬁne, human point that

may well make the difference between success and failure, fair and unfair, right and

wrong ... no IBM computer has an education in the humanities.

— Tom Watson

Management is that for which there is no algorithm. Where there is an algorithm,

it’s administration.

— Roger Needham

25.1 Introduction

So far we’ve discussed a great variety of security applications, techniques and

concerns. If you’re a working IT manager or consultant, paid to build a secure

system, you will by now be looking for a systematic way to select protection

aims and mechanisms. This brings us to the topics of system engineering, risk

analysis and, ﬁnally, the secret sauce: how you manage a team to write secure

code.

Business schools reckon that management training should be conducted

largely through case histories, stiffened with focussed courses on basic topics

such as law, economics and accounting. I have broadly followed their model

in this book. We went over the fundamentals, such as protocols, access control

815

816 Chapter 25 ■Managing the Development of Secure Systems

and crypto, and then looked at a lot of different applications with a lot of case

histories.

Now we have to pull the threads together and discuss how to go about solv-

ing a general security engineering problem. Organizational issues matter here

as well as technical ones. It’s important to understand the capabilities of the

staff who’ll operate your control systems, such as guards and auditors, to take

account of the managerial and work-group pressures on them, and get feed-

back from them as the system evolves. You also have to instil suitable ways of

thinking and working into your development team. Success is about attitudes

and work practices as well as skills. There are tensions: how do you get people

to think like criminals, yet work enthusiastically for the good of the product?

25.2 Managing a Security Project

The hardest part of the project manager’s job is usually ﬁguring out what to

protect and how. Threat modelling and requirements engineering are what

separate out the star teams from the also-rans.

The ﬁrst killer problem is understanding the tradeoff between risk and

reward. Security people naturally focus too much on the former and neglect

the latter. If the client has a turnover of $10 m, proﬁts of $1 m and theft losses

of $150,000, the security consultant may make a loss-reduction pitch about

‘how to increase your proﬁts by 15%’; but it could well be in the shareholders’

interests to double the turnover to $20 m, even if this triples the losses to

$450,000. Assuming the margins stay the same, the proﬁt is now $1.85 m,

up 85%.

So if you’re the owner of the company, don’t fall into the trap of believing

that the only possible response to a vulnerability is to ﬁx it, and distrust the

sort of consultant who can only talk about ‘tightening security’. Often it’s

too tight already, and what you really need to do is just focus it slightly

differently. But the security team — whether internal developers or external

consultants — usually has an incentive to play up the threats, and as it has more

expertise on the subject it’s hard to gainsay. The same mechanisms that drive

national overreaction to terrorist incidents are at work in the corporation too.

25.2.1 A Tale of Three Supermarkets

My thumbnail case history to illustrate this point concerns three supermarkets.

Among the large operational costs of running a retail chain are the salaries of

the checkout and security staff, and the stock shrinkage due to theft. Checkout

queues aggravate your customers, so cutting staff isn’t always an option, and

working them harder might mean more shrinkage. So what might technology

do to help?

25.2 Managing a Security Project 817

One supermarket in South Africa decided to automate completely. All

produce would carry an RFID tag, so that an entire trolley-load could be

scanned automatically. If this had worked, it could have killed both birds

with one stone; the same RFID tags could have cut staff numbers and made

theft harder. There was a pilot, but the idea couldn’t compete with barcodes.

Customers had to use a special trolley, which was large and ugly — and the RF

tags also cost money. There has been a lot of investment in RFID, but there’s

still a problem: tags ﬁxed to goods that conduct electricity, such as canned

drinks, are hard to read reliably.

Another supermarket in a European country believed that much of their

losses were due to a hard core of professional thieves, and wanted to use RFID

to cut this. When they eventually realized this wouldn’t work, they then talked

of building a face-recognition system to alert the guards whenever a known

villain came into a store. But current technology can’t do that with low enough

error rates. In the end, the chosen route was civil recovery. When a shoplifter

is caught, then even after the local magistrates have ﬁned her a few bucks,

the supermarket sues her in the civil courts for wasted time, lost earnings,

attorneys’ fees and everything else they can think of; and then armed with a

judgement for a few thousand bucks they go round to her house and seize

all the furniture. So far so good. But their management spent their time and

energy getting vengeance on petty thieves rather than increasing sales. Soon

they started losing market share and saw their stock price slide. Diverting

effort from marketing to security was probably a symptom of their decline

rather than a cause, but may have contributed to it.

The supermarket that seemed to be doing best when I wrote the ﬁrst edition

in 2001 was Waitrose in England, which had just introduced self-service

scanning. When you go into their store you swipe your store card in a machine

that dispenses a portable barcode scanner. You scan the goods as you pick

them off the shelves and drop them into your shopping bag. At the exit you

check in the scanner, get a printed list of everything you’ve bought, swipe your

credit card and head for the car park. This might seem rather risky — but then

so did the self-service supermarket back in the days when traditional grocers’

shops had all the goods behind the counter. In fact, there are several subtle

control mechanisms at work. Limiting the service to store card holders not

only lets you exclude known shoplifters, but also helps market the store card.

By having one you acquire a trusted status visible to any neighbors you meet

while shopping — so losing your card (whether by getting caught stealing,

or more likely falling behind on your payments) could be embarrassing. And

trusting people removes much of the motive for cheating as there’s no kudos

in beating the system. Of course, should the guard at the video screen see

someone lingering suspiciously near the racks of hundred-dollar wines, it can

always be arranged for the system to ‘break’ as the suspect gets to the checkout,

which gives the staff a non-confrontational way to recheck the bag’s contents.

818 Chapter 25 ■Managing the Development of Secure Systems

Since then, the other supermarkets in the UK have adopted self-service,

but the quality of the implementation varies hugely. The most defensive

stores — where the security folks have had too much say in the design — force

you to scan at specially-designed self-service checkout lanes, and weigh each

item as you place it on the packing stand after scanning, in an attempt to detect

anyone packing an item without scanning it ﬁrst. These systems are ﬂaky

and frequently interrupt the shopper with complaints, which communicate

distrust. Waitrose seems to be going from strength to strength, while the most

defensive store (with the most offensive systems) is facing a takeover bid as

Iwrite.

25.2.2 Risk Management

Security policies tend to come from a company’s risk management mecha-

nisms. Risk management is one of the world’s largest industries: it includes

not just security engineers but also ﬁre and casualty services, insurers, the road

safety industry and much of the legal profession. Yet it is startling how little is

really known about the subject. Engineers, economists, actuaries and lawyers

all come at the problem from different directions, use different language and

arrive at quite incompatible conclusions. There are also strong cultural factors

at work. For example, if we distinguish risk as being where the odds are known

but the outcome isn’t, from uncertainty where even the odds are unknown,

then most people are more uncertainty-averse than risk-averse. Where the

odds are directly perceptible, a risk is often dealt with intuitively; but even

there, our reactions are colored by the various cognitive biases discussed in

Chapter 2. Where the science is unknown or inconclusive, people are free to

project all sorts of fears and prejudices. But risk management is not just a

matter of actuarial science colored by psychology. Organisations matter too,

whether governments or businesses.

The purpose of business is proﬁt, and proﬁt is the reward for risk. Security

mechanisms can often make a real difference to the risk/reward equation

but ultimately it’s the duty of a company’s board of directors to get the

balance right. In this risk management task, they may draw on all sorts of

advice — lawyers, actuaries, security engineers — as well as listening to their

marketing, operations and ﬁnancial teams. A sound corporate risk manage-

ment strategy involves much more than attacks on information systems; there

are non-IT operational risks such as ﬁres and ﬂoods as well as legal risks,

exchange rate risks, political risks, and many more. Company bosses need the

big picture view to take sensible decisions, and a difﬁcult part of their task is to

ensure that advisers from different disciplines work together closely enough,

but without succumbing to groupthink.

In the culture that’s grown up since Enron and Sarbanes-Oxley, risk man-

agement is supposed to drive internal control. The theory and practice of

25.2 Managing a Security Project 819

this are somewhat divergent. In theory, internal controls are about mitigating

and managing the tensions between employees’ duty to maximise share-

holder utility, and their natural tendency to maximise their own personal

utility instead. At the criminal end of things this encompasses theft and fraud

from the company; I discussed in Chapter 10 how to control that. However,

internal controls are also about softer conﬂicts of interest. Managers build

empires; researchers tackle interesting problems rather than proﬁtable ones;

programmers choose tools and platforms that will look good on their CVs,

rather than those best suited to the company’s tasks. A large body of orga-

nizational theory applies microeconomic analysis to behaviour in ﬁrms in

an attempt to get a handle on this. One of its effects is the growing use of

stock options and bonus schemes to try to align employees’ interests with

shareholders’.

The practice of risk management has been largely determined by the rules

evolved by the Big Four audit ﬁrms in response to Sarbanes-Oxley. A typi-

cal ﬁrm will show that it’s discharging its responsibilities by keeping a risk

them in some kind of order. Risks then get ‘owners’, senior managers who

are responsible for monitoring them and deciding on any speciﬁc counter-

measures. Thus the ﬁnance director might be allocated exchange-rate and

interest-rate risks, some of which he’ll hedge; operational risks like ﬁres and

ﬂoods will be managed by insurance; and the IT director might end up with

the system risks. The actual controls tend to evolve over time, and I’ll discuss

the process in more detail in section 25.4.1.2 later.

25.2.3 Organizational Issues

It goes without saying that advisers should understand each others’ roles and

work together rather than trying to undermine each other. But, human nature

being what it is, the advisers may cosy up with each other and entrench a

consensus view that steadily drifts away from reality. So the CEO, or other

responsible manager, has to ask hard questions and stir the cauldron a bit.

It’s also important to have a variety of experts, and to constantly bring in

new people. One of the most important changes post-Enron is the expectation

that companies should change their auditors from time to time; and one of

the most valuable tasks the security engineer gets called on to perform is

when you’re brought in, as an independent outsider, to challenge groupthink.

On perhaps a third of the consulting assignments I’ve done, there’s at least

one person at the client company who knows exactly what the problem is

and how to ﬁx it — they just need a credible mercenary to beat up on the

majority of their colleagues who’ve got stuck in a rut. (This is one reason why

famous consulting ﬁrms that exude an air of quality and certainty may have a

competitive advantage over specialists, but a generalist consultant may have

820 Chapter 25 ■Managing the Development of Secure Systems

difﬁculty telling which of the ten different dissenting views from insiders is

the one that must be listened to.)

Although the goals and management structures in government may be

slightly different, exactly the same principles apply. Risk management is often

harder because people are more used to compliance with standards rather

than case-by-case requirements engineering. Empire-building is a particular

problem in the public sector. James Coyne and Normal Kluksdahl present

in [331] a classic case study of information security run amok at NASA. There,

the end of military involvement in Space Shuttle operations led to a security

team being set up at the Mission Control Center in Houston to ﬁll the vacuum

left by the DoD’s departure. This team was given an ambitious charter; it

became independent of both development and operations; its impositions

became increasingly unrelated to budget and operational constraints; and its

relations with the rest of the organization became increasingly adversarial. In

the end, it had to be overthrown or nothing would have got done.

The main point is that it’s not enough, when doing a security requirements

analysis, to understand the education, training and capabilities of the guards

(and the auditors, and the checkout staff, and everyone else within the trust

perimeter). Motivation is critical, and many systems fail because their designers

make unrealistic assumptions about it. Organizational structures matter. There

are also risk dynamics that can introduce instability. For example, an initially

low rate of fraud can make people complacent and careless, until suddenly

things explode. Also, an externally induced change in the organization — such

as a merger, political uncertainty — can undermine morale and thus control.

(Part of my younger life as a security consultant was spent travelling to places

where local traumas were causing bank fraud to rocket, such as Hong Kong in

1989.)

So you have to make allowance in your designs for the ways in which human

frailties express themselves through the way people behave in organizations.

25.2.3.1 The Complacency Cycle and the Risk Thermostat

Phone fraud in the USA has a seven year cycle: in any one year, one of the

‘Baby Bells’ is usually getting badly hurt. They hire experts, clean things up

and get everything under control — at which point another of them becomes

the favored target. Over the next six years, things gradually slacken off, then

it’s back to square one. This is a classic example of organizational complacency.

How does it come about?

Some interesting and relevant work has been done on how people manage

their exposure to risk. John Adams studied mandatory seat belt laws, and

established that they don’t actually save lives: they just transfer casualties

from vehicle occupants to pedestrians and cyclists. Seat belts make drivers

feel safer, so they drive faster in order to bring their perceived risk back up to

25.2 Managing a Security Project 821

its previous level. He calls this a risk thermostat and the model is borne out in

other applications too [10, 11]. The complacency cycle can be thought of as the

risk thermostat’s corporate manifestation. Firms where managers move every

two years and the business gets reorganised every ﬁve just can’t maintain

a long corporate memory of anything that wasn’t widespread knowledge

among the whole management; and problems that have been ‘solved’ tend to

be forgotten. But risk management is an interactive business that involves all

sorts of feedback and compensating behavior. The resulting system may be

stable, as with road trafﬁc fatalities; or it may oscillate, as with the Baby Bells.

Feedback mechanisms can also limit the performance of risk reduction

systems. The incidence of attacks, or accidents, or whatever the organization

is trying to prevent, will be reduced to the point at which there are not enough

of them — as with the alarm systems described in Chapter 10 or the intrusion

detection systems described in section 21.4.4. Then the sentries fall asleep, or

real alarms are swamped by false ones, or organizational budgets are eroded

to (and past) the point of danger. I mentioned in Chapter 12 how for 50 years

the U.S. Air Force never lost a nuclear weapon. Then the ﬁve people who

were supposed to check independently whether a cruise missile carried a live

warhead or a blank failed to do so — each relied on the others. Six warheads

were duly lost for 36 hours. Colonels will be court-martialled, and bombs will

be counted carefully for a while. But eventually the courts martial will be

forgotten. (How would you organize it differently?)

25.2.3.2 Interaction with Reliability

Poor internal control often results from systems where lots of transactions are

always going wrong and have to be corrected manually. Just as in electronic

warfare, noise degrades the receiver operating characteristic. A high tolerance

of chaos undermines control, as it creates a high false alarm rate for many of

the protection mechanisms at once. It also tempts staff: when they see that

errors aren’t spotted they conclude that theft won’t be either.

The correlation between quality and security is a recurring theme in the

literature. For example, it has been shown that investment in software quality

will reduce the incidence of computer security problems, regardless of whether

security was a target of the quality program or not; and that the most effective

quality measure from the security point of view is the code walk-through [470].

The knowledge that one’s output will be read and criticized has a salutary

effect on many programmers.

Reliability can be one of your biggest selling points when trying to get

a client’s board of directors to agree on protective measures. Mistakes cost

money; no-one really understands what software does; if mistakes are found

then the frauds should be much more obvious; and all this can be communi-

cated to top management without embarrassment on either side.

822 Chapter 25 ■Managing the Development of Secure Systems

25.2.3.3 Solving the Wrong Problem

Faced with an intractable problem, it is common for people to furiously attack

a related but easier one; we saw the effects of this in the public policy context

earlier in section 24.3.10. Displacement activity is also common in the private

sector, where an example comes from the smartcard industry. As discussed

in section 16.7.4, the difﬁculty of protecting smartcards against probing and

power-analysis attacks led the industry to concentrate on securing the chip

mask instead. Technical manuals are available only under NDA; plant visitors

have to sign an NDA at reception; much technical material isn’t available at

all; and vendor facilities have almost nuclear-grade physical security. Physical

security overkill may impress naive customers — but almost all of the real

attacks on ﬁelded smartcard systems used technical attacks that didn’t depend

on design information.

One organizational driver for this is an inability to deal with uncertainty.

Managers prefer approaches that they can implement by box-ticking their

way down a checklist. So if an organization needs to deal with an actual

risk, then some way needs to be found to keep it as a process, and stop it

turning into a due-diligence checklist item. But there is constant pressure to

replace processes with checklists, as they are less demanding of management

attention and effort. The quality bureaucracy gets in the way here; ﬁrms want-

ing quality-assurance certiﬁcation are prodded to document their business

processes and make them repeatable. I noted in section 8.7 that bureaucratic

guidelines had a strong tendency to displace critical thought; instead of think-

ing through a system’s protection requirements, designers just reached for

their checklists.

Another organizational issue is that when exposures are politically sensitive,

some camouﬂage may be used. The classic example is the question of whether

attacks come from insiders or outsiders. We’ve seen in system after system

that the insiders are the main problem, whether because some of them are

malicious or because most of them are careless. But it’s often hard to enforce

controls too overtly against line managers and IT staff, as this will alienate

them, and it’s also hard to get them to manage such controls themselves. It’s

not easy to sell a typical company’s board of directors on the need for proper

defences against insider attack, as this impugns the integrity and reliability

of the staff who report to them. Most company boards are (quite rightly) full of

entrepreneurial, positive, people with conﬁdence in their staff and big plans for

the future, rather than dyspeptic bean-counters who’d like to control everyone

even more closely. So the complaint of information security managers down

the years — that the board doesn’t care — may not actually be a bug, but a

feature.

Often a security manager will ask for, and get, money to defend against

nonexistent ‘evil hackers’ so that he can spend most of it on controls to manage

25.2 Managing a Security Project 823

the real threat, namely dishonest or careless staff. I would be cautious about

this strategy because protection mechanisms without clear justiﬁcations are

likely to be eroded under operational pressure — especially if they are seen

as bureaucratic impositions. Often it will take a certain amount of subtlety

and negotiating skill, and controls will have to be marketed as a way of

reducing errors and protecting staff. Bank managers love dual-control safe

locks because they understand that it reduces the risk of their families being

taken hostage; and requiring two signatures on transactions over a certain

limit means extra shoulders to take the burden when something goes wrong.

But such consensus on the need for protective measures is often lacking

elsewhere.

25.2.3.4 Incompetent and Inexperienced Security Managers

Things are bad enough when even a competent IT security manager has to use

guile to raise money for an activity that many of his management colleagues

regard as a pure cost. In real life, things are even worse. In many traditional

companies, promotions to top management jobs are a matter of seniority and

contacts; so if you want to get to be the CEO you’ll have to spend maybe

20 or 30 years in the company without offending too many people. Being a

security manager is absolutely the last thing you want to do, as it will mean

saying no to people all the time. It’s hardly surprising that the average tenure

of computer security managers at U.S. government agencies is only seven

months [605].

Matters are complicated by reorganizations in which central computer

security departments may be created and destroyed every few years, while

the IT audit function oscillates between the IT department, an internal audit

department and outside auditors or consultants. The security function is even

less likely than other business processes to receive sustained attention and

analytic thought, and more likely to succumb to a box-ticking due diligence

mentality. Also, the loss of institutional memory is often a serious problem.

25.2.3.5 Moral Hazard

Companies often design systems so that the risk gets dumped on third parties.

This can easily create a moral hazard by removing the incentives for people

to take care, and for the company to invest in risk management techniques.

I mentioned in Chapter 10 how banks in some countries claimed that their

ATMs could not possibly make mistakes, so that any disputes must be the

customer’s fault. This led to a rise in fraud as staff got lazy and even crooked.

So, quite in addition to the public policy aspects, risk denial can often make

the problem worse: a company can leave itself open to staff who defraud it

knowing that a prosecution would be too embarrassing.

824 Chapter 25 ■Managing the Development of Secure Systems

Another kind of moral hazard is created when people who take system

design decisions are unlikely to be held accountable for their actions. This can

happen for many reasons. IT staff turnover could be high, with much reliance

placed on contract staff; a rising management star with whom nobody wishes

to argue can be involved as a user in the design team; imminent business

process re-engineering may turn loyal staff into surreptitious job-seekers. In

any case, when you are involved in designing a secure system, it’s a good idea

to look round your colleagues and ask yourself which of them will shoulder

the blame three years later when things go wrong.

Another common incentive failure occurs when one part of an organization

takes the credit for the proﬁt generated by some activity, while another part

picks up the bills when things go wrong. Very often the marketing department

gets the praise for increased sales, while the ﬁnance department is left with the

bad debts. A rational ﬁrm would strike a balance between risk and reward,

but internal politics can make ﬁrms behave irrationally. The case of the three

supermarkets, mentioned above, is just one example. Companies may swing

wildly over a period of years from being risk takers to being excessively risk

averse, and (less often) back again. John Adams found that risk taking and risk

aversion are strongly associated with different personality types: the former

tend to be individualists, a company’s entrepreneurs, while the latter tend to

be hierarchists. As the latter usually come to dominate bureaucracies, it is not

surprising that stable, established organizations tend to be much more risk

averse than rational economics would dictate.

So what tools and concepts can help us cut through the fog of bureau-

cratic inﬁghting and determine a system’s protection requirements from ﬁrst

principles?

The rest of this chapter will be organized as follows. The next section

will look at basic methodological issues such as top-down versus iterative

development. After that, I’ll discuss how these apply to the speciﬁc problem

of security requirements engineering. Having set the scene, I’ll then return to

risk management and look at technical tools. Then I’ll come back and discuss

how you manage people. That’s really critical. How do you get people to care

about vulnerabilities and bugs? This is partly incentives and partly culture;

the two reinforce each other, and many companies get it wrong.

25.3 Methodology

Software projects usually take longer than planned, cost more than budgeted

for and have more bugs than expected. (This is sometimes known as ‘Cheops’

law’ after the builder of the Great Pyramid.) By the 1960s, this had become

known as the software crisis, although the word ‘crisis’ is hardly appropriate

25.3 Methodology 825

for a state of affairs that has now lasted (like computer insecurity) for two

generations. Anyway, the term software engineering was proposed by Brian

Randall in 1968 and deﬁned to be:

Software engineering is the establishment and use of sound engineering principles

in order to obtain economically software that is reliable and works efﬁciently on

real machines.

This encompassed the hope that the problem could be solved in the same

way that one builds ships and aircraft, with a proven scientiﬁc foundation

and a set of design rules [954]. Since then much progress has been made.

However, the results of the progress have been unexpected. Back in the late

1960s, people hoped that we’d cut the number of large software projects failing

from the 30% or so that was observed at the time. Now, we still see about 30%

of large projects failing — but the failures are much bigger. The tools get us

farther up the complexity mountain before we fall off, but the rate of failure

appears to be exogenous, set by such factors as company managers’ appetite

for risk1.

Anyway, software engineering is about managing complexity, of which

there are two kinds. There is the incidental complexity involved in programming

using inappropriate tools, such as the assembly languages which were all that

some early machines supported; programming a modern application with

a graphical user interface in such a language would be impossibly tedious

and error-prone. There is also the intrinsic complexity of dealing with large

and complex problems. A bank’s administrative systems, for example, may

involve tens of millions of lines of code and be too complex for any one person

to understand.

Incidental complexity is largely dealt with using technical tools. The most

important of these are high-level languages that hide much of the drudgery of

dealing with machine-speciﬁc detail and enable the programmer to develop

code at an appropriate level of abstraction. There are also formal methods that

enable particularly error-prone design and programming tasks to be checked.

The obvious security engineering example is provided by the BAN logic for

verifying cryptographic protocols, described in section 3.8.

Intrinsic complexity usually requires methodological tools that help divide

up the problem into manageable subproblems and restrict the extent to which

these subproblems can interact. There are many tools on the market to help

you do this, and which you use may well be a matter of your client’s policy.

But there are basically two approaches — top-down and iterative.

1A related, and serious, problem is that while 30% of large projects fail in industry, perhaps only

30% of large projects in the public sector succeed; for a topical case history, see the National

Academies’ report on the collapse of the FBI’s attempt to modernise its case ﬁle system [860].

826 Chapter 25 ■Managing the Development of Secure Systems

25.3.1 Top-Down Design

The classical model of system development is the waterfall model developed

by Win Royce in the 1960s for the U.S. Air Force [1090]. The idea is that you

start from a concise statement of the system’s requirements; elaborate this into

a speciﬁcation; implement and test the system’s components; then integrate

them together and test them as a system; then roll out the system for live

operation (see Figure 25.1). Until recently, this was how all systems for the

U.S. Department of Defense had to be developed.

The idea is that the requirements are written in the user language, the

speciﬁcation is written in technical language, the unit testing checks the units

against the speciﬁcation and the system testing checks whether the require-

ments are met. At the ﬁrst two steps in this chain there is feedback on

whether we’re building the right system (validation) and at the next two

on whether we’re building it right (veriﬁcation). There may be more than

four steps: a common elaboration is to have a sequence of reﬁnement steps as

the requirements are developed into ever more detailed speciﬁcations. But

that’s by the way.

The critical thing about the waterfall model is that development ﬂows

inexorably downwards from the ﬁrst statement of the requirements to the

deployment of the system in the ﬁeld. Although there is feedback from each

stage to its predecessor, there is no system-level feedback from (say) system

testing to the requirements. Therein lie the waterfall model’s strengths, and

also its weaknesses.

The strengths of the waterfall model are that it compels early clariﬁcation

of system goals, architecture, and interfaces; it makes the project manager’s

Requirements

Validate Specification

Validate

Implementation

& unit testing

Verify

Integration &

system testing

Verify

Refine

Code

Build

Field

Operations &

maintenance

Figure 25.1: The waterfall model

25.3 Methodology 827

task easier by providing deﬁnite milestones to aim at; it may increase cost

transparency by enabling separate charges to be made for each step, and

for any late speciﬁcation changes; and it’s compatible with a wide range

of tools. Where it can be made to work, it’s often the best approach. The

critical question is whether the requirements are known in detail in advance

of any development or prototyping work. Sometimes this is the case, such as

when writing a compiler or (in the security world) designing a cryptographic

processor to implement a known transaction set and pass a certain level of

evaluation.

But very often the detailed requirements aren’t known in advance and then

an iterative approach is necessary. There are quite a few possible reasons

for this. Perhaps the requirements aren’t understood yet by the customer,

and a prototype is necessary to clarify them rather than more discussion;

the technology may be changing; the environment could be changing; or

a critical part of the project may involve the design of a human-computer

interface, which will probably involve several prototypes. In fact, very often

the designer’s most important task is to help the customer decide what he

wants, and although this can sometimes be done by discussion, there will

often be a need for some prototyping2.

The most common reason of all for using an iterative development is that

we’re starting from an existing product which we want to improve. Even in

the early days of computing, most programmer effort was always expended

on maintaining and enhancing existing programs rather than developing new

ones. Nowadays, as software becomes ever more packaged and the packages

become ever more complex, the reality in many software ﬁrms is that ‘the

maintenance is the product’. The only way to write something as complex as

Ofﬁce is to start off from an existing version and enhance it. That does not

mean that the waterfall model is obsolete; on the contrary, it may be used to

manage a project to develop a major new feature. However, we also need

to think of the overall management of the product, and that’s likely to be based

on iteration.

25.3.2 Iterative Design

So many development projects need iteration, whether to ﬁrm up the speciﬁ-

cation by prototyping, or to manage the complexity of enhancing an already

large system.

2The Waterfall Model had a precursor in a methodology developed by Gerhard Pahl and

Wolfgang Beitz in Germany just after World War 2 for the design and construction of mechanical

equipment such as machine tools [1001]; apparently one of Pahl’s students later recounted that it

was originally designed as a means of getting the engineering student started, rather than as an

accurate description of what experienced designers actually do. Win Royce also saw his model

as a means of starting to get order out of chaos, rather than as a totally prescriptive system it

developed into.

828 Chapter 25 ■Managing the Development of Secure Systems

In the ﬁrst case, a common approach is Barry Boehm’s spiral model in which

development proceeds through a pre-agreed number of iterations in which a

prototype is built and tested, with managers being able to evaluate the risk

at each stage so they can decide whether to proceed with the next iteration

or to cut their losses. It’s called the spiral model because the process is often

depicted as in Figure 25.2.

In the second case, the standard model is evolutionary development. An early

advocate for this approach was Harlan Mills, who taught that one should

build the smallest system that works, try it out on real users, and then add

functionality in small increments. This is how the packaged software industry

works: software products nowadays quickly become so complex that they

could not be economically developed (or redeveloped) from scratch. Indeed,

Microsoft has tried more than once to rewrite Word, but gave up each time.

Perhaps the best book on the evolutionary development model is by Maguire,

a Microsoft author [829]. In this view of the world, products aren’t the result

of a project but of a process that involves continually modifying previous

versions.

Progress

Prototype

Development

plan

Risk

analysis

Product

design

Settle final design

Code

Test system

Ship

Commit Test

Figure 25.2: The spiral model

25.3 Methodology 829

Unfortunately, evolutionary development tends to be neglected in academic

courses and books on software engineering, and it can cause some particular

problems for the security engineer.

The critical thing about evolutionary development is that just as each

generation of a biological species has to be viable for the species to continue,

so each generation of an evolving software product must be viable. The

core technology is regression testing. At regular intervals — typically once a

day — all the teams working on different features of a product check in their

code, which gets compiled to a build that is then tested automatically against

a large set of inputs. The regression test checks whether things that used to

work still work, and that old bugs haven’t found their way back. Of course,

it’s always possible that a build just doesn’t work at all, and there may be

quite long disruptions as a major change is implemented. So we consider the

current ‘generation’ of the product to be the last build that worked. One way

or another, we always have viable code that we can ship out for beta testing or

whatever our next stage is.

The technology of testing is probably the biggest practical improvement

in software engineering during the 1990s. Before automated regression tests

were widely used, engineers used to reckon that 15% of bug ﬁxes either

introduced new bugs or reintroduced old ones [9]. But automated testing

is less useful for the security engineer for a number of reasons. Security

properties are more diverse, and security engineers are fewer in number,

so we haven’t had as much investment in tools and the available tools are

much more fragmentary and primitive than those available to the general

software engineering community. Many of the ﬂaws that we want to ﬁnd and

ﬁx — such as stack overﬂow attacks — tend to appear in new features rather

than to reappear in old ones. Speciﬁc types of attack are also often easier to

ﬁx using speciﬁc remedies — such as the canary in the case of stack overﬂow.

And many security ﬂaws cross a system’s levels of abstraction, such as when

speciﬁcation errors interact with user interface features — the sort of problem

for which it’s difﬁcult to devise automated tests. But regression testing is still

really important. It ﬁnds functionality that has been affected by a change but

not fully understood.

Much the same applies to safety critical systems, which are similar in many

respects to secure systems. Some useful lessons can be drawn from them.

25.3.3 Lessons from Safety-Critical Systems

Critical computer systems can be deﬁned as those in which a certain class

of failure is to be avoided if at all possible. Depending on the class of

failure, they may be safety-critical, business-critical, security-critical, critical to

the environment or whatever. Obvious examples of the safety-critical variety

include ﬂight controls and automatic braking systems. There is a large literature

830 Chapter 25 ■Managing the Development of Secure Systems

on this subject, and a lot of methodologies have been developed to help manage

risk intelligently.

Overall, these methodologies tend to follow the waterfall view of the

universe. The usual procedure is to identify hazards and assess risks; decide

on a strategy to cope with them (avoidance, constraint, redundancy ...); to trace

the hazards down to hardware and software components which are thereby

identiﬁed as critical; to identify the operator procedures which are also critical

and study the various applied psychology and operations research issues; and

ﬁnally to decide on a test plan and get on with the task of testing. The outcome

of the testing is not just a system you’re conﬁdent to run live, but a safety case

to justify running it.

The safety case will provide the evidence, if something does go wrong, that

you exercised due care; it will typically consist of the hazard analysis, the

documentation linking this to component reliability and human factor issues,

and the results of tests (both at component level and system level) which show

that the required failure rates have been achieved.

The ideal system design avoids hazards entirely. A good illustration comes

from the motor reversing circuits in Figure 25.3. In the ﬁrst design on the left,

a double-pole double-throw switch reverses the current passing from the

battery through the motor. However, this has a potential problem: if only one

of the two poles of the switch moves, the battery will be short circuited and

a ﬁre may result. The solution is to exchange the battery and the motor, as in

the modiﬁed circuit on the right. Here, a switch failure will only short out the

motor, not the battery.

Hazard elimination is useful in security engineering too. We saw an example

in the early design of SWIFT in section 10.3.1: there, the keys used to authen-

ticate transactions between one bank and another were exchanged between

the banks directly. In this way, SWIFT personnel and systems did not have

the means to forge a valid transaction and had to be trusted much less. In

general, minimizing the trusted computing base is to a large extent an exercise

in hazard elimination.

•

•M

(a) (b)

Figure 25.3: Hazard elimination in motor reversing circuit

25.3 Methodology 831

Once as many hazards have been eliminated as possible, the next step is to

identify failures that could cause accidents. A common top-down way of iden-

tifying the things that can go wrong is fault tree analysis as a tree is constructed

whose root is the undesired behavior and whose successive nodes are its pos-

sible causes. This carries over in a fairly obvious way to security engineering,

and here’s an example of a fault tree (or threat tree, as it’s often called in security

engineering) for fraud from automatic teller machines (see Figure 25.4).

Successful card forgery

Shoulder

surfing

Cryptanalysis of DES

False

terminal

attack

Abuse of

security

module

Trojan Theft of

keys

Bank insider Maintenance

contractor

Bug in

ATM

Encryption

replacement

Falsify

auth

response

Protocol failure

Figure 25.4: A threat tree

Threat trees are used in the U.S. Department of Defense. You start out

from each undesirable outcome, and work backwards by writing down each

possible immediate cause. You then work backwards by adding each precursor

condition, and recurse. Then by working round the tree’s leaves you should be

able to see each combination of technical attack, operational blunder, physical

penetration and so on which would break security. This can amount to an

attack manual for the system, and so it may be highly classiﬁed. Nonetheless, it

must exist, and if the system evaluators or accreditors can ﬁnd any signiﬁcant

extra attacks, then they may fail the product.

Returning to the safety-critical world, another way of doing the hazard

analysis is failure modes and effects analysis (FMEA), pioneered by NASA, which

is bottom-up rather than top-down. This involves tracing the consequences

of a failure of each of the system’s components all the way up to the effect

on the mission. This is often useful in security engineering; it’s a good idea

to understand the consequences of a failure of any one of your protection

mechanisms.

832 Chapter 25 ■Managing the Development of Secure Systems

A really thorough analysis of failure modes may combine top-down and

bottom-up approaches. There are various ways to manage the resulting mass

of data. For example, one can construct a matrix of hazards against safety

mechanisms, and if the safety policy is that each serious hazard must be

constrained by at least two independent mechanisms, then we can check that

there are two entries in each of the relevant columns. In this way, we can

demonstrate graphically that in the presence of the hazard in question, at least

two failures will be required to cause an accident. This methodology goes

across unchanged to security engineering, as I’ll discuss below.

The safety-critical systems community has a number of techniques for deal-

ing with failure and error rates. Component failure rates can be measured

statistically; the number of bugs in software can be tracked by various tech-

niques which I’ll discuss in the next chapter; and there is a lot of experience

with the probability of operator error at different types of activity. The bible for

human-factors engineering in safety-critical systems is James Reason’s book

‘Human Error’; I would probably consider anyone who was building human

interfaces to security-critical systems and who hadn’t read this book to be

negligent if something went wrong.

The telegraphic summary is that the error rate depends on the familiarity

and complexity of the task, the amount of pressure and the number of cues to

success. Where a task is simple, performed often and there are strong cues

to success, the error rate might be 1 in 100,000 operations. However, when a

task is performed for the ﬁrst time in a confusing environment where logical

thought is required and the operator is under pressure, then the odds can

be against successful completion of the task. Quite a lot is known about the

cognitive biases and other psychological factors that make particular types of

error more common, and a prudent engineer will understand and avoid these.

Nonetheless, designers of things like nuclear reactors are well aware (at least

since Three Mile Island) that no matter how many design walkthroughs you

do, it’s when the red lights go on for the ﬁrst time that the worst mistakes

get made.

Similarly, in security systems, it tends to be important but rarely performed

tasks such as getting senior managers to set up master crypto keys where the

most egregious blunders can be expected. A classic example from [34] was

when a bank wished to create a set of three master keys to link their cash

machine network to VISA and needed a terminal to drive the security module.

A contractor obligingly lent them a laptop PC, together with software which

emulated the desired type of terminal. With this the senior managers duly

created the required keys and posted them off to VISA. None of them realized

that most PC terminal emulation software packages can be set to log all the

transactions passing through, and this is precisely what the contractor did. He

captured the clear zone key as it was created, and later used it to decrypt the

25.3 Methodology 833

bank’s master PIN key. The lesson to take from this is that security usability

isn’t just about presenting a nice intuitive interface to the end-user that accords

with common mental models of threat and protection in the application area,

as discussed in Chapter 2. It’s pervasive, and extends all the way through the

system’s operations, at the back end as well as the front.

So when doing security requirements engineering, special care has to be paid

to the skill level of the staff who will perform each critical task and estimates

made of the likelihood of error. Be cautious here: an airplane designer can rely

on a fairly predictable skill level from anyone with a commercial pilot’s licence,

and even a shipbuilder knows the strengths and weaknesses of a sailor in the

Navy. Usability testing can (and should) be integrated with staff training: when

pilots go for their six-monthly refresher courses in the simulator, instructors

throw all sorts of combinations of equipment failure, bad weather, cabin crisis

and air-trafﬁc-control confusion at them. They observe what combinations of

stress result in fatal accidents, and how these differ across cockpit types. This

in turn provides valuable feedback to the cockpit designers.

The security engineer usually has no such luck. Many security failures

remind me of a remark made by a ranger at Yosemite about the devices

provided to keep bears from getting at campers’ food supplies: that it’s an

impossible engineering problem because the brighter bears are smarter than

the dumber campers.

As well as the problem of testing usability, there are also technical testability

issues. A common problem with redundant systems is fault masking:ifthe

output is determined by majority voting between three processors, and one of

them fails, then the system will continue to work ﬁne — but its safety margin

will have been eroded. Several air crashes have resulted from ﬂying an airliner

with one of the ﬂight control systems dysfunctional; although pilots may be

intellectually aware that one of the cockpit displays is unreliable, their training

may lead them to rely on it under pressure rather than checking with other

instruments. So a further failure can be catastrophic. In such cases, it’s better to

arrange things so that displays give no reading at all rather than an inaccurate

one. A security example is the ATM problem mentioned in section 10.4.2 where

a bank issued all its customers with the same PIN. In such cases, the problem

often isn’t detected until much later. The fault gets masked by the handling

precautions applied to PINs, which ensure that even the bank’s security and

audit staff only get hold of the PIN mailer for their own personal account. So

some thought is needed about how faults can remain visible and testable even

when their immediate effects are masked.

Our ﬁnal lesson from safety critical systems is that although there will be a

safety requirements speciﬁcation and safety test criteria as part of the safety

case for the lawyers or regulators, it is good practice to integrate this with the

general requirements and test documentation. If the safety case is a separate

834 Chapter 25 ■Managing the Development of Secure Systems

set of documents, then it’s easy to sideline it after approval is obtained and

thus fail to maintain it properly. If, on the other hand, it’s an integral part of

the product’s management, then not only will it likely get upgraded as the

product is, but it is also much more likely to be taken heed of by experts from

other domains who might be designing features with possible interactions.

As a general rule, safety must be built in as a system is developed, not

retroﬁtted; the same goes for security. The main difference is in the failure

model. Rather than the effects of random failure, we’re dealing with a hostile

opponent who can cause some of the components of our system to fail at the

least convenient time and in the most damaging way possible. In effect, our

task is to program a computer which gives answers which are subtly and

maliciously wrong at the most inconvenient moment possible. I’ve described

this as ‘programming Satan’s computer’ to distinguish it from the more

common problem of programming Murphy’s [74]. This provides an insight

into one of the reasons security engineering is hard: Satan’s computer is hard

to test [1126].

25.4 Security Requirements Engineering

In Chapter 8, I deﬁned a security policy model to be a concise statement of

the protection properties that a system, or generic type of system, must have.

This was driven by the threat model, which sets out the attacks and failures

with which the system must be able to cope. The security policy model is

further reﬁned into a security target, which is a more detailed description of

the protection mechanisms a speciﬁc implementation provides, and how they

relate to the control objectives. The security target forms the basis for testing

and evaluation of a product. The policy model and the target together may

be referred to loosely as the security policy, and the process of developing a

security policy and obtaining agreement on it from the system owner is the

process of requirements engineering.

Security requirements engineering is often the most critical task of managing

secure system development, and can also be the hardest. It’s where ‘the rubber

hits the road’. It’s at the intersection of the most difﬁcult technical issues, the

most acute bureaucratic power struggles, and the most determined efforts at

blame avoidance.

The available methodologies have consistently lagged behind those available

to the rest of the system engineering world [123]. In my view, the critical insight

is that the process of generating a security policy and a security target is not

essentially different from the process of producing code. Depending on the

application, you can use a top-down, waterfall approach; a limited iterative

approach such as the spiral model; or a continuing iterative process such as

the evolutionary model. In each case, we need to build in the means to manage

25.4 Security Requirements Engineering 835

risk, and have the risk assessment drive the development or evolution of the

security policy.

Risk management must also continue once the system is deployed. It’s

rather hard to tell what a new invention will be useful for, and this applies to

the dark side too: novel attacks are just as difﬁcult to predict as anything else

about the future. Phone companies spent the 1970s ﬁguring out ways to stop

phone phreaks getting free calls, but once premium-rate numbers appeared

the real problem became stopping fraud. We worried about crooks hacking

bank smartcards, and put in lots of back-end protection for the early electronic

purses; the attacks came on pay-TV smartcards instead, while the bank fraud

folks concentrated on mag-stripe fallback and on phishing. People worried

about the security of credit card numbers used in transactions on the net, but it

turned out that the main threat to online businesses was refunds and disputes.

As they say, ‘The street ﬁnds its own uses for things.’ So you can’t expect to get

the protection requirements completely right at the ﬁrst attempt. We’ve also

seen many cases where the policy and mechanisms were set when a system

was ﬁrst built, and then undermined as the environment (and the product)

evolved, but the protection did not.

If you’re running a company, it’s futile to spend a lot of money on trying

to think up new attacks; that’s research, and best left to university folks like

me. What you do need is twofold: a mechanism for stopping your developers

building systems that are vulnerable to known bugs like stack overﬂows and

weak cryptography; and a mechanism for monitoring, and acting on, changing

protection requirements.

Unlike in the previous section, we’ll look at the case of evolving protection

requirements ﬁrst, as it is more common.

25.4.1 Managing Requirements Evolution

Most of the time, security requirements have to be tweaked for one of four

reasons. First, we might need to ﬁx a bug. Second, we may want to improve

the system; as we get more experience of the kind of attacks that happen, we

will want to tune some aspect of the controls. Third, we may want to deal with

an evolving environment. For example, if an online ordering system that was

previously limited to a handful of major suppliers is to be extended to all a

ﬁrm’s suppliers then the controls are likely to need review. Finally, there may

be a change in the organization. Firms are continually undergoing mergers,

management buyouts, business process re-engineering, you name it.

Of course, any of these could result in such a radical change that we would

consider it to be a redevelopment rather than an evolution. The dividing line

between the two is inevitably vague, but many evolutionary ideas carry over

into one-off projects, and many systems’ evolution contains occasional large

leaps that are engineered in speciﬁc projects.

836 Chapter 25 ■Managing the Development of Secure Systems

25.4.1.1 Bug Fixing

Most security enhancements fall into the category of bug ﬁxes or product

tuning. Fortunately, they are usually the easiest to cope with provided you

have the right mechanisms for getting information about bugs, testing ﬁxes

and shipping upgrades.

If you sell software that’s at all security critical — and most anything that

can communicate with the outside world is potentially so — then the day will

come when you hear of a vulnerability or even an attack. In the old days,

vendors could take months to respond with a new version of the product, and

would often do nothing at all but issue a warning (or even a denial). That

doesn’t work any more: public expectations are higher now. With mass market

products you can expect press publicity; even with more specialized products

there is a risk of it. Expectations are backed by laws. By 2007, most U.S. states

had security breach notiﬁcation laws, obliging ﬁrms to notify attacks to all

individuals whose privacy could have thereby been compromised, and the

European Union had such a law in the pipeline too. Now it’s not inevitable that

a vulnerability report will trigger such a law — if you’re lucky the alarm won’t

be raised because of an exploit, but from one of your customers’ technical

staff noticing a problem and reporting it to stop it becoming an exploit. But,

either way, you need a plan to deal with it. This will have four components:

monitoring, repair, distribution and reassurance.

First, you need to be sure that you learn of vulnerabilities as soon as you

can — and preferably no later than the press (or the bad guys) do. Listening

to customers is important: you need an efﬁcient way for them to report bugs.

It may be an idea to provide some incentive, such as points towards their next

upgrade, lottery tickets or even cash. The idea of vulnerability markets was

ﬁrst suggested by Jean Camp and Catherine Wolfram in 2000 [256]; two ﬁrms,

iDefense and Tipping Point, are now openly buying vulnerabilities, so the

market actually exists. Unfortunately, the prices are not published and they

only trade in bugs in the major platforms; but this shouldn’t stop you setting

up a reward scheme. Then, however you get the bug reports in, you then

need to make someone responsible for monitoring them, and also for reading

relevant mailing lists, such as bugtraq [239].

Second, you need to be able to respond appropriately. In organizations

such as banks with time-critical processing requirements, it’s normal for one

member of each product team to be ‘on call’ with a pager in case something

goes wrong at three in the morning and needs ﬁxing at once. This might be

excessive for a small software company, but you should still know the home

phone numbers of everyone who might be needed urgently; see to it that there’s

more than one person with each critical skill; and have supporting procedures.

For example, emergency bug ﬁxes must be run through the full testing process

25.4 Security Requirements Engineering 837

as soon as possible, and the documentation’s got to be upgraded too. This is

critical for evolutionary security improvement, but too often ignored: where

the bug ﬁx changes the requirements, you need to ﬁx their documentation

too (and perhaps your threat model, and even top level risk management

paperwork).

Third, you need to be able to distribute the patch to your customers rapidly.

So it needs to be planned in advance. The details will vary depending on

your product: if you only have a few dozen customers running your code

on servers at data centers that are staffed 24 x 7, then it may be very easy,

but if it involves patching millions of copies of consumer software then a lot

of care is needed. It may seem simple enough to get your customers to visit

your website once a day and check for upgrades, but this just doesn’t work.

There is a serious tension between the desire to patch quickly to forestall

attacks, and the desire to delay so as to test the patch properly [135]: pioneers

who apply patches quickly end up discovering problems that break their

systems, but laggards are more vulnerable to attack. There are also two quite

different tensions between the vendor and the customer. First, the vendor

would usually like to patch more quickly than the customer at the operational

level, and second, the customer would probably want the threat of eventual

breach disclosure, because without it the vendor would be less likely to issue

patches at all [89].

Considerations like these led Microsoft to ‘patch Tuesday’, the policy of

releasing a whole set of patches on the second Tuesday of every month. A

monthly cycle seems a reasonable compromise between security, dependability

and manageability. Individual customers usually patch automatically, while

ﬁrms know to schedule testing of their enterprise systems for then so they

can patch as quickly as possible thereafter. Most recent malware exploits have

targeted vulnerabilities that were already patched — the bad guys reverse-

engineer the patches to ﬁnd the vulns and then get the machines that were

patched late or not at all. So once a patch cycle is set up, it can become a

treadmill. There are also quite a lot of details you have to get right to set

up such a scheme — coping with trafﬁc volumes, giving customers adequate

legal notiﬁcation of the effects of changes, and securing the mechanism against

abuse. But as time goes on, I expect that more and more ﬁrms will have to

introduce patch cycle management, as software gets more complex (and thus

buggy), and as it spreads into more and more devices.

Finally, you need a plan to deal with the press. The last thing you need is

for dozens of journalists to phone up and be stonewalled by your switchboard

operator as you struggle madly to ﬁx the bug. Have a set of press releases for

incidents of varying severity ready to roll, so that you only have to pick the

right one and ﬁll in the details. The release can then ship as soon as the ﬁrst

(or perhaps the second) journalist calls.

838 Chapter 25 ■Managing the Development of Secure Systems

25.4.1.2 Control Tuning and Corporate Governance

The main process by which organizations like banks develop their bookkeeping

systems and their other internal controls is by tuning them in the light of

experience. A bank with 25,000 staff might be sacking about one staff member

a day for petty theft or embezzlement, and it’s traditionally the internal audit

department that will review the loss reports and recommend system changes

to reduce the incidence of the most common scams. I gave some examples in

section 10.2.3.

It is important for the security engineer to have some knowledge of internal

controls. There is a shortage of books on this subject: audit is largely learned on

the job, but know-how is also transmitted by courses and through accounting

standards documents. There is a survey of internal audit standards by Janet

Colbert and Paul Bowen [312]; the most inﬂuential is the Risk Management

Framework from the Committee of Sponsoring Organizations (COSO), a group of

U.S. accounting and auditing bodies [318]. This is the yardstick by which your

system will be judged if it’s used in the U.S. public sector or by companies

quoted on U.S. equity markets. The standard reference on COSO is a book by

Steven Root [1082], who also tells the story of how U.S. accounting standards

evolved in the years prior to Enron.

The COSO model is targeted not just on internal control but on the reliability

of ﬁnancial reporting and compliance with laws and regulations. Its basic

process is an evolutionary cycle: in a given environment, you assess the risks,

design controls, monitor their performance, and then go round the loop again.

COSO emphasizes soft aspects of corporate culture more than hard system

design issues so may be seen as a guide to managing and documenting

the process by which your system evolves. However, its core consists of

the internal control procedures whereby senior management check that their

control policies are being implemented and achieving their objectives, and

modify them if not.

It is also worthwhile for the security engineer to learn about the more

specialized information systems audit function. The IS auditor should not have

line responsibility for security but should monitor how it’s done, look into

things that are substandard or appear suspicious, and suggest improvements.

Much of the technical material is common with security engineering; if you

have read and understood this book so far, you should be able to get well

over 50% on the Certiﬁed Information Systems Auditor (CISA) exam (details

are at [639]). The Information Systems Audit and Control Association, which

administers CISA, has a reﬁnement of COSO known as the Control Objectives for

Information and related Technology (CobiT) which is more attuned to IT needs,

more international and more accessible than COSO (it can be downloaded

from [638]). It covers much more than engineering requirements, as issues

such as personnel management, change control and project management are

25.4 Security Requirements Engineering 839

also the internal auditor’s staples. (The working security engineer needs some

familiarity with this material too.)

These general standards are necessarily rather vague. They provide the

engineer with a context and a top-level checklist, but rarely with any clear

guidance on speciﬁc measures. For example, CobiT 5.19 states: ‘Regarding

malicious software, such as computer viruses or trojan horses, management

should establish a framework of adequate preventative, detective and correc-

tive control measures’. More concrete standards are often developed to apply

such general principles to speciﬁc application areas. For example, when I was

working in banking security in the 1980’s, I relied on guidelines from the Bank

for International Settlements [113]. Where such standards exist, they are often

the ultimate fulcrum of security evolutionary activity.

Further important drivers will be your auditors’ interpretation of speciﬁc

accounting laws, such as Sarbanes-Oxley for U.S. publicly-listed companies,

Gramm-Leach-Bliley for U.S. ﬁnancial-sector ﬁrms, and HIPAA for U.S. health-

care providers. Sarbanes-Oxley, for example, requires information security

measures to be documented, which ﬁts well enough with CobiT. In Europe

some comparable pressure comes from privacy laws. Recently the spread of

security-breach disclosure laws has created much greater sensitivity about the

protection of personal information about customers (and in particular about

their credit card numbers, dates of birth or any other data that could be used in

ﬁnancial fraud). There are now breach disclosure laws in most U.S. states, and

they’re in the pipeline for Europe. If you have to disclose that your systems

have been hacked and millions of customer credit card numbers compromised,

you can expect to be sued and to see your stock fall by several percent. If it

happens to you more than once, you can expect to lose customers: customer

churn might only be 2% after one notiﬁed breach, but 30% after two and near

100% after three [1356]. The silver lining in this cloud is that, for the ﬁrst time

ever, information security has become a CEO issue; this means that you’ll

occasionally have access to the boss to make your case for investment.

It’s also a good idea to have good channels of communication to your

internal audit department. But it’s not a good idea to rely on them completely

for feedback. Usually the people who know most about how to break the

system are the ordinary staff who actually use it. Ask them.

25.4.1.3 Evolving Environments and the Tragedy of the Commons

I’ve discussed a lot of systems that broke after their environment changed and

appropriate changes to the protection mechanisms were skimped, avoided

or forgotten. Card-and-PIN technology that worked ﬁne with ATMs became

vulnerable to false-terminal attacks when used with retail point-of-sale termi-

nals; smartcards that were perfectly good for managing credit card numbers

and PINs were inadequate to keep out the pay-TV pirates; and even very

840 Chapter 25 ■Managing the Development of Secure Systems

basic mechanisms such as authentication protocols had to be redesigned once

they started to be used by systems where the main threat was internal rather

than external. Military environments evolve particularly rapidly in wartime,

as attack and defence coevolve; R. V. Jones attributes much of the UK’s relative

success in electronic warfare in World War 2 to the fact that the Germans used

a rigid top-down development methodology, which resulted in beautifully

engineered equipment that was always six months too late [670].

Changes in the application aren’t the only problem. An operating system

upgrade may introduce a whole new set of bugs into the underlying platform.

Changes of scale as businesses go online can alter the cost-beneﬁt equation,

as can the fact that many system users may be in foreign jurisdictions with

ineffective computer crime laws (or none at all). Also, attacks that were known

by experts for many years to be possible, but which were ignored because they

didn’t happen in practice, can suddenly start to happen — good examples

being phishing and the distributed denial-of-service attack.

Where you own the system, things are merely difﬁcult. You manage risk by

ensuring that someone in the organization has responsibility for maintaining its

security rating; this may involve an annual review driven by your internal audit

bureaucracy, or be an aspect of change control. Maintaining organizational

memory is hard, thanks to the high turnover of both IT and security staff which

we discussed in section 25.2.3.4 above. Keeping developer teams interested

and up-to-date in security can be a hard problem, and I’ll return to it towards

the end of this chapter.

That’s tough enough, but where many of the really intractable problems

arise is where no-one owns the system at all. The responsibility for established

standards, such as how ATMs check PINs, is diffuse. In that case, the company

which developed most of the standards (IBM) lost its leading industry role and

its successor, Microsoft, is not interested in the ATM market. Cryptographic

equipment is sold by many specialist ﬁrms; although VISA used to certify

equipment, they stopped in about 1990 and Mastercard never got into that

business. The EMV consortium got going later, but for a while there was no

one person or company in charge, and responsibility for standards outside the

world of EMV smartcards is not always clear. So each player — equipment

maker or bank — had a motive to push the boundaries just a little bit further,

in the hope that when eventually something did go wrong, it would happen

to somebody else.

This problem is familiar to economists, who call it the tragedy of the com-

mons [806]. If a hundred peasants are allowed to graze their sheep on the village

common, where the grass is ﬁnite, then whenever another sheep is added its

owner gets almost the full beneﬁt, while the other ninety-nine suffer only a

very small disadvantage from the decline in the quality of the grazing. So they

aren’t motivated to object, but rather to add another sheep of their own and get

25.4 Security Requirements Engineering 841

as much of the declining resource as they can. The result is a dustbowl. In the

world of agriculture, this problem is tackled by community mechanisms, such

as getting the parish council set up a grazing control committee. One of the

challenges facing the world of computer security is to devise the appropriate

mix of technical and organizational mechanisms to achieve the same sort of

result that was already achieved by a tenth-century Saxon village, only on the

much larger and more diffuse scale of the Internet.

25.4.1.4 Organizational Change

Organizational issues are not just a contributory factor in security failure, as

with the loss of organizational memory and the lack of community mechanisms

for monitoring changing threat environments. They can often be a primary

cause.

The early 1990s saw a management fashion for business process re-engineering

which often meant using changes in business computer systems to compel

changes in the way people worked. There have been some well-documented

cases in which poorly designed systems interacted with resentful staff to cause

a disaster.

Perhaps the best known case is that of the London Ambulance Service. They

had a manual system in which incoming emergency calls were written on

forms and sent by conveyer belt to three controllers who allocated vehicles and

passed the form to a radio dispatcher. Industrial relations were poor, and there

was pressure to cut costs: managers got the idea of solving all these problems

by automating. Lots of things went wrong, and as the system was phased

in it became clear that it couldn’t cope with established working practices,

such as crew taking the ‘wrong’ ambulance (staff had favorite vehicles with

long-serving staff getting the better ones). Managers didn’t want to know and

forced the system into use on the 26th October 1992 by reorganizing the room

so that controllers and dispatchers had to use terminals rather than paper.

The result was meltdown. A number of positive feedback loops became

established which caused the system progressively to lose track of vehicles.

Exception messages built up, scrolled off screen and were lost; incidents

were held as allocators searched for vehicles; as the response time stretched,

callbacks from patients increased (the average ring time for emergency callers

went over ten minutes); as congestion increased, the ambulance crews got

frustrated, pressed the wrong buttons on their new data terminals, couldn’t

get a result, tried calling on the voice channel, and increased the congestion; as

more and more crews fell back on the methods they understood, they took the

wrong vehicles even more; many vehicles were sent to an emergency, or none;

and ﬁnally the whole service collapsed. It’s reckoned that perhaps twenty

people died as a direct result of not getting paramedic assistance in time. By

842 Chapter 25 ■Managing the Development of Secure Systems

the afternoon it was the major news item, the government intervened, and on

the following day the system was switched back to semi-manual operation.

This is only one of many such disasters, but it’s particularly valuable to

the engineer as it was extremely well documented by the resulting public

enquiry [1205]. In my own professional experience I’ve seen cases where

similar attempts to force through changes in corporate culture by replacing

computer systems have so undermined morale that honesty became a worry.

(Much of my consulting work has had to do with environments placed under

stress by corporate reorganization or even national political crises.)

In extreme cases, a step change in the environment brought on by a

savage corporate restructuring will be more like a one-off project than an

evolutionary change. There will often be some useful base to fall back on, such

as an understanding of external threats; but the internal threat environment

may become radically different. This is particularly clear in banking. Fifteen

years ago, bank branches were run by avuncular managers and staffed by

respectable middle-aged ladies who expected to spend their entire working

lives there. Nowadays the managers have been replaced by product sales

specialists and the teller staff are youngsters on near-minimum wages who

turn over every year or so. It’s simply not the same business.

25.4.2 Managing Project Requirements

This brings us to the much harder problem of how to do security require-

ments engineering for a one-off project. A common example in the 1990s was

building an e-commerce application from scratch, whether for a start-up or for

an established business desperate to create new online distribution channels.

A common example in the next ten years might be an established applica-

tion going online as critical components acquire the ability to communicate:

examples I’ve discussed in Part II include postage meters, burglar alarms and

door locks. There will be many more.

Building things from scratch is an accident-prone business; many large

development projects have crashed and burned. The problems appear to be

very much the same whether the disaster is a matter of safety, of security or

of the software simply never working at all; so security people can learn a

lot from the general software engineering literature, and indeed the broader

engineering literature. For example, according to Herb Simon’s classic model

of engineering design, you start off from a goal, desiderata, a utility function

and budget constraints; then work through a design tree of decisions until you

ﬁnd a design that’s ‘good enough’; then iterate the search until you ﬁnd the

best design or run out of time [1153].

At least as important as guidance on ‘how to do it’ are warnings about

how not to. The classic study of large software project disasters was written

by Bill Curtis, Herb Krasner, and Neil Iscoe [339]: they found that failure to

25.4 Security Requirements Engineering 843

understand the requirements was mostly to blame: a thin spread of application

domain knowledge typically led to ﬂuctuating and conﬂicting requirements

which in turn caused a breakdown in communication. They suggested that

the solution was to ﬁnd an ‘exceptional designer’ with a deep understanding

of the problem who would assume overall responsibility.

The millennium bug gives another useful data point, which many writ-

ers on software engineering still have to digest. If one accepts that many

large commercial and government systems needed extensive repair work, and

the conventional wisdom that a signiﬁcant proportion of large development

projects are late or never delivered at all, then the prediction of widespread

chaos at the end of 1999 was inescapable. It didn’t happen. Certainly, the risks

to the systems used by small and medium sized ﬁrms were overstated [50]; nev-

ertheless, the systems of some large ﬁrms whose operations are critical to the

economy, such as banks and utilities, did need substantial ﬁxing. But despite

the conventional wisdom, there have been no reports of signiﬁcant organi-

zations going belly-up. This appears to support Curtis, Krasner, and Iscoe’s

thesis. The requirement for Y2K bug ﬁxes was known completely: ‘I want this

system to keep on working, just as it is now, through into 2000 and beyond’.

So the requirements engineer needs to acquire a deep knowledge of the

application as well as of the people who might attack it and the kind of

tools they might use. If domain experts are available, well and good. When

interviewing them try to distinguish things that are done for a purpose from

those which are just ‘how things are done around here’. Probe constantly for the

reasons why things are done, and be sensitive to after-the-fact rationalizations.

Focus particularly on the things that are going to change. For example,

if dealing with customer complaints depends on whether the customer is

presentable or not, and your job is to take this business online, then ask the

experts what alternative controls might work in a world where it’s much

harder to tell a customer’s age, sex and social class. (This should probably have

been done round about the time of the civil rights movement in the 1960’s, but

better late than never.)

When tackling a new application, dig into its history. I’ve tried to do that

throughout this book, and bring out the way in which problems repeat. To

ﬁnd out what electronic banking will be like in the twenty-ﬁrst century, it’s a

good idea to know what it was like in the nineteenth; human nature doesn’t

change much. Historical parallels will also make it much easier for you to sell

your proposal to your client’s board of directors.

An inﬂuential recent publication is a book on threat modelling by Frank

Swiderski and Window Snyder [1236]. This describes the methodology

adopted by Microsoft following its big security push. The basic idea is that

you list the assets you’re trying to protect (ability to do transactions, access to

classiﬁed data, whatever) and also the assets available to an attacker (perhaps

the ability to subscribe to your system, or to manipulate inputs to the smartcard

844 Chapter 25 ■Managing the Development of Secure Systems

you supply him, or to get a job at your call center). You then trace through the

system, from one module to another. You try to ﬁgure out what the trust levels

are and where the attack paths might be; where the barriers are; and what

techniques, such as spooﬁng, tampering, repudiation, information disclosure,

service denial and elevation of privilege, might be used to overcome particular

barriers. The threat model can be used for various purposes at different points

in the security development lifecycle, from architecture reviews through tar-

geting code reviews and penetration tests. What you’re likely to ﬁnd is that

in order to make the complexity manageable, you have to impose a security

policy — as an abstraction, or even just a rule of thumb that enables you to

focus on the exceptions. Hopefully the security policies discussed in Part II of

this book will give you some guidance and inspiration.

You will likely ﬁnd that a security requirements speciﬁcation for a new

project requires iteration, so it’s more likely to be spiral model than waterfall

model. In the ﬁrst pass, you’ll describe the new application and how it differs

from any existing applications for which loss histories are available, set out

a model of the risks as you perceive them, and draft a security policy (I’ll

have more to say on risk analysis and management in the next section). In the

second pass, you might get comments from your client’s middle management

and internal auditors, while meantime you scour the literature for useful

checklist items and ideas you can recycle. The outcome of this will be a

revised, more quantitative threat model; a security policy; and a security

target which sketches how the policy will be implemented in real life. It will

also set out how a system can be evaluated against these criteria. In the third

pass, the documentation will circulate to a wider group of people including

your client’s senior management, external auditors, insurers and perhaps an

external evaluator. The Microsoft model does indeed support iteration, and its

authors advise that it be used even at the level of features when a product is

undergoing evolution.

25.4.3 Parallelizing the Process

Often there isn’t an expert to hand, as when something is being done for

the ﬁrst time, or when we’re building a competitor to a proprietary system

whose owners won’t share their loss history with us. An interesting question

here is how to brainstorm a speciﬁcation by just trying to think of all the

things that could go wrong. The common industry practice is to hire a single

consulting ﬁrm to draw up a security target; but the experience described

in 12.2.3 suggested that using several experts in parallel would be better.

People with backgrounds in crypto, access control, internal audit and so on

will see a problem from different angles. There is also an interesting analogy

with the world of software testing where it is more cost efﬁcient to test in

parallel rather than series: each tester has a different focus in the testing space

25.4 Security Requirements Engineering 845

and will ﬁnd some subset of ﬂaws faster than the others. (I’ll look at a more

quantitative model of this in the next chapter.)

This all motivated me to carry out an experiment in 1999 to see if a high-

quality requirements speciﬁcation could be assembled quickly by getting a

lot of different people to contribute drafts. The idea was that most of the

possible attacks would be considered in at least one of them. So in one of our

University exam questions, I asked what would be a suitable security policy

for a company planning to bid for the licence for a public lottery.

The results are described in [49]. The model answer was that attackers,

possibly in cahoots with insiders, would try to place bets once the result of

the draw is known, whether by altering bet records or forging tickets; or place

bets without paying for them; or operate bogus vending stations which would

pay small claims but disappear if a client won a big prize. The security policy

that follows logically from this is that bets should be registered online with a

server which is secured prior to the draw, both against tampering and against

the extraction of sufﬁcient information to forge a winning ticket; that there

should be credit limits for genuine vendors; and that there should be ways of

identifying bogus vendors.

Valuable and original contributions from the students came at a number

of levels, including policy goal statements, discussions of particular attacks,

and arguments about the merits of particular protection mechanisms. At

the policy level, there were a number of shrewd observations on the need

to maintain public conﬁdence and the threat from senior managers in the

operating company. At the level of technical detail, one student discussed

threats from refund mechanisms, while another looked at attacks on secure

time mechanisms and observed that the use of the radio time signal in lottery

terminals would be vulnerable to jamming (this turned out to be a real

vulnerability in one existing lottery).

The students also came up with quite a number of routine checklist items of

the kind that designers often overlook — such as ‘tickets must be associated

with a particular draw’. This might seem obvious, but a protocol design which

used a purchase date, ticket serial number and server-supplied random chal-

lenge as input to a MAC computation might appear plausible to a superﬁcial

inspection. Experienced designers appreciate the value of such checklists3.

The lesson to be learned from this case study is that requirements engineer-

ing, like software testing, is susceptible to a useful degree of parallelization.

So if your target system is something novel, then instead of paying a single

consultant to think about it for twenty days, consider getting ﬁfteen people

3They did miss one lottery fraud that actually happened — when a couple won about half a

million dollars, the employee to whom they presented the winning ticket claimed it himself and

absconded overseas with the cash. The lottery compounded the failure by contesting the claim in

court, and losing [765]. One might have thought their auditors and lawyers could have advised

them better.

846 Chapter 25 ■Managing the Development of Secure Systems

with diverse backgrounds to think about it for a day each. Have brainstorming

sessions with a variety of staff, from your client company, its suppliers, and

industry bodies.

But beware — people will naturally think of the problems that might make

their own lives difﬁcult, and will care less about things that inconvenience

others. We learned this the hard way at our university where a number of

administrative systems were overseen by project boards made up largely

of staff from administrative departments. This was simply because the pro-

fessors were too busy doing research and teaching to bother. We ended up

with systems that are convenient for a small number of administrators, and

inconvenient for a much larger number of professors. When choosing the

membership of your brainstorming sessions, focus groups and project boards,

it’s worth making some effort to match their membership to the costs and risks

to which the ﬁrm is exposed. If a university has ﬁve times as many professors

as clerks, you should have this proportion involved in design; and if you’re

designing a bank system that will be the target of attack, then don’t let the

marketing folks drive the design. Make sure you have plenty internal audit

folks, customer helpline managers and other people whose lives will be made

a misery if there’s suddenly a lot more fraud.

25.5 Risk Management

That brings us to our next topic. Whether our threat model and security policy

evolve or are developed in a one-off project, at their heart lie business decisions

about priorities: how much to spend on protection against what. This is risk

management, and it must be done within a broader framework of managing

non-IT risks.

Many ﬁrms sell methodologies for this. Some come in the form of do-it-

yourself PC software, while others are part of a package of consultancy services.

Which one you use may be determined by your client’s policies; for example,

if you’re selling anything to the UK government you’re likely to have to use

a system called CRAMM. The basic purpose of such systems is to prioritise

security expenditure, while at the same time providing a ﬁnancial case for it

to senior management.

The most common technique is to calculate the annual loss expectancy (ALE)

for each possible loss scenario. This is the expected loss multiplied by the

number of incidents expected in an average year. A typical ALE analysis for a

bank’s computer systems might consist of several hundred entries, including

items such as we see in Figure 25.5.

Note that accurate ﬁgures are likely to be available for common losses (such

as ‘teller takes cash’), while for the uncommon, high-risk losses such as a large

funds transfer fraud, the incidence is largely guesswork.

25.5 Risk Management 847

Loss type Amount Incidence ALE

SWIFT fraud $50,000,000 .005 $250,000

ATM fraud (large) $250,000 .2 $100,000

ATM fraud (small) $20,000 .5 $10,000

Teller takes cash $3,240 200 $648,000

Figure 25.5: Computing annualized loss expectancy (ALE)

ALEs have long been standardized by NIST as the technique to use in

U.S. government procurements [1005], and the audit culture post-Enron is

spreading them everywhere. But in real life, the process of producing such a

table is all too often just iterative guesswork. The consultant lists all the threats

he can think of, attaches notional probabilities, works out the ALEs, adds them

all up, and gets a ludicrous result: perhaps the bank’s ALE exceeds its income.

He then tweaks the total down to whatever will justify the largest security

budget he thinks the board of directors will stand (or which his client, the chief

internal auditor, has told him is politically possible). The loss probabilities are

then massaged to give the right answer. (Great invention, the spreadsheet.)

I’m sorry if this sounds a bit cynical; but it’s what seems to happen more often

than not. The point is, ALEs may be of some value, but they should not be

elevated into a religion.

Insurance can be of some help in managing large but unlikely risks. But

the insurance business is not completely scientiﬁc either. For years the annual

premium for bankers’ blanket bond insurance, which covered both computer

crime and employee disloyalty, was 0.5% of the sum insured. This represented

pure proﬁt for Lloyds, which wrote the policies; then there was a large

claim, and the premium doubled to 1% per annum. Such policies may have a

deductible of between $50,000,000 and $10,000,000 per incident, and so they

only remove a small number of very large risks from the equation. As for

nonbanks, business insurance used to cover computer risks up until about

1998, when underwriters started excluding it because of the worries about

the millennium bug. When it became available again in 2001, the premiums

were much higher than before. They have since come down, but insurance

is historically a cyclical industry: companies compete with low premiums

whenever rising stock-market values boost the value of their investment

portfolios, and jack up prices when markets fall. Around 2000, the end of the

dotcom boom created a downswing that coincided with the millennium bug

scare. Even now that markets are returning to normal, some kinds of cover are

still limited because of correlated risks. Insurers are happy to cover events of

known probability and local effect, but where the risk are unknown and the

effect could be global (for example, a worm that took down the Internet for

several days) markets tend to fail [200].

848 Chapter 25 ■Managing the Development of Secure Systems

Anyway, a very important reason for large companies to take out computer

crime cover — and do many other things — is due diligence. The risks that are

being tackled may seem on the surface to be operational risks but are actually

legal, regulatory and PR risks. Often they are managed by ‘following the

herd’ — being just another one of the millions of gnu on the African veld, to

reuse our metaphor for Internet security. This is one reason why information

security is such a fashion-driven business. During the mid 1980’s, hackers

were what everyone talked about (even if their numbers were tiny), and ﬁrms

selling dial-back modems did a roaring business. From the late 80’s, viruses

took over the corporate imagination, and antivirus software made some people

rich. In the mid-1990s, the ﬁrewall became the star product. The late 1990s

saw a frenzy over PKI. These are the products that CFOs see on TV and in the

ﬁnancial press. Amidst all this hoopla, the security professional must retain a

healthy scepticism and strive to understand what the real threats are.

25.6 Managing the Team

It’s now time to pull all the threads together and discuss how you manage a

team of developers who’re trying to produce — or enhance — a product that

does useful work, while at the same time not introduce vulnerabilities that

turn out to be show-stoppers. For this, you need to build a team with the right

culture, the right mix of skills, and above all the right incentives.

One of the hardest issues to get right is the balance between having everyone

on the team responsible for securing their own code, and having a security

guru on whom everyone relies.

There is a growing consensus that, in order to get high-quality software, you

have to make programmers test their own code and ﬁx their own bugs. An

important case history is how Microsoft beat IBM in the PC operating systems

market. During the late 1980s, Microsoft and IBM collaborated on OS/2, but

the partnership broke up in 1990 after which Microsoft developed Windows

into the dominant position it has today. An important reason was Microsoft’s

impatience at IBM’s development process, which was slow and produced

bloated code. IBM followed the waterfall model, being careful to specify

modules properly before they were written; it also divided up its developers

into analysts, programmers and testers. This created a moral hazard, especially

when teams were working under pressure: programmers would write a lot

of shoddy code and ‘throw it over the wall’ for the testers to ﬁx up. As a

result, the code base was often so broken that it wouldn’t run, so it wasn’t

possible to use regression tests to start pulling the bugs out. Microsoft took

the view that they did not have programmers or testers, only developers: each

programmer was responsible for ﬁxing his own software, and a lot of attention

25.6 Managing the Team 849

was paid to ensuring that the software would build every night for testing.

One of the consequences was that people who wrote buggy software ended

up spending most of their time hunting and ﬁxing bugs in their code, so more

of the code base ended up being written by the more careful programmers.

Another consequence was Microsoft won the battle to rule the world of 32-bit

operating systems; their better development methodology let them take a $100

bn market from IBM. The story is told by Steve Maguire in [829].

When engineering systems to be secure — as opposed to merely on time

and on budget — you certainly need to educate all your developers to the

point that they understand the basics, such as the need to sanitise input to

prevent overﬂows, and the need to lock variables to prevent race conditions.

But there is a strong case for some extra specialised knowledge and input,

especially at the testing phase. You can lecture programmers about stack

overﬂows until you’re blue in the face, and they’ll dutifully stare at their

code until they’ve convinced themselves that it doesn’t have any. It’s only

when someone knowledgeable runs a fuzzing tool on it, and it breaks, that the

message really gets through. So how do you square the need for specialists,

in order to acquire and maintain know-how and tools, with the need for

developers to test their own code, in order to ensure that most of your code is

written by the most careful coders?

A second, and equally hard, problem is how you maintain the security

of a system in the face of incremental development. You might be lucky

to start off with a product that’s got a clean architecture tied to a well-

understood protection proﬁle, in which case the problem may be maintaining

its coherence as repeated bug ﬁxes and feature enhancements add complexity.

The case history of cryptographic processors which I described in the chapter

on API security shows how you can suddenly pass a point at which feature

interactions fatally undermine your security architecture. An even worse (and

more likely) case is where you start off with a product that’s already messy, and

you simultaneously have to ﬁx security problems and provide new features.

The former require the product to be made simpler, while the latter are

simultaneously making it more complex.

A large part of the answer lies in how you recruit, train and manage your

team of developers, and create a culture in which they get progressively better

at writing secure code. Some useful insights come from the Capability Maturity

Model developed by the Software Engineering Institute at Carnegie-Mellon

University [873]. Although this is aimed at dependability and at delivering

code on time rather than speciﬁcally at security, their research shows that

capability is something that develops in groups; it’s not just a purely individual

thing. This is especially important if your job is to write (say) the software for

a new mobile phone and you’ve got 20 people only one of whom has any real

security background.

850 Chapter 25 ■Managing the Development of Secure Systems

The trick lies in managing the amount of specialisation in the team, and

the way in which the specialists (such as the security architect and the testing

guru) interact with the other developers. Let’s think ﬁrst of all the stuff you

need to keep track of to manage the development of secure code.

First, there are some things that everybody should know. Everyone must

understand that security software and software security aren’t the same

thing; and that just about any application can have vulnerabilities. Every

developer has to know about the bugs that most commonly lead to vulner-

abilities — stack overﬂows, integer overﬂows, race conditions and off-by-one

errors. They should even understand the more exotic stuff like double frees.

Personally I’d ask questions about these topics when recruiting, to ﬁlter out

the clueless. If you’re stuck with an existing team then you just have to train

them — get them to read Gary McGraw’s book ‘Software Security — Building

Security In’ [858], and Michael Howard and David LeBlanc’s ‘Writing Secure

Code’ [627]. (Getting them to read this book too is unlikely to do them any

harm, though I say it myself.) Everyone on your team should also know about

the speciﬁc problems relevant to your application. if you’re developing web

applications, they have to know about cross-site attacks; if you’re doing an

accounting system, they need to know about COSO and internal controls.

Second, you need to think hard about the tool support your team will

need. If you want to prevent most stack overﬂows being written, you’ll need

static analysis tools — and these had better be tools that you can maintain

and extend yourself. Bugs tend to be correlated: when you ﬁnd one, you’d

better look for similar ones elsewhere in your code base. Indeed, one of the

big improvements Microsoft’s made in its development process since Bill’s

security blitz is that when a bug is found, they update their tools to ﬁnd all

similar ones. You’d better be able to do this too. You also need fuzzers so you

can check code modules for vulnerability to overﬂows of various kinds; these

are often not obvious to visual inspection, and now that the bad guys have

automated means of ﬁnding them, you need the same.

Third, you need to think about the libraries you’ll use. Professional devel-

opment teams avoid a large number of the problems described in this book

by using libraries. You avoid crypto problems — timing attacks, weak keys,

zero keys — by using good libraries, which you’ll probably buy in. You avoid

many buffer overﬂows by using your own standard libraries for I/O and for

those functions that your chosen programming language does badly. Your

libraries should enforce a type system, so that normal code just can’t process

wicked input.

Fourth, your tools and libraries have to support your architecture. The really

critical thing here is that you need to be able to evolve APIs safely. A system’s

architecture is deﬁned more than anything else by its interfaces, and it decays

by a thousand small cuts: by a programmer needing a ﬁle handling routine

25.6 Managing the Team 851

that uses two more parameters than the existing one, and who therefore

writes a new routine — which may be dangerous in itself, or may just add

to complexity and thus contribute indirectly to an eventual failure. You need to

use, and build on, whatever structures your programming language provides

so that you can spot API violations using types.

This is already more than most developers could cope with individually. So

how do you manage the inevitable specialisation? One approach is inspired by

Fred Brooks’ famous book, ‘The Mythical Man-Month’, in which he describes

the lessons learned from developing the world’s ﬁrst large software product,

the operating system for the IBM S/360 mainframe [231]. He describes the

‘chief programmer team’, a concept evolved by his colleague Harlan Mills,

in which a chief programmer — a highly productive coder — is supported

by a number of other staff including a toolsmith, a tester and a language

lawyer. Modern development teams don’t quite ﬁt this vision, as there will

be a number of developers, but a variant of it makes sense if you’re trying to

develop secure code.

Assume that there will be an architect — perhaps the lead developer, per-

haps a specialist — who will act, as Brooks puts it, as the agent, the approver

and the advocate for the user. Assume that you’ll also have a toolsmith, a tester

and a language lawyer. One of these should be your security guru. If the tough

issue is evolving a security policy, or even just tidying up a messy product and

giving it the logical coherence needed for customers to understand it and use

it safely, then the security guy should probably be the architect. If the tough

issue is pulling out lots of implementation errors that have crept into a legacy

product over years — as with Microsoft’s security jihad on buffer overﬂows in

Windows XP — then it should probably be the tester, at least in the beginning.

If the task then changes to one of evolving the static analysis tools so that these

vulnerabilities don’t creep back, the security mantle will naturally pass to the

toolsmith. And if the APIs are everything, and there is constant pressure to add

extra features that will break them — as in the crypto processor case history I

discussed in the chapter on API security — then the language lawyer will have

to pick up the burden of ensuring that the APIs retain whatever type-safety

and other properties are required to prevent attacks.

So far so good. However, it’s not enough to have the skills: you need to get

people to work together. There are many ways to do this. One possibility is

the ‘bug lunch’ where developers are encouraged to discuss the latest subtle

errors that they managed to ﬁnd and remove. Whatever format you use, the

critical factor is to create a culture in which people are open about stuff that

broke and got ﬁxed. It’s bad practice if people who ﬁnd bugs (even bugs that

they coded themselves) just ﬁx them quietly; as bugs are correlated, there are

likely to be more. An example of good practice is in air trafﬁc control, where

it’s expected that controllers making an error should not only ﬁx it but declare

852 Chapter 25 ■Managing the Development of Secure Systems

it at once by open outcry: ‘I have Speedbird 123 at ﬂight level eight zero in the

terminal control area by mistake, am instructing to descend to six zero’. That

way any other controller with potentially conﬂicting trafﬁc can notice, shout

out, and coordinate. Software is less dramatic, but is no different: you need

to get your developers comfortable with sharing their experiences with each

other, including their errors.

Another very useful team-building exercise is the adoption of a standard

style. One of the chronic problems with poorly-managed teams is that the

codebase is in a chaotic mixture of styles, with everybody doing his own thing.

The result is that when a programmer checks out some code to work on it, he

may well spend half an hour formatting it and tweaking it into his own style.

For efﬁciency reasons alone, you want to stop this. However, if your goal is

to write secure code, there’s another reason. When you ﬁnd a bug, you want

to know whether it was a design error or an implementation error. If you

have no idea what the programmer who wrote it was thinking about, this can

be hard. So it’s important to have comments in the code which tell what the

programmer thought he was doing. But teams can easily ﬁght about the ‘right’

quantity and style of comments: in the OS/2 saga, IBM used a lot more than

Microsoft did, so the IBM folks saw the guys from Redmond as a bunch of

hackers, and they responded by disdaining the men from Armonk as a bunch

of bureaucrats. So for goodness’ sake sit everyone down and let them spend

an afternoon hammering out what your house style will be. Provided there’s

enough for understanding bugs, it doesn’t matter hugely what the style is:

but it does matter that there is a consistent style that people accept and that

is ﬁt for purpose. Creating this style is a far better team-building activity than

spending the afternoon paintballing.

25.7 Summary

Managing a project to build, or enhance, a system that has to be secure is a

hard problem. This used to be thought of as ‘security software’ — producing

a product such as an antivirus monitor or encryption program using expert

help. The reality nowadays is often that you’re writing a system that has to do

real work — a web application, for example, or a gadget that listens to network

trafﬁc — and you want to keep out any vulnerabilities that would make it a

target for attack. In other words, you want software security — and that isn’t

the same as security software.

Understanding the requirements is often the hardest part of the whole

process. Like developing the system itself, security requirements engineering

can involve a one-off project; it can be a limited iterative process; or it can

be a matter of continuous evolution. Evolution is becoming the commonest

as systems get larger and longer-lived, whether as packaged software, online

Research Problems 853

services or gadgets. Security requirements are complicated by changes of

scale, of business structures and — above all — of the environment, where the

changes might be in the platform you use, the legal environment you work

in, or the threats you face. Systems are ﬁelded, get popular, and then get

attacked.

Writing secure code has to be seen in this context: the big problem is to

know what you’re trying to do. However, even given a tight speciﬁcation, or

constant feedback from people hacking your product, you’re not home and

dry. There are a number of challenges in hiring the right people, keeping them

up to date with attacks, backing them up with expertise in the right places

which they’ll actually use, reinforcing this with the right tools and language

conventions, and above all creating an environment in which they work to

improve their security capability.

Research Problems

The issues discussed in this chapter are among the hardest and the most

important of any on our ﬁeld. However, they tend to receive little attention

because they lie at the boundaries with software engineering, applied psy-

chology, economics and management. Each of these interfaces appears to be

a potentially productive area of research. Security economics in particular

has made great strides in the last few years, and people are starting to work

on psychology. There is a thriving research community in decision science,

where behavioural economists, psychologists and marketing folks look at why

people really take the decisions they do; this ﬁeld is ripe for mining by security

researchers.

Yet we have all too little work on how these disciplines can be applied to

organisations. For a start, it would be useful if someone were to collect a library

of case histories of security failures caused by unsatisfactory incentives in

organisations, such as [587, 662]. What might follow given a decent empirical

foundation? For example, if organisational theory is where microeconomic

analysis is applied to organisations, with a little psychology thrown in, then

what would be the shape of an organisational theory applied to security? The

late Jack Hirshleifer took the view that we should try to design organizations

in which managers were forced to learn from their mistakes: how could we do

that? How might you set up institutional structures to monitor changes in the

threat environment and feed them through into not just systems development

but into supporting activities such as internal control? Even more basically,

how can you design an organization that is ‘incentive-compatible’ in the sense

that staff behave with an appropriate level of care? And what might the cultural

anthropology of organisations have to say? We saw in the last chapter how the

response of governments to the apparently novel threats posed by Al-Qaida

854 Chapter 25 ■Managing the Development of Secure Systems

was maladaptive in many ways: how can you do corporate governance so that

the ﬁrm doesn’t fall prey to similar problems?

Further Reading

Managing the development of information systems has a large, diffuse and

multidisciplinary literature. There are classics which everyone should read,

such as Fred Brooks’ ‘Mythical Man Month’ [231] and Nancy Leveson’s

‘Safeware’ [786]. Standard textbooks on software engineering such as Roger

Pressman [1041] and Hans van Vliet [1281] cover the basics of project manage-

ment and requirements engineering. The economics of the software life cycle

are discussed by Brooks and by Barry Boehm [199]. The Microsoft approach to

managing software evolution is described by Steve McGuire [829], while their

doctrine on threat modelling is discussed in a book by Frank Swiderski and

Window Snyder [1236].

There are useful parallels with other engineering disciplines. An interesting

book by Henry Petroski discusses the history of bridge building, why bridges

fall down, and how civil engineers learned to learn from the collapses:

what tends to happen is that an established design paradigm is stretched

and stretched until it suddenly fails for some unforeseen reason [1021]. For a

survey of risk management methods and tools, see Richard Baskerville [123] or

Donn Parker [1005]. Computer system failures are another necessary subject of

study; a must-read fortnightly source is the comp.risks newsgroup of which

a selection has been collated and published in print by Peter Neumann [962].

Organizational aspects are discussed at length in the business school litera-

ture, but this can be bewildering to the outsider. If you’re only going to read one

book, make it Lewis Pinault’s ‘Consulting Demons’ — the confessions of a for-

mer insider about how the big consulting ﬁrms rip off their customers [1028].

John Micklethwait and Adrian Wooldridge provide a critical guide to the more

academic literature and draw out a number of highly relevant tensions, such

as the illogicality of management gurus who tell managers to make their orga-

nizations more ﬂexible by sacking people, while at the same time preaching

the virtues of trust [882]. As for the theory of internal control, such as it is, the

best book is by Steven Root, who discusses its design and evolution [1082].

The best introductory book I know to the underlying microeconomics is by

Carl Shapiro and Hal Varian [1159]: the new institutional school has written

much more on the theory of the ﬁrm.

Finally, the business of managing secure software development is getting

more attention (at last). Microsoft’s security VP Mike Nash describes the back-

ground to the big security push and the adoption of the security development

lifecycle at [929]. The standard books I’d get everyone to read are Michael

Further Reading

There is a whole industry devoted to promoting the assurance and evaluation

biz, supported by mountains of your tax dollars. Their enthusiasm can even

have the ﬂavour of religion. Unfortunately, there are nowhere near enough

people writing heresy.

CHAPTER

Conclusions

We are in the middle of a huge change in how security is done.

Ten years ago, the security manager of a large company was usually a retired

soldier or policemen, for whom ‘computer security’ was an unimportant

speciality he left to the computer department, with occasional help from

outside specialists. In ten years’ time, his job will be occupied by a systems

person; she will consider locks and guards to be a relatively unimportant

speciality that she’ll farm out to a facilities management company, with an

occasional review by outside specialists.

Ten years ago, security technology was an archipelago of mutually suspi-

cious islands — the cryptologists, the operating system protection people, the

burglar alarm industry, right through to the chemists who did funny banknote

inks. We all thought the world ended at our shore. Security engineering is

now on the way to becoming an established discipline; the islands are already

being joined up by bridges, and practitioners now realise they have to be

familiar with all of them. The banknote ink man who doesn’t understand digi-

tal watermarks, and the cryptologist who’s only interested in communications

conﬁdentiality mechanisms, are poor value as employees. In ten years’ time,

everyone will need to have a systems perspective and design components that

can be integrated into a larger whole.

Ten years ago, information security was said to be about ‘conﬁdentiality,

integrity and availability’. These priorities are already reversed in many appli-

cations. Security engineering is about ensuring that systems are predictably

dependable in the face of all sorts of malice, from bombers to botnets. And as

attacks shift from the hard technology to the people who operate it, systems

must also be resilient to error, mischance and even coercion. So a realistic

understanding of human stakeholders — both staff and customers — is crit-

ical; human, institutional and economic factors are already as important as

technical ones. The ways in which real systems provide dependability will

become ever more diverse, and tuning the security policy to the application

889

890 Chapter 27 ■Conclusions

will be as essential as avoiding technical exploits. In ten years’ time, protec-

tion goals will not just be closer to the application, they will be more subtle:

examples include privacy, safety, and accountability. Conﬂicts between goals

will be more common; where one principal wants accountability and another

wants deniability, it’s hard to please them both.

Ten years ago, the better information security products were designed for

governments in secret and manufactured in small quantities by cosseted cost-

plus defence contractors. Already, commercial uses dwarf government ones,

and the rough and tumble of the marketplace has taken over. In ten years’

time it’ll be interesting to see whether civil government uses any technologies

different from standard commercial ones, and even the military will make

increasing use of off-the-shelf hardware and software.

Ten years ago, government policy towards information security was devoted

to maintaining the effectiveness of huge communications intelligence networks

built up over the Cold War. Crypto controls turned out to be almost irrelevant

to real policy needs and were largely abandoned in 2000. Surveillance is still

an important policy issue, but privacy, DRM, consumer protection and even

electronic voting are acquiring comparable importance.

The biggest technical challenge is likely to be systems integration and

assurance. Ten years ago, the inhabitants of the different islands in the security

archipelago all had huge conﬁdence in their products. The cryptologists

believed that certain ciphers couldn’t be broken; the smartcard vendors claimed

that probing out crypto keys held in their chips was absolutely physically

impossible; and the security printing people said that holograms couldn’t be

forged without a physics PhD and $20 m worth of equipment. At the system

level, too, there was much misplaced conﬁdence. The banks claimed that

their automatic teller machines could not even conceivably make a mistaken

debit; the multilevel secure operating systems crowd sold their approach as

the solution for all system protection problems; and people assumed that a

security evaluation done by a laboratory licensed by a developed country’s

government would be both honest and competent. These comfortable old

certainties have all evaporated. Instead, security has become part of the larger

dependability problem. We build better and better tools, and these help the

system builders to get a little bit further up the complexity mountain, but in

the end they fall off. A proportion of large complex system projects fail, just

like in the 1970s; but we build much bigger disasters nowadays.

Complexity is the real enemy of security. The distinction between outsiders

and insiders used to simplify the business, but as everything gets connected up

it’s disappearing fast. Protection used to be predicated on a few big ideas and

on propositions that could be stated precisely, while now the subject is much

more diverse and includes a lot of inexact and heuristic knowledge. The system

life-cycle is also changing: in the old days, a closed system was developed in a

ﬁnite project, while now systems evolve and accumulate features without limit.

Conclusions 891

Changes in the nature of work are signiﬁcant: while previously a bank’s chief

internal auditor would remember all the frauds of the previous thirty years and

prevent the data processing department repeating the errors that caused them,

the new corporate culture of transient employment and ‘perpetual revolution’

(as Mao described it) has trashed corporate memory. Economics will continue

to ensure that insecure systems get built — and the liability will be dumped

on others whenever possible. Governments will try to keep up, but they’re

too slow and they can often be bought off for a while. So there will be many

regulatory failures too.

The net effect of all these changes is that the protection of information in

computer systems is no longer a scientiﬁc discipline, but an engineering one.

The security engineer of the twenty-ﬁrst century will be responsible for

systems that evolve constantly and face a changing spectrum of threats. She

will have a large and constantly growing toolbox. A signiﬁcant part of her

job will be keeping up to date technically: understanding the latest attacks,

learning how to use new tools, and keeping up on the legal and policy fronts.

Like any engineer, she’ll need a solid intellectual foundation; she will have to

understand the core disciplines such as cryptology, access control, information

ﬂow, networking and signal detection. She’ll also need to understand the basics

of management: how accounts work, the principles of ﬁnance and the business

processes of her client. But most important of all will be the ability to manage

technology and play an effective part in the process of evolving a system

to meet changing business needs. The ability to communicate with business

people, rather than just with other engineers, will be vital; and experience will

matter hugely. I don’t think anybody with this combination of skills is likely

to be unemployed — or bored — anytime soon.

Finally, the rampant growth of the security-industrial complex since 9/11,

and the blatant fearmongering of many governments, are a scar on the world

and on our profession. We have a duty to help it heal, and we can do that in

many different ways. My own path has been largely research — developing

the disciplines of security engineering and security economics, so that we can

tell not just what works in the lab but what can be made to work in the world.

The dissemination of knowledge is important, too — that’s what this book

is about. Economic growth also helps, and education: it’s the poor and the

uneducated who are most swayed by fearmongering (whether from Western

leaders, or from bin Laden). At least in countries with educated populations,

the voters have started to recognise the excesses of the ‘War on Terror’ and to

deal with them. Just as individuals learn through experience to compensate

for our psychological biases and deal more rationally with risk, so our societies

learn and adapt too. Democracy is the key mechanism for that. So the ﬁnal

way in which security engineers can contribute is by taking part in the policy

debate. The more we can engage the people who lead the discussions on

emerging threats, the faster our societies will adapt to deal with them.

Bibliography

[1] M Abadi, ‘‘Explicit Communications Revisited: Two New Attacks on

Authentication Protocols’’, in IEEE Transactions on Software Engineer-

ing v 23 no 3 (Mar 97) pp 185–186

[2] M Abadi, RM Needham, ‘‘Prudent Engineering Practice for Crypto-

graphic Protocols’’, IEEE Transactions on Software Engineering v22no

1 (Jan 96) pp 6–15; also as DEC SRC Research Report no 125 (June 1

1994) at ftp://gatekeeper.pa.dec.com/pub/DEC/SRC/research-

reports/SRC-125.pdf

[3] A Abbasi, HC Chen, ‘‘Visualizing Authorship for Identiﬁcation’’, in ISI

2006, LNCS 3975 pp 60–71

[4] H Abelson, RJ Anderson, SM Bellovin, J Benaloh, M Blaze, W Difﬁe,

J Gilmore, PG Neumann, RL Rivest, JI Schiller, B Schneier, ‘‘The Risks

of Key Recovery, Key Escrow, and Trusted Third-Party Encryption’’,

in World Wide Web Journal v 2 no 3 (Summer 1997) pp 241–257

[5] A Abulaﬁa, S Brown, S Abramovich-Bar, ‘‘A Fraudulent Case Involving

Novel Ink Eradication Methods’’, in Journal of Forensic Sciences v41

(1996) pp 300–302

[6] DG Abraham, GM Dolan, GP Double, JV Stevens, ‘‘Transaction Security

System’’, in IBM Systems Journal v 30 no 2 (1991) pp 206–229

[7] N Achs, ‘‘VISA confronts the con men’’, Cards International

(20 Oct 1992) pp 8–9

893

894 Bibliography

[8] A Acquisti, A Friedman, R Telang, ‘‘Is There a Cost to Privacy

Breaches?’’, Fifth Workshop on the Economics of Information Security

(2006)

[9] EN Adams, ‘‘Optimising preventive maintenance of software prod-

ucts’’, lBM Journal of Research and Development, v 28 no 1 (1984)

pp 2–14

[10] J Adams, ‘Risk’, University College London Press (1995), ISBN

1-85728-067-9

[11] J Adams, ‘‘Cars, Cholera and Cows: the management of risk and

uncertainty’’, in Policy Analysis no 335, Cato Institute, Washington,

1999; at http://www.cato.org/pubs/pas/pa-335es.html

[12] B Adida, M Bond, J Clulow, A Lin, RJ Anderson, RL Rivest, ‘‘A Note

on EMV Secure Messaging in the IBM 4758 CCA’’, at www.ross-

anderson.com

[13] Y Adini, Y Moses, S Ullman, ‘‘Face recognition: The Problem of Com-

pensating for Changes in Illumination Direction’’, in IEEE Transactions

on Pattern Analysis and Machine Intelligence v 19 no 7 (July 97)

pp 721–732

[14] A Adler, ‘‘Sample images can be independently restored from face

recognition templates’’, in Proc. Can. Conf. Elec. Comp. Eng. (2003)

pp 1163–1166; at http://www.sce.carleton.ca/faculty/adler/

publications/publications.html

[15] A Adler, ‘‘Vulnerabilities in biometric encryption systems’’, in NATO

RTA Workshop: Enhancing Information Systems Security–Biometrics

(IST-044-RWS-007), at http://www. sce.carleton.ca/faculty/

adler/publications/publications.html

[16] The AES Lounge, http://www.iaik.tu-graz.ac.at/research/

krypto/AES/

[17] C Ajluni, ‘‘Two New Imaging Techniques Promise To Improve IC

Defect Identiﬁcation’’, in Electronic Design v 43 no 14 (10 July 1995)

pp 37–38

[18] Y Akdeniz, ‘‘Regulation of Child Pornography on the Internet’’ (Dec

1999), at http://www.cyber-rights.org/reports/child.htm

[19] G Akerlof, ‘‘The Market for ‘Lemons: Quality Uncertainty and the

Market Mechanism’’, in The Quarterly Journal of Economics v84no3

(1970) pp 488–500

Bibliography 895

[20] R Albert, HW Jeong, AL Barab´

asi, ‘‘Error and attack tolerance of

complex networks’’, in Nature v 406 no 1 (2000) pp 387–482

[21] J Alfke, ‘‘Facebook and Decentralized Identiﬁers’’, in Thought Palace

Dec 2 2007; at http://mooseyard.com/Jens/2007/12/facebook-and-

decentralized-identifiers

[22] Alliance to Outfox Phone Fraud, hosted by Verizon at http://www

.bell-atl.com/security/fraud/ but now withdrawn; as of August

2007, Verizon had its own scam alert page at http://www22

.verizon.com/pages/securityalerts/

[23] M Allman, V Paxson, ‘‘Issues and Etiquette Concerning Use of Shared

Measurement Data’’, in Internet Measurement Conference (IMC 2007), at

http://www.imconf.net/imc-2007/papers/imc80.pdf

[24] F Almgren, G Andersson, T Granlund, L Ivansson, S Ulfberg, ‘‘How

We Cracked the Code Book Ciphers’’, at http://codebook.org

[25] American Society for Industrial Security, http://www.asisonline.org

[26] American Statistical Association, Privacy, Conﬁdentiality, and Data

Security web site,athttp://www.amstat.org/comm/cmtepc/

[27] E Amoroso, ‘Fundamentals of Computer Security Technology’,Prentice

Hall (1994); ISBN 0-13-10829-3

[28] B Andersen, M Frenz, ‘‘The Impact of Music Downloads and P2P

File-Sharing on the Purchase of Music: A Study for Industry Canada’’,

2007, at http://strategis.ic.gc.ca/epic/site/ippd-dppi.nsf/en/

hip01456e.html

[29] J Anderson, ‘Computer Security Technology Planning Study’, ESD-TR-73-

51, US Air Force Electronic Systems Division (1973) http://csrc

.nist.gov/publications/history/index.html

[30] M Anderson, C North, J Grifﬁn, R Milner, J Yesberg, K Yiu, ‘‘Starlight:

Interactive Link’’, in 12th Annual Computer Security Applications Con-

ference (1996) proceedings published by the IEEE, ISBN 0-8186-

7606-XA, pp 55–63

[31] M Anderson, W Seltzer, Ofﬁcial Statistics and Statistical Conﬁdentiality:

Recent Writings and Essential Documents,athttp://www.uwm.edu/

%7 Emargo/govstat/integrity.htm

[32] RJ Anderson, ‘‘Solving a Class of Stream Ciphers’’, in Cryptologia vXIV

no 3 (July 1990) pp 285–288

896 Bibliography

[33] RJ Anderson, ‘‘Why Cryptosystems Fail’’ in Communications of the

ACM v 37 no 11 (November 1994) pp 32–40; earlier version at

http://www.cl.cam.ac.uk/users/rja14/wcf.html

[34] RJ Anderson, ‘‘Liability and Computer Security: Nine Principles’’, in

Computer Security–ESORICS 94, Springer LNCS v 875 pp 231–245

[35] RJ Anderson, ‘‘Crypto in Europe–Markets, Law and Policy’’, in

Cryptography: Policy and Algorithms, Springer LNCS v 1029 pp 75–89

[36] RJ Anderson, ‘‘Clinical System Security–Interim Guidelines’’, in

British Medical Journal v 312 no 7023 (13th January 1996) pp 109–111;

http://www.cl.cam.ac.uk/ftp/users/rja14/guidelines.txt

[37] RJ Anderson, ‘Security in Clinical Information Systems’, published by

the British Medical Association (1996); ISBN 0-7279-1048-5

[38] RJ Anderson, ‘‘A Security Policy Model for Clinical Information

Systems’’, in Proceedings of the 1996 IEEE Symposium on Security and

Privacy pp 30–43 http://www.cl.cam.ac.uk/users/rja14/policy11/

policy11.html

[39] RJ Anderson, ‘‘An Update on the BMA Security Policy’’, in [43]

pp 233–250; http://www.cl.cam.ac.uk/ftp/users/rja14/bmaupdate

.ps.gz

[40] RJ Anderson, C Manifavas, C Sutherland, ‘‘NetCard — A Practical

Electronic Cash Scheme’’ in Security Protocols (1996), Springer LNCS

v 1189 pp 49–57

[41] RJ Anderson, ‘‘The Eternity Service’’, in Proceedings of Pragocrypt 96

(GC UCMP, ISBN 80-01-01502-5) pp 242–252

[42] RJ Anderson (ed), Proceedings of the First International Workshop on

Information Hiding (1996), Springer LNCS v 1174

[43] RJ Anderson (ed), ‘Personal Medical Information–Security, Engineering

and Ethics’, Springer-Verlag (1997) ISBN 3-540-63244-1

[44] RJ Anderson, ‘‘GSM hack–operator ﬂunks the challenge’’, in comp

.risks v 19.48: http://catless.ncl.ac.uk/Risks/19.48.html

[45] RJ Anderson, ‘‘On the Security of Digital Tachographs’’, in Computer

Security–ESORICS 98, Springer LNCS v 1485 pp 111–125;

http://www.cl.cam.ac.uk/ftp/users/rja14/tacho5.ps.gz

Bibliography 897

[46] RJ Anderson, ‘‘Safety and Privacy in Clinical Information Systems’’,

in ‘Rethinking IT and Health’, J Lenaghan (ed), IPPR (Nov 98) (ISBN

1-86030-077-4) pp 140–160

[47] RJ Anderson, ‘‘The DeCODE Proposal for an Icelandic Health

Database’’; part of this was published in Læknablaðið (The Icelandic

Medical Journal) v 84 no 11 (Nov 98) pp 874–5; full text available from

http://www.cl.cam.ac.uk/users/rja14/#Med

[48] RJ Anderson, ‘‘The Formal Veriﬁcation of a Payment System’’, in

Industrial Strength Formal Methods: A Practitioners Handbook,MG

Hinchey and JP Bowen (editors), Springer Verlag (Sep 1999,

1-85233-640-4) pp 43–52

[49] RJ Anderson, ‘‘How to Cheat at the Lottery (or, Massively Parallel

Requirements Engineering)’’, in 15th Annual Computer Security Appli-

cation Conference (1997); proceedings published by IEEE Computer Soci-

ety, ISBN 0-7695-0346-2, pp xix–xxvii; at http://www.cl.cam.ac.uk/

~rja14/lottery/lottery.html

[50] RJ Anderson, ‘‘The Millennium Bug–Reasons not to Panic’’, at

http://www.ftp.cl.cam.ac.uk/ftp/users/rja14/y2k.html

[51] RJ Anderson, ‘‘Comments on the Security Targets for the Icelandic

Health Database’’, at http://www.cl.cam.ac.uk/ftp/users/rja14/

iceland-admiral.pdf

[52] ‘‘The Correctness of Crypto Transaction Sets’’, in Proceedings of

Protocols 2000, Springer LNCS v 2133 pp 125–141

[53] Ross Anderson, ‘‘Cryptography and Competition Policy–Issues with

‘Trusted Computing’ ’’, Second Workshop on Economics and

Information Security (2003)

[54] Ross Anderson, ‘‘Open and Closed Systems are Equivalent (that is, in

an ideal world)’’, in Perspectives on Free and Open Source Software,MIT

Press 2005, pp 127–142

[55] RJ Anderson, ‘‘Closing the Phishing Hole–Fraud, Risk and

Nonbanks’’, at Nonbanks in the Payments System: Innovation, Com-

petition, and Risk, US Federal Reserve, Santa Fe, May 2–4 2007

[56] RJ Anderson, ‘‘RFID and the Middleman’’, in Proceedings of the Eleventh

International Conference on Financial Cryptography and Data Security,

February 2007

898 Bibliography

[57] RJ Anderson, ‘‘Searching for Evil’’, Tech talk given at Google, Aug

24 2007, at http://video.google.com/videoplay?docid=-

1380463341028815296

[58] RJ Anderson, ‘Security Economics Resource Page’,athttp://www.cl

.cam.ac.uk/~rja14/econsec.html

[59] RJ Anderson, SJ Bezuidenhoudt, ‘‘On the Reliability of Electronic Pay-

ment Systems’’, in IEEE Transactions on Software Engineering v22no5

(May 1996) pp 294–301; http://www.cl.cam.ac.uk/ftp/users/rja14/

prepay-meters.pdf

[60] RJ Anderson, E Biham, LR Knudsen,‘‘Serpent: A Proposal for the

Advanced Encryption Standard’’, submitted to NIST as an AES

candidate; a short version of the paper appeared at the AES conference,

August 1998; both papers available at [61]

[61] RJ Anderson, E Biham, L Knudsen, ‘The Serpent Home Page’,

http://www.cl.cam.ac.uk/~rja14/serpent.html

[62] RJ Anderson, R B¨

ohme, R Clayton, T Moore, ‘Security Economics and

the Internal Market’, ENISA, 2008

[63] RJ Anderson, M Bond, ‘‘API-Level Attacks on Embedded Systems’’, in

IEEE Computer v 34 no 10 (October 2001) pp 67–75

[64] RJ Anderson, M Bond, ‘‘Protocol Analysis, Composability and Com-

putation’’ in Computer Systems: Theory, Technology and Applications,

Springer 2003, pp 7–10

[65] RJ Anderson, M Bond, J Clulow, S Skorobogatov, ‘Cryptographic proces-

sors–a survey’, Cambridge University Computer Laboratory Technical

Report no 641 (July 2005); shortened version in Proc. IEEE v94no2

(Feb 2006) pp 357–369

[66] RJ Anderson, I Brown, R Clayton, T Dowty, D Korff, E Munro,

‘Children’s Databases–Safety and Privacy’, Information Commissioner’s

Ofﬁce, UK, Nov 2006

[67] RJ Anderson, B Crispo, JH Lee, C Manifavas, V Maty´

as, FAP Petitcolas,

‘The Global Internet Trust Register’, MIT Press (1999) (ISBN 0-262-51105-

3) http://www.cl.cam.ac.uk/Research/Security/Trust-Register/

[68] RJ Anderson, MG Kuhn, ‘‘Tamper Resistance–a Cautionary Note’’,

in Proceedings of the Second Usenix Workshop on Electronic Commerce

(Nov 96) pp 1–11; http://www.cl.cam.ac.uk/users/rja14/tamper

.html

Bibliography 899

[69] RJ Anderson, MG Kuhn, ‘‘Low Cost Attacks on Tamper Resistant

Devices’’, in Security Protocols–Proceedings of the 5th International

Workshop (1997) Springer LNCS v 1361 pp 125–136

[70] RJ Anderson, MG Kuhn, ‘‘Soft Tempest–An Opportunity for NATO’’, at

Protecting NATO Information Systems In The 21st Century, Washington

DC, Oct 25–26, 1999

[71] RJ Anderson, JH Lee, ‘‘Jikzi: A New Framework for Secure Publishing’’,

in Security Protocols 99, Springer LNCS v 1976 pp 21–36

[72] RJ Anderson, TW Moore, ‘‘Information Security Economics–and

Beyond’’, in Advances in Cryptology–Crypto 2007, Springer LNCS 4622,

pp 68–91

[73] RJ Anderson, RM Needham, ‘‘Robustness principles for public key

protocols’’, in Advances in Cryptology–Crypto 95 Springer LNCS v 963

pp 236–247; http://www.cl.cam.ac.uk/ftp/users/rja14/robust-

ness.ps.gz

[74] RJ Anderson, RM Needham, ‘‘Programming Satan’s Computer’’ in

‘Computer Science Today’, Springer Lecture Notes in Computer Science

v 1000 (1995) pp 426–441; http://www.cl.cam.ac.uk/ftp/

users/rja14/satan.ps.gz

[75] RJ Anderson, RM Needham, A Shamir, ‘‘The Steganographic File Sys-

tem’’, in Proceedings of the Second International Workshop on Information

Hiding, Springer LNCS v 1525 pp 74–84

[76] RJ Anderson, MR Roe, ‘‘The GCHQ Protocol and Its Problems’’, in

Advances in Cryptology–Eurocrypt 97, Springer LNCS v 1233

pp 134–148; http://www.cl.cam.ac.uk/ftp/users/rja14/euro-

clipper.ps.gz

[77] CM Andrew, V Mitrokhin, ‘The Sword and the Shield: The Mitrokhin

Archive and the Secret History of the KGB’, Basic Books (1999) ISBN

0-46500310-9

[78] M Andrews, JA Whitaker, ‘How to Break Web Software’, Addison-

Wesley 2006

[79] http://www.anonymizer.com

[80] JC Anselmo, ‘‘US Seen More Vulnerable to Electromagnetic Attack’’, in

Aviation Week and Space Technology v 146 no 4 (28/7/97) p 67

900 Bibliography

[81] A Ant´

on, ‘‘Is That Vault Really Protecting Your Privacy?’’, at theprivacy-

place.org Oct 9 2007; at http://theprivacyplace.org/2007/10/09/is-

that-vault-really-protecting-your-privacy/

[82] Anonymity Bibliography, 2007, at http://freehaven.net/anonbib/

[83] APACS, ‘‘Fraud abroad drives up card fraud losses’’, October 3 2007; at

http://www.apacs.org.uk/media centre/press/03.10.07.html; see

also The Register,http://www.theregister.co.uk/2007/10/03/card

fraud trends/

[84] APACS, ‘‘Payment Advice–Protect Your PIN’’, Aug 16 2007; at

http://www.apacs.org.uk/media centre/press/08 16 07.htmland

www.cardwatch.org.uk

[85] T Appleby, ‘‘Chilling debit-card scam uncovered’’, in The Globe and

Mail (10/12/1999) p 1

[86] Arbor Networks Inc., ‘Infrastructure Security Report’, 2007, at

http://www.arbornetworks.com/report

[87] US Army, ‘Electromagnetic Pulse (EMP) and Tempest Protection for

Facilities’, Corps of Engineers Publications Depot, Hyattsville (1990)

[88] A Arora, R Krishnan, A Nandkumar, R Telang, YB Yang, ‘‘Impact of

Vulnerability Disclosure and Patch Availability–An Empirical Analy-

sis’’, Third Workshop on the Economics of Information Security (2004)

[89] A Arora, CM Forman, A Nandkumar, R Telang, ‘‘Competitive and

strategic effects in the timing of patch release’’, in Workshop on the

Economics of Information Security (2006)

[90] SE Asch, ‘Social Psychology’, OUP 1952

[91] D Asonov, R Agrawal, ‘‘Keyboard Acoustic Emanations’’, IBM

Almaden Research Center, 2004

[92] ‘ASPECT–Advanced Security for Personal Communications Technologies’,

at http://www.esat.kuleuven.ac.be/cosic/aspect/index.html

[93] Associated Press, ‘‘Charges dropped against Ex-HP chairwoman–

Three others charged in boardroom spying case receive no jail time’’,

Mar 14 2007, at http://www.msnbc.msn.com/id/17611695/

[94] R Atkinson, ‘‘The single most effective weapon against our deployed

forces’’ and ‘‘The IED problem is getting out of control. We’ve got to

stop the bleeding’’, in the Washington Post, Sep 30 2007; ‘‘There was

atwo-yearlearningcurve... and a lot of people died in those two

years’’, Oct 1 2007; ‘‘You can’t armor your way out of this problem’’,

Bibliography 901

Oct 2 2007; ‘‘If you don’t go after the network, you’re never going

to stop these guys. Never’’, Oct 3 2007; all linked from http://

smallwarsjournal.com/blog/2007/09/print/weapon-of-choice/

[95] D Aubrey-Jones, ‘‘Internet — Virusnet?’’, in Network Security (Feb 97)

pp 15–19

[96] D Aucsmith, ‘‘Tamper-Resistant Software: An Implementation’’, in

[42] pp 317–333

[97] D Aucsmith (editor), Proceedings of the Second International Workshop

on Information Hiding (Portland, Apr 98), Springer LNCS v 1525

[98] B Audone, F Bresciani, ‘‘Signal Processing in Active Shielding and

Direction-Finding Techniques’’, IEEE Transactions on Electromagnetic

Compatibility v 38 no 3 (August 1996) pp 334–340

[99] R Axelrod, The Evolution of Cooperation, Basic Books (1984)

[100] I Ayres, SD Levitt, ‘‘Measuring Positive Externalities from Unobserv-

able Victim Precaution: An Empirical Analysis of Lojack’’, in Quarterly

Journal of Economics v 108 no 1 (Feb 1998), http://www.nber.org/

papers/w5928

[101] ‘‘Barclays winning card fraud war’’, D Austin, in Banking Technology

(April 94) p 5

[102] D Austin, ‘‘Flood warnings’’, in Banking Technology (Jul–Aug 1999)

pp 28–31

[103] ‘‘Computer Combat Rules Frustrate the Pentagon’’, in Aviation

Week and Space Technology v 147 no 11 (15/9/97) pp 67–68

[104] J Bacon, ‘Concurrent Systems’, Addison-Wesley (1997); ISBN 0-201-

17767-6

[105] J Bacon, K Moody, J Bates, R Hayton, CY Ma, A McNeil, O Seidel,

M Spiteri, ‘‘Generic Support for Distributed Applications’’, in IEEE

Computer (March 2000) pp 68–76

[106] L Badger, DF Sterne, DL Sherman, KM Walker, SA Haghighat,

‘‘Practical Domain and Type Enforcement for UNIX,’’ in Proceedings of

the 1995 IEEE Symposium on Security and Privacy pp 66–77

[107] M Baggott, ‘‘The smart way to ﬁght fraud’’, Scottish Banker (Nov 95)

pp 32–33

[108] B Bain, ‘‘Justice says no to private PCs for telework’’, in FCW.com,Sep

13 2007; at http://www.fcw.com/article103746-09-13-07

902 Bibliography

[109] SA Baker, PR Hurst, ‘The Limits of Trust’, Kluwer Law International

(1998) ISBN 9-0411-0639-1

[110] ‘‘Card Fraud: Banking’s Boom Sector’’, in Banking Automation Bulletin

for Europe (Mar 92) pp 1–5

[111] D Balfanz, EW Felten, ‘‘Hand-Held Computers Can Be Better

Smart Cards’’, in Eighth USENIX Security Symposium (1999), ISBN

1-880446-28-6, pp 15–23

[112] J Bamford, ‘The Puzzle Palace: A Report on NSA, America’s Most Secret

Agency’, Houghton, Mifﬂin (1982–3rd Printing–revised edition due

out shortly), ISBN 0-395-31286-8

[113] Bank for International Settlements, ‘Security and Reliability in Electronic

Systems for Payments’, British Computer Society (1982–no ISBN)

[114] Bank for International Settlements, http://www.bis.org/

[115] E Bangeman, ‘‘The insanity of France’s anti-ﬁle-sharing plan: L’ ´

Etat,

c’est IFPI’’, in Ars Technica Nov 25 2007; at http://arstechnica.com/

news.ars/post/20071125-the-insanity-and-genius-of-frances-

anti-file-sharingplan.html

[116] M Barbaro, T Zeller, ‘‘A Face Is Exposed for AOL Searcher No. 4417749’’,

in New York Times Aug 9 2006, at http://query.nytimes.com/gst/

fullpage.html?res=9E0CE3DD1F3FF93AA3575BC0A9609C8B63

[117] E Barkan, E Biham, N Keller, ‘‘Instant Ciphertext-Only Cryptanalysis

of GSM Encrypted Communication’’ Technion Technical Report

CS-2006-07, at http://www.cs.technion.ac.il/~biham/

[118] RL Barnard, ‘Intrusion Detection Systems’, Butterworths (1988) ISBN

0-409-90030-3

[119] A Barnett, ‘‘Britain’s UFO secrets revealed’’, in The Observer (4/6/2000)

at http://www.observer.co.uk/uk news/story/0,6903,328010,00

.html

[120] S Baron-Cohen, The Essential Difference: Men, Women, and the Extreme

Male Brain, Penguin, 2003 ISBN 0141011017

[121] J Barr, ‘‘The Gates of Hades’’, in Linux World April 2000; at http://www

.linuxworld.com/linuxworld/lw-2000-04/lw-04-vcontrol 3.html

[122] B Barrow, B Quinn, ‘‘Millions in danger from chip and pin fraudsters’’

in Daily Mail June 5th 2006

Bibliography 903

[123] R Baskerville, ‘‘Information Systems Security Design Methods:

Implications for Information Systems Development’’, in ACM Comput-

ing Surveys v 265 (1993) pp 375–414

[124] PJ Bass, ‘‘Telephone Cards and Technology Development as Experi-

enced by GPT Telephone Systems’’, in GEC Review v 10 no 1 (95) pp

14–19

[125] J Battelle, ‘The Search: How Google and Its Rivals Rewrote the Rules of

Business and Transformed Our Culture’, Portfolio, 2005

[126] W Bax, V Dekker, ‘‘Met zijn allen meekijken in de medische kaarten-

bak’’, in Trouw Dec 11 2007, at http://www.trouw.nl/deverdieping/

overigeartikelen/article867144.ece/

[127] S Baxter, ‘‘US hits panic button as air force ‘loses’ nuclear missiles’’, in

Sunday Times Oct 21 2007; at http://www.timesonline.co.uk/tol/

news/world/us and americas/article2702800.ece

[128] ‘‘Great Microprocessors of the Past and Present’’, at http://www.cs

.uregina.ca/~bayko/cpu.html

[129] BBC News Online, ‘‘Tax records ‘for sale’ scandal’’, Jan 16 2003, at

http://news.bbc.co.uk/1/hi/business/2662491.stm

[130] BBC News Online, ‘‘ ‘Relief’ over ﬁngerprint verdict’’, Feb 7 2006, at

http://news.bbc.co.uk/1/hi/scotland/4689218.stm

[131] BBC News Online, ‘‘UN warns on password ‘explosion’ ’’, Dec 4 2006, at

http://news.bbc.co.uk/1/hi/technology/6199372.stm

[132] BBC News Online, ‘‘Schools get rules on biometrics’’, July 23 2007, at

http://news.bbc.co.uk/1/hi/education/6912232.stm

[133] BBC News Online, ‘‘Mobile phone technology turns 20’’, Sep 7 2007, at

http://news.bbc.co.uk/1/hi/technology/6983869.stm

[134] BBC News Online, ‘‘PC stripper helps spam to spread’’, Oct 30 2007, at

http://news.bbc.co.uk/1/hi/technology/7067962.stm

[135] S Beattie, S Arnold, C Cowan, P Wagle, C Wright, ‘‘Timing the

Application of Security Patches for Optimal Uptime’’, in LISA XVI

(2002) pp 101–110

[136] F Beck, ‘Integrated Circuit Failure Analysis–A Guide to Preparation

Techniques’, Wiley (1998), ISBN 0-471-97401-3

[137] J Beck, ‘‘Sources of Error in Forensic Handwriting Examination’’, in

Journal of Forensic Sciences v 40 (1995) pp 78–87

904 Bibliography

[138] GS Becker, ‘‘Crime and Punishment: An Economic Approach’’, in

Journal of Political Economy v 76 no 2 (March/April 1968) pp 169–217

[139] L Beckwith, M Burnett, V Grigoreanu, S Weidenbeck, ‘‘Gender HCI:

What About the Software?’’, in Computer (Nov 2006) pp 97–101

[140] L Beckwith, C Kissinger, M Burnett, S Weidenbeck, J Lowrance,

A Blackwell, C Cook, ‘‘Tinkering and Gender in End-User Pro-

grammers’ Debugging’’, in CHI ’06, Montreal, April 2006; at

http://eusesconsor-tium.org/gender/

[141] S Begley, ‘‘Fingerprint Matches Come Under More Fire As Potentially

Fallible’’, Wall Street Journal Oct 7 2005 p B1; at http://online.wsj

.com/article print/SB112864132376462238.html

[142] HA Beker, C Amery, ‘‘Cryptography Policy’’, at http://

www.baltimore.com/library/whitepapers/mn cryptography.html

[143] HJ Beker, JMK Friend, PW Halliden, ‘‘Simplifying key management in

electronic fund transfer point of sale systems’’, in Electronics Letters v

19 (1983) pp 442–443

[144] H Beker, F Piper, ‘Cipher Systems’, Northwood (1982)

[145] H Beker, M Walker, ‘‘Key management for secure electronic funds

transfer in a retail environment’’, in Advances in Cryptology–Crypto 84

Springer LNCS v 196 pp 401–410

[146] DE Bell, L LaPadula, ‘Secure Computer Systems’, ESD-TR-73-278, Mitre

Corporation; v I and II: November 1973, v III: Apr 1974

[147] M Bellare, J Kilian, P Rogaway, ‘‘The Security of Cipher Block

Chaining’’ in Advances in Cryptology–Crypto 94 Springer LNCS v 839

pp 341–358

[148] M Bellare, P Rogaway, ‘‘Optimal Asymmetric Encryption’’, in

Advances in Cryptology–Eurocrypt 94, Springer LNCS v 950

pp 103–113; see also RFC 2437, http://sunsite.auc.dk/RFC/rfc/

rfc2437.html

[149] SM Bellovin, ‘‘Packets Found on an Internet’’, in Computer Communica-

tions Review v 23 no 3 (July 1993) pp 26–31

[150] SM Bellovin, ‘‘Defending Against Sequence Number Attacks’’, RFC

1948 (May 1996) at http://sunsite.auc.dk/RFC/rfc/rfc1948.html

[151] SM Bellovin, ‘‘Problem Areas for the IP Security Protocols,’’ in Pro-

ceedings of the Sixth Usenix Unix Security Symposium (1996); at http://

www.cs.columbia.edu/~smb/papers/badesp.pdf

Bibliography 905

[152] SM Bellovin, ‘‘Debit-card fraud in Canada’’, in comp.risks v 20.69;

at http://catless.ncl.ac.uk/Risks/20.69.html

[153] SM Bellovin, ‘‘Permissive Action Links’’, at http://www.research.att

.com/~smb/nsam-160/

[154] SM Bellovin, ‘ICMP Traceback Messages’, Internet Draft, March 2000, at

http://search.ietf.org/internet-drafts/draft-bellovin-

itrace-00.txt

[155] SM Bellovin, ‘‘More on Comcast Blocking Peer-to-Peer Trafﬁc’’, Oct

22 2007, at http://www.cs.columbia.edu/~smb/blog/2007-10/2007-

10-22.html; and ‘‘Comcast Apparently Blocking Some Peer-to-Peer

Trafﬁc’’, Oct 19 2007, ibid.

[156] S Bellovin, M Blaze, E Brickell, C Brooks, V Cerf, W Difﬁe, S Landau,

J Peterson, J Treichler, ‘‘Security Implications of Applying the

Communications Assistance to Law Enforcement Act to Voice over IP’’

http:// www.itaa.org/news/docs/CALEAVOIPreport.pdf

[157] SM Bellovin, WR Cheswick, ‘Firewalls and Internet Security: Repelling

the Wily Hacker’, Addison-Wesley (1994); ISBN: 0-201-63357-4

[158] SM Bellovin, M Merritt, ‘‘Encrypted Key Exchange: Password-Based

Protocols Secure Against Dictionary Attacks’’, in Proceedings of the

IEEE Symposium on Security and Privacy (1992) pp 72–84

[159] M Benantar, R Guski, KM Triodle, ‘‘Access control systems: From

host-centric to network-centric computing’’, in IBM Systems Journal v

35 no 1 (96) pp 94–112

[160] W Bender, D Gruhl, N Morimoto, A Lu, ‘‘Techniques for Data Hiding’’,

in IBM Systems Journal v 35 no 3–4 (96) pp 313–336

[161] T Benkart, D Bitzer, ‘‘BFE Applicability to LAN Environments’’, in

Seventeenth National Computer Security Conference (1994); proceedings

published by NIST, pp 227–236

[162] AD Biderman, H Zimmer, ‘The Manipulation of Human Behavior’, Wiley

1961; at http://www.archive.org/details/TheManipulationOfHuman-

Behavior

[163] F Bergadano, B Crispo, G Ruffo, ‘‘Proactive Password Checking with

Decision Trees’’, in 4th ACM Conference on Computer and Communica-

tions Security (1997), proceedings published by the ACM, ISBN

0-89791-912-2, pp 67–77

[164] DJ Bernstein, ‘Cache-Timing Attacks on AES’, preprint, 2005

906 Bibliography

[165] T Berson, ‘‘Skype Security Evaluation’’, Oct 18 2005, from http://

share.skype.com/sites/security/2005/10/skype security and

encryption.html

[166] T Berson, G Barksdale, ‘‘KSOS: Development Methodology for a Secure

Operating System’’, AFIPS Conference proceedings (1979)

[167] Bewert, ‘‘All About NSA’s and AT&T’s Big Brother Machine, the Narus

6400’’, in Dailykos Apr 7 2006; at http://www.dailykos.com/

storyonly/2006/4/8/14724/28476/

[168] K Biba, ‘Integrity Considerations for Secure Computer Systems’,Mitre

Corporation MTR-3153 (1975)

[169] E Biham, A Biryukov, ‘‘Cryptanalysis of Skipjack Reduced to 31

Rounds Using Impossible Differentials’’ in Advances in Cryptology–

Eurocrypt 97, Springer LNCS v 1592 pp 12–23

[170] E Biham, A Shamir, ‘Differential Cryptanalysis of the Data Encryption

Standard’, Springer (1993) ISBN 0-387-97930-1

[171] E Biham, A Shamir, ‘‘Differential Fault Analysis of Secret Key Cryp-

tosystems’’, in Advances in Cryptology–Crypto 97 Springer LNCS v 1294

pp 513–525

[172] E Biham, O Dunkelman, S Indesteege, N Keller, B Preneel, ‘‘How To

Steal Cars–A Practical Attack on KeeLoq’’, 2007, at http://www.cosic

.esat.kuleuven.be/keeloq/

[173] A Biryukov, A Shamir, D Wagner, ‘‘Real Time Cryptanalysis of A5/1

on a PC’’, in Fast Software Encryption (2000)

[174] R Bishop, R Bloomﬁeld, ‘‘A Conservative Theory for Long-Term

Reliability-Growth Prediction’’, in IEEE Transactions on Reliability v45

no 4 (Dec 96) pp 550–560

[175] DM Bishop, ‘‘Applying COMPUSEC to the battle ﬁeld’’, in 17th

Annual National Computer Security Conference (1994) pp 318–326

[176] M Bishop, M Dilger, ‘‘Checking for Race Conditions in File Accesses’’,

in Computing Systems Usenix v 9 no 2 (Spring 1996) pp 131–152

[177] Wolfgang Bitzer, Joachim Opfer ‘Schaltungsanordnung zum Messen der

Korrelationsfunktion zwischen zwei vorgegebenen Signalen’ [Circuit

arrangement for measuring the correlation function between two

provided signals]. German Patent DE 3911155 C2, Deutsches Patent

amt, November 11, 1993

Bibliography 907

[178] J Blackledge, ‘‘Making Money from Fractals and Chaos: Microbar’’, in

Mathematics Today v 35 no 6 (Dec 99) pp 170–173

[179] RD Blackledge, ‘‘DNA versus ﬁngerprints’’, in Journal of Forensic

Sciences v 40 (1995) p 534

[180] B Blair, ‘‘Keeping Presidents in the Nuclear Dark’’, in Bruce Blair’s

Nuclear Column, Feb 11 2004, at http://www.cdi.org/blair/

permissiveaction-links.cfm

[181] GR Blakley, ‘‘Safeguarding cryptographic keys’’, in Proceedings of NCC

AFIPS (1979), pp 313–317

[182] B Blakley, R Blakley, RM Soley, ‘CORBA Security: An Introduction to

Safe Computing with Objects’ Addison-Wesley (1999)

ISBN 0-201-32565-9

[183] MA Blaze, ‘‘Protocol Failure in the Escrowed Encryption Standard’’, in

Second ACM Conference on Computer and Communications Security, 2–4

November 1994, Fairfax, Va; proceedings published by the ACM ISBN

0-89791-732-4, pp 59–67; at http://www.crypto.com/papers/

[184] Matt Blaze, ‘‘Cryptology and Physical Security: Rights Ampliﬁcation

in Master-Keyed Mechanical Locks’’, at IEEE Symposium on Security

and Privacy 2003, at http://www.crypto.com/papers/mk.pdf

[185] MA Blaze, ‘‘Toward a Broader View of Security Protocols’’, in Security

Protocols 2004, Springer LNCS v 3957, pp 106–132

[186] MA Blaze, ‘‘Safecracking for the computer scientist’’, U. Penn Technical

Report (2004), at http://www.crypto.com/papers/

[187] MA Blaze, SM Bellovin, ‘‘Tapping, Tapping On My Network Door’’, in

Communications of the ACM (Oct 2000), Inside Risks 124; at http://

www.crypto.com/papers/carnivore-risks.html

[188] MA Blaze, J Feigenbaum, J Lacy, ‘‘Decentralized Trust Management’’,

in Proceedings of the 1996 IEEE Symposium on Security and Privacy

pp 164–173

[189] D Bleichenbacher, ‘‘Chosen Ciphertext Attacks against Protocols Based

on the RSA Encryption Standard PKCS #1’’, in Advances in Cryptology–

Crypto 98 Springer LNCS v 1462 pp 1–12

[190] G Bleumer, ‘Electronic Postage Systems–Technology, Security, Eco-

nomics’, Springer 2006; ISBN 0-387-29313-2

[191] G Bleumer, M Schunter, ‘‘Digital patient assistants: privacy vs cost in

compulsory health insurance’’, in Health Informatics Journal v 4 nos 3–4

(Dec 1998) pp 138–156

908 Bibliography

[192] B Blobel, ‘‘Clinical record Systems in Oncology. Experiences and

Developments on Cancer Registers in Eastern Germany’’, in [43]

pp 39–56

[193] JA Bloom, IJ Cox, T Kalker, JPMG Linnartz, ML Miller, CBS Traw,

‘‘Copy Protection for DVD Video’’, in Proceedings of the IEEE v87no7

(July 1999) pp 1267–1276

[194] P Bloom, ‘Descartes’ Baby: How Child Development Explains What Makes

Us Human’, Arrow (2005)

[195] ER Block, ‘Fingerprinting’, Franklin Wells (1970), SBN 85166-435-0

[196] S Blythe, B Fraboni, S Lall, H Ahmed, U de Riu, ‘‘Layout Reconstruction

of Complex Silicon Chips’’, in IEEE Journal of Solid-State Circuits v28

no 2 (Feb 93) pp 138–145

[197] WE Boebert, ‘‘Some Thoughts on the Occasion of the NSA Linux

Release’’, in Linux Journal, Jan 24 2001; at http://www.linuxjournal.

com/article/4963

[198] WE Boebert, RY Kain, ‘‘A Practical Alternative to Hierarchical

Integrity Policies’’, in 8th National Computer Security Conference (1985),

proceedings published by NIST p 18

[199] BW Boehm, ‘Software Engineering Economics’, Prentice Hall (1981),

ISBN 0-13-822122-7

[200] Rainer Boehme and Gaurav Kataria, ‘‘Models and Measures for

Correlation in Cyber-Insurance’’, at WEIS 2006

[201] N Bohm, I Brown, B Gladman, ‘Electronic Commerce–Who Carries the

Risk of Fraud?’, Foundation for Information Policy Research (2000),

available from http://www.fipr.org

[202] M Bond, ‘Understanding Security APIs,’, PhD Thesis, Cambridge, 2004

[203] M Bond, ‘‘BOOM! HEADSHOT! (Building Neo-Tactics on Network-

Level Anomalies in Online Tactical First-Person Shooters)’’ (2006), at

http://www.lightbluetouchpaper.org/2006/10/02/

[204] M Bond, ‘‘Action Replay Justice’’, Nov 22 2007, at http://www

.lightbluetouchpaper.org/2007/11/22/action-replay-justice/

[205] M Bond, SJ Murdoch, J Clulow, ‘Laser-printed PIN Mailer Vulnerability

Report’, 2005, at http://www.cl.cam.ac.uk/~sjm217/

[206] D Boneh, RA Demillo, RJ Lipton, ‘‘On the Importance of Checking

Cryptographic Protocols for Faults’’, in Advances in Cryptol-

ogy–Eurocrypt 97, Springer LNCS v 1233 pp 37–51

Bibliography 909

[207] D Boneh, M Franklin, ‘‘Identity-Based Encryption from the Weil

Pairing’’, in Advances in Cryptology–Proceedings of CRYPTO 2001,

Springer LNCS 2139 pp 213–29

[208] L Boney, AH Tewﬁk, KN Hamdy, ‘‘Digital Watermarks for Audio

Signals’’, in Proceedings of the 1996 IEEE International Conference on

Multimedia Computing and Systems, pp 473–480

[209] V Bontchev, ‘‘Possible macro virus attacks and how to prevent them’’,

in Computers and Security v 15 no 7 (96) pp 595–626

[210] N Borisov, I Goldberg, D Wagner, ‘‘Intercepting Mobile Communica-

tions: The Insecurity of 802.11’’, at Mobicom 2001

[211] NS Borenstein, ‘‘Perils and Pitfalls of Practical Cybercommerce’’, in

Communications of the ACM v 39 no 6 (June 96) pp 36–44

[212] E Bovenlander, invited talk on smartcard security, Eurocrypt 97,

reported in [69]

[213] E Bovenlander, RL van Renesse, ‘‘Smartcards and Biometrics: An

Overview’’, in Computer Fraud and Security Bulletin (Dec 95) pp 8–12

[214] C Bowden, Y Akdeniz, ‘‘Cryptography and Democracy: Dilemmas of

Freedom’’, in Liberating Cyberspace: Civil Liberties, Human Rights, and

the Internet Pluto Press (1999) pp 81–125

[215] D Bowen, ‘Top-to-Bottom Review’, Aug 2007, at http://www.sos.ca

.gov/elections/elections vsr.htm

[216] D Boyd ‘‘Facebook’s ‘Privacy Trainwreck’: Exposure, Invasion, and

Drama’’, in Apophenia Blog Sep 8th 2006, at http://www.danah.org/

papers/FacebookAndPrivacy.html

[217] M Brader, ‘‘Car-door lock remote control activates another car’s

alarm’’, in comp.risks 21.56 (Jul 2001)

[218] M Brader, ‘‘How to lose 10,000,000 pounds’’, in comp.risks v24

no 25, Apr 19 2006, at http://archives.neohapsis.com/archives/

risks/2006/0012.html

[219] RM Brady, RJ Anderson, RC Ball, ‘Murphy’s law, the ﬁtness of evolving

species, and the limits of software reliability’, Cambridge University

Computer Laboratory Technical Report no 471 (1999)

[220] S Brands, ‘Rethinking Public Key Infrastructures and Digital Certiﬁcates

–Building in Privacy’, MIT Press (2000) http://www.freetechbooks

.com/about390.html

910 Bibliography

[221] JT Brassil, S Low, NF Maxemchuk, ‘‘Copyright Protection for the

Electronic Distribution of Text Documents’’, in Proceedings of the IEEE

v 87 no 7 (July 1999) pp 1181–1196

[222] H Bray, ‘‘ ’Face testing’ at Logan is found lacking’’, in Boston Globe

July 17 2002

[223] M Brelis, ‘‘Patients’ ﬁles allegedly used for obscene calls’’, in Boston

Globe April 11, 1995; also in comp.risks v17no7

[224] DFC Brewer, MJ Nash, ‘‘Chinese Wall model’’, in Proceedings of the

1989 IEEE Computer Society Symposium on Security and Privacy

pp 215–228

[225] B Brewin, ‘‘CAC use nearly halves DOD network intrusions, Croom

says’’, in fcw.com, Jan 25 2007, at http://www.fcw.com/article97480-

01-25-07

[226] M Briceno, I Goldberg, D Wagner, ‘‘An implementation of the GSM

A3A8 algorithm’’, at http://www.scard.org/gsm/a3a8.txt

[227] D Brin, ‘The Transparent Society: Will Technology Force Us to Choose

Between Privacy and Freedom?’, Perseus Press (1999) ISBN: 0-73820144-

8; magazine version in Wired, Dec 1996, at http://www.wired.com/

wired/archive/4.12/fftransparent.html

[228] R Briol ‘‘Emanation: How to keep your data conﬁdential’’, in Symposium

on Electromagnetic Security For Information Protection, SEPI 91,

Rome, 1991

[229] British Standard 8220-1.2000, ‘Guide for Security of Buildings Against

Crime–Part 1: Dwellings’

[230] M Broersma, ‘‘Printer makers rapped over reﬁll restrictions’’,

ZDnet Dec 20 2002, at http://news.zdnet.co.uk/story/0,,t269-

s2127877,00.html

[231] F Brooks, ‘The Mythical Man-Month: Essays on Software Engineering’,

Addison-Wesley (1995 Anniversary Edition)

[232] D Brown, ‘‘Techniques for Privacy and Authentication in Personal

Communications Systems’’, in IEEE Personal Communications v2

no4(Aug95)pp6–10

[233] D Brumley, D Boneh, ‘‘Remote timing attacks are practical’’, in

Computer Networks v 48 no 5 (Aug 2005) pp 701–716

[234] D Brown, ‘‘Unprovable Security of RSA-OAEP in the Standard Model’’,

IACR eprint no 2006/223, at http://eprint.iacr.org/2006/223

Bibliography 911

[235] I Brown, L Edwards, C Marsden, ‘‘Stalking 2.0: privacy protection in a

leading Social Networking Site’’, in GikII 2–law, technology and popular

culture (2007); at http://www.law.ed.ac.uk/ahrc/gikii/docs2/

edwards.pdf

[236] JDR Buchanan, RP Cowburn, AV Jausovec, D Petit, P Seem, XO Gang,

D Atkinson, K Fenton DA Allwood, MT Bryan, ‘‘ ‘Fingerprinting’

documents and packaging’’, in Nature v 436 no 28 (July 2005) p 475

[237] JM Buchanan, ‘‘The Constitution of Economic Policy’’, 1986 Nobel

Prize Lecture, at http://nobelprize.org/nobel prizes/economics/

laureates/1986/buchanan-lecture.html

[238] H Buehler, interview with Swiss Radio International, 4/7/1994, at

http://www.funet.fi/pub/crypt/mirrors/idea.sec.dsi.unimi.it/

rpub.cl.msu.edu/crypt/docs/hans-buehler-crypto-spy.txt

[239] Bug Traq http://archives.neohapsis.com/archives/bugtraq/

[240] Bull, Dassault, Diebold, NCR, Siemens Nixdorf and Wang Global,

‘Protection Proﬁle: Automatic Cash Dispensers / Teller Machines’, version

1.0 (1999), at http://www.commoncriteriaportal.org/

[241] Bundesamt f ¨

ur Sicherheit in der Informationstechnik (German Infor-

mation Security Agency), ‘Common Criteria Protection Proﬁle–Health

Professional Card (HPC)–Heilberufsausweis (HPA)’, BSI-PP-0018, at

http://www.commoncriteriaportal.org/

[242] Bundesamt f ¨

ur Sicherheit in der Informationstechnik (German Informa-

tion Security Agency), ‘Schutzmaßnahmen gegen Lauschangriffe’ [Protec-

tion against bugs], Faltbl¨

atter des BSI v 5, Bonn, 1997; http://www.bsi

.bund.de/literat/faltbl/laus005.htm

[243] Bundesamt f ¨

ur Sicherheit in der Informationstechnik (German

Information Security Agency), ‘Elektromagnetische Schirmung von

Geb¨

auden, 2007, BSI TR-03209

[244] Bundesverfasngsgericht, ‘‘Beschluss des Ersten Senats’’, Apr 4 2006, 1

BvR 518/02 Absatz-Nr. (1–184), at http://www.bverfg.de/

entscheidungen/rs20060404 1bvr051802.html

[245] J Bunnell, J Podd, R Henderson, R Napier, J Kennedy-Moffatt,

‘‘Cognitive, associative and conventional passwords: Recall and

guessing rates’’, in Computers and Security v 16 no 7 (1997)

pp 645–657

912 Bibliography

[246] J Burke, P Warren, ‘‘How mobile phones let spies see our every move’’,

in The Observer Oct 13 2002; at http://observer.guardian

.co.uk/uk news/story/0,6903,811027,00.html

[247] RW Butler, GB Finelli, ‘‘The infeasibility of experimental quantiﬁcation

of life-critical software reliability’’, in ACM Symposium on Software for

Critical Systems (1991), ISBN 0-89791-455-4, pp 66–76

[248] Buro Jansen & Janssen, ‘Making up the rules: interception versus privacy’,

8/8/2000, at http://www.xs4all.nl/~respub/crypto/english/

[249] M Burrows, M Abadi, RM Needham, ‘‘A Logic of Authentication’’, in

Proceedings of the Royal Society of London A v 426 (1989) pp 233–271;

earlier version published as DEC SRC Research Report 39,

ftp://gatekeeper.pa.dec.com/pub/DEC/SRC/research-reports/

SRC-039.pdf

[250] RW Byrne, A Whiten, ‘Machiavellian Intelligence–Social Expertise and

the Evolution of Intellect in Monkeys, Apes and Humans’, Oxford, 1988;

see also A Whiten, RW Byrne, ‘Machiavellian Intelligence II–Extensions

and Evaluations’, Cambridge 1997

[251] ‘‘Long Distance Phone Scam Hits Internet Surfers’’, in business-

knowhow.com,athttp://www.businessknowhow.com/newlong.htm

[252] F Caldicott, ‘Report on the review of patient-identiﬁable information’,

Department of Health, 1997

[253] California Secretary of State, ‘A Report on the Feasibility of Internet

Voting’ (January 2000), at http://www.ss.ca.gov/executive/ivote/

[254] J Calvert, P Warren, ‘‘Secrets of McCartney bank cash are leaked’’, in

The Express, February 9 2000, pp 1–2

[255] J Camenisch, JM Piveteau, M Stadler, ‘‘An Efﬁcient Fair Payment

System’’, in 3rd ACM Conference on Computer and Communications

Security (1996), proceedings published by the ACM, ISBN

0-89791-829-0, pp 88–94

[256] J Camp, C Wolfram, ‘‘Pricing Security’’, in Proceedings of the CERT

Information Survivability Workshop (Oct 24–26 2000) pp 31–39

[257] J Camp, S Lewis, ‘Economics of Information Security’, Springer 2004

[258] D Campbell, ‘‘Somebody’s listening’’, in The New Statesman (12 August

1988) pp 1, 10–12; at http://jya.com/echelon-dc.htm

Bibliography 913

[259] D Campbell, ‘‘Making history: the original source for the 1988 ﬁrst

Echelon report steps forward’’ (25 February 2000), at http://cryptome

.org/echelon-mndc.htm

[260] D Campbell, ‘‘Operation Ore Exposed’’, PC Pro, July 2005, at http://

www.pcpro.co.uk/features/74690/operation-ore-exposed/

page1.html

[261] D Campbell, ‘‘Sex, Lies and the Missing Videotape’’, PC Pro, April

2007, at http://ore-exposed.obu-investigators.com/PC PRO

Operation Ore Exposed 2.html

[262] D Campbell, P Lashmar, ‘‘The new Cold War: How America spies on

us for its oldest friend–the Dollar’’, in The Independent (2 July 2000), at

http://www.independent.co.uk/news/World/Americas/2000-07/

coldwar020700.shtml

[263] JC Campbell, N Ikegami, ‘The Art of Balance in Health Pol-

icy–Maintaining Japan’s Low-Cost, Egalitarian System’, Cambridge

University Press (1998) ISBN 0-521-57122-7

[264] JP Campbell, ‘‘Speaker Recognition: A Tutorial’’, in Proceedings of the

IEEE v 85 no 9 (Sep 97) pp 1437–1462

[265] K Campbell, L Gordon, M Loeb and L Zhou, ‘‘The economic cost of

publicly announced information security breaches: empirical evidence

from the stock market’’, in Journal of Computer Security v 11 no 3 (2003)

pp 431–448

[266] C Cant, S Wiseman, ‘‘Simple Assured Bastion Hosts’’, in 13th Annual

Computer Security Application Conference (1997); proceedings published

by IEEE Computer Society, ISBN 0-8186-8274-4 ACSAC, pp 24–33

[267] ‘‘Dark horse in lead for ﬁngerprint ID card’’, Card World Independent

(May 94) p 2

[268] ‘‘German A555 takes its toll’’, in Card World International (12/94–

1/95) p 6

[269] ‘‘High tech helps card fraud decline’’ in Cards International no 117

(29 Sep 94)

[270] ‘‘Visa beefs up its anti-fraud technology’’, in Cards International no 189

(12/12/97) p 5

[271] JM Carlin, ‘‘UNIX Security Update’’, at Usenix Security 93

pp 119–130

914 Bibliography

[272] J Carr, ‘‘Doing nothing is just not an option’’, in The Observer (18/6/

2000),at http://www.guardian.co.uk/technology/2000/jun/18/

onlinesecurity.politics

[273] J Carroll, ‘Big Blues: The Unmaking of IBM’, Crown Publishers (1993),

ISBN 0-517-59197-9

[274] H Carter, ‘‘Car clock ﬁxer jailed for nine months’’, in The Guardian

(15/2/2000) p 13

[275] R Carter, ‘‘What You Are ... Not What You Have’’, in Interna-

tional Security Review Access Control Special Issue (Winter 93/94)

pp 14–16

[276] S Castano, M Fugini, G Martella, P Samarati, ‘Database Security’,

Addison-Wesley, 1994; ISBN 0-201-59375-0

[277] L Cauley, ‘‘NSA has massive database of Americans’ phone calls’’, in

USA Today Nov 11 2005, at http://www.usatoday.com/news/

washington/2006-05-10-nsa x.htm

[278] Center for Democracy and Technology, http://www.cdt.org/

[279] ‘‘The Nature and Scope of Governmental Electronic Surveillance

Activity’’, Center for Democracy and Technology, July 2006, at

http://www.cdt.org/wiretap/wiretap overview.html

[280] DW Chadwick, PJ Crook, AJ Young, DM McDowell, TL Dornan,

JP New, ‘‘Using the internet to access conﬁdential patient records: a

case study’’, in British Medical Journal v 321 (9 September 2000)

pp 612–614; at http://bmj.com/cgi/content/full/321/7261/612

[281] Chaos Computer Club, ‘How to fake ﬁngerprints?’,athttp://www

.ccc.de/biometrie/fingerabdruck kopieren.xml?language=en

[282] L Chapman, ‘Your disobedient servant’, Penguin Books (1979)

[283] Chartered Institute of Building Services Engineers, ‘Security Engineer-

ing’, Applications Manual AM4 (1991)

[284] D Chaum, ‘‘Untraceable electronic mail, return addresses, and

digital pseudonyms’’, in Communications of the ACM v24no2

(Feb 1981)

[285] D Chaum, ‘‘Blind signatures for untraceable payments’’, in Crypto 82,

Plenum Press (1983) pp 199–203

[286] D Chaum, ‘‘The Dining Cryptographers Problem: Unconditional

Sender and Recipient Untraceability’’, in Journal of Cryptology v1

(1989) pp 65–75

Bibliography 915

[287] D Chaum, A Fiat, M Naor, ‘‘Untraceable Electronic Cash’’, in Advances

in Cryptology — CRYPTO ’88, Springer LNCS v 403 pp 319–327

[288] R Chellappa, CL Wilcon, S Sirohey, ‘‘Human and Machine Recognition

of Faces: A Survey’’, in Proceedings of the IEEE v 83 no 5 (May 95)

pp 705–740

[289] HJ Choi, private discussion

[290] ‘Security Protocols–5th International Workshop’, B Christianson et al

(ed), Springer LNCS v 1360 (1998)

[291] ‘Security Protocols–6th International Workshop’, B Christianson et al

(ed), Springer LNCS v 1550 (1999). Later workshops had proceedings

as follows: 2000, v 2133; 2001, v 2467; 2002, 2845; 2003, v 3364; 2004,

v 3957.

[292] F Church (chairman), ‘Intelligence Activities–Senate Resolution 21’,

US Senate, 94 Congress, First Session, at http://cryptome.org/

nsa-4th.htm

[293] WS Ciciora, ‘‘Inside the set-top box’’, in IEEE Spectrum v12no4(Apr

95) pp 70–75

[294] T Claburn, ‘‘Former DuPont Scientist Sentenced For Trade Secret

Theft’’, in Information Week Nov 8 2007; at http://www.information-

week.com/shared/printableArticle.jhtml?articleID=202804057

[295] D Clark, D Wilson, ‘‘A Comparison of Commercial and Military Com-

puter Security Policies’’, in Proceedings of the 1987 IEEE Symposium on

Security and Privacy pp 184–194

[296] R Clark, ‘The man who broke Purple’, Little, Brown (1977) ISBN

0-316-14595-5

[297] I Clarke, ‘The Free Network Project Homepage’,athttp://freenet

.source-forge.net/

[298] RW Clarke, ‘‘The Theory of Crime prevention Though Environmental

Design’’, at www.cutr.usf.edu/security/documents%5CCPTED%5C

Theory%20of%20CPTED.pdf; see also ‘Situational Crime Prevention:

successful case studies’ (2nd edition), Harrow and Heston (1997) and

‘‘Situational Crime Prevention–Everybody’s Business’’, ACPC95, at

http://www.acpc.org.au/CONF95/Clarke.htm

[299] R Clayton, ‘‘Techno-Risk’’, at Cambridge International Symposium on

Economic Crime (2003), at http://www.cl.cam.ac.uk/~rnc1/talks/

030910-TechnoRisk.pdf

[300] R Clayton, ‘Anonymity and traceability in cyberspace’, PhD Thesis, 2005;

Cambridge University Technical Report UCAM-CL-TR-653

916 Bibliography

[301] R Clayton, ‘‘Insecure Real-Word Authentication Protocols (or Why

Phishing Is So Proﬁtable)’’, at Cambridge Security Protocols Workshop

2005,athttp://www.cl.cam.ac.uk/~rnc1/phishproto.pdf

[302] R Clayton, private conversation, 2006

[303] R Clayton, ‘‘The Rising Tide: DDoS by Defective Designs and

Defaults’’, at SRUTI 06; at http://www.cl.cam.ac.uk/~rnc1/rising-

tide.pdf

[304] R Clayton, ‘‘When ﬁrmware attacks! (DDoS by D-Link)’’, Light Blue

Touchpaper,athttp://www.lightbluetouchpaper.org/2006/04/07/

[305] R Clayton, ‘‘There aren’t that many serious spammers any more’’,

Light Blue Touchpaper,athttp://www.lightbluetouchpaper.org/

2007/04/03/

[306] R Clayton, M Bond, ‘‘Experience Using a Low-Cost FPGA Design to

Crack DES Keys’’, CHES Workshop (2002), Springer LNCS 2523

pp 579–592

[307] R Clayton, G Davies, C Hall, A Hilborne, K Hartnett, D Jones, P Mans-

ﬁeld, K Mitchell, R Payne, N Titley, D Williams, ‘LINX Best Current

Practice– Traceability’, Version 1.0, 18/5/1999, at http://www.linx

.net/noncore/bcp/traceability-bcp.html

[308] R Clayton, S Murdoch, R Watson, ‘‘Ignoring the Great Firewall of

China’’, at 6th Workshop on Privacy Enhancing Technologies (2006), at

http://www.cl.cam.ac.uk/~rnc1/ignoring.pdf

[309] J Clulow, ‘The Design and Analysis of Cryptographic APIs for Security

Devices’, MSc Thesis, University of Natal 2003

[310] A Cohen, ‘A Perfect Store’, Back Bay Books, 2003

[311] FB Cohen, ‘A Short Course on Computer Viruses’, Wiley (1994) ISBN

0-471-00769-2

[312] JL Colbert, PL Bowen, ‘A Comparison of Internal Controls: COBIT, SAC,

COSO and SAS 55/78’,athttp://www.isaca.org/bkr cbt3.htm

[313] A Collins, ‘‘Court decides software time-locks are illegal’’, in Computer

Weekly (19 August 93) p 1

[314] D Cohen, J Hashkes, ‘‘A system for controlling access to broadcast

transmissions’’, European Patent no EP0428252

Bibliography 917

[315] P Collier, A Hoefﬂer, ‘‘Greed and grievance in civil war’’, in Oxford

Economic Papers v 56 (2004) pp 563–595, at http://oep.oxfordjournals

.org/cgi/content/abstract/56/4/563

[316] D Comer, ‘‘Cryptographic techniques — secure your wireless

designs’’, in EDN (18/1/96) pp 57–68

[317] ‘‘Telecomms Fraud in the Cellular Market: How Much is Hype and

HowMuchisReal?’,inComputer Fraud and Security Bulletin (Jun 97)

pp 11–14

[318] Committee of Sponsoring Organizations of the Treadway Commission

(CSOTC),‘Internal Control–Integrated Framework’ (COSO Report, 1992);

from http://www.coso.org/

[319] ‘Communicating Britain’s Future’,athttp://www.fipr.org/polarch/

labour.html

[320] ‘‘Kavkaz-Tsentr says Russians hacking Chechen web sites’’; ‘‘

‘Information war’ waged on web sites over Chechnya’’, in Com-

munications Law in Transition Newsletter v 1 no 4 (Feb 2000), at

http://pcmlp.socleg.ox.ac.uk/transition/issue04/russia.htm

[321] Computer Emergency Response Team Coordination Center, at

http://www.cert.org/

[322] JB Condat, ‘‘Toll fraud on French PBX systems’’, in Computer Law and

Security Report v 10 no 2 (Mar/April 94) pp 89–91

[323] J Connolly, ‘‘Operation Chain Link: The Deployment of a Firewall

at Hanscom Air Force Base’’, Twelfth Annual Computer Security

Applications Conference (1996), proceedings published by the IEEE,

ISBN 0-8186-7606-X, pp 170–177

[324] E Constable, ‘‘American Express to reduce the risk of online fraud’’

[325] US Consumer Reports, ‘State of the Net’, Sep 6 2007, reported in

http://www.webknowhow.net/news/news/060831ConsumerReport-

OnlineThreats.html

[326] D Coppersmith, ‘The Data Encryption Standard (DES) and its Strength

Against Attacks’, IBM report RC 18613 (81421)

[327] Council of Europe, ‘Convention For the Protection of Individuals with

Regard to Automatic Processing of Personal Data’, European Treaty

Series no 108 (January 28, 1981): at http://www.privacy.org/pi/

intl orgs/coe/dp convention 108.txt

918 Bibliography

[328] R Cordery, L Pintsov, ‘‘History and Role of Information Security in

Postage Evidencing and Payment’’, in Cryptologia v XXIX no 3 (Jul 2005)

pp 257–271

[329] C Cowan, C Pu, D Maier, H Hinton, J Walpole, P Bakke, S Beattie, A

Grier, P Wagle, Q Zhang, ‘‘StackGuard: Automatic Adaptive Detection

and Prevention of Buffer-Overﬂow Attacks’’, 7th Usenix Security

Conference (1998) pp 63–77

[330] LH Cox, JP Kelly, R Patil, ‘‘Balancing quality and conﬁdentiality for

multivariate tabular data’’ in Privacy in Statistical Data Bases (2004)

Springer LNCS v 3050 pp 87–98

[331] JW Coyne, NC Kluksdahl, ‘‘ ‘Mainstreaming’ Automated Information

Systems Security Engineering (A Case Study in Security Run Amok)’’,

in Second ACM Conference on Computer and Communications Security

(1994) proceedings published by the ACM, ISBN 0-89791-732-4,

pp 251–257; at http://www.acm.org/pubs/contents/proceedings/

commsec/191177/

[332] J Cradden, ‘‘Printer-makers hit by new EU law’’, in Electricnews.net

December 19 2002, at http://www.electricnews.net/news.html?

code=8859027

[333] L Cranor, S Garﬁnkel, ‘Security Usability’, O’Reilly 2005, ISBN

0-596-80827-9

[334] S Craver, ‘‘On Public-key Steganography in the Presence of an

Active Warden’’, in Proceedings of the Second International Workshop on

Information Hiding (1998), Springer LNCS v 1525 pp 355–368

[335] SA Craver, M Wu, BD Liu, A Stubbleﬁeld, B Swartzlander, DS

Wallach, D Dean, EW Felten, ‘‘Reading Between the Lines: Lessons

from the SDMI Challenge’’, in Usenix Security Symposium (2000), at

http://www.cs.princeton.edu/~felten

[336] RJ Creasy, ‘‘The origin of the VM/370 time-sharing system’’, in IBM

Journal of Research and Development v 25 no 5 (Sep 1981) pp 483–490, at

http://www.research.ibm.com/journal/rd/255/libmrd2505M.pdf

[337] B Crispo, M Lomas, ‘‘A Certiﬁcation Scheme for Electronic Commerce’’,

in Security Protocols (1996), Springer LNCS v 1189 pp 19–32

[338] Cryptome.org, Deepwater documents, May 2007; at http://cryptome

.org/deepwater/deepwater.htm

Bibliography 919

[339] W Curtis, H Krasner, N Iscoe, ‘‘A Field Study of the Software Design

Process for Large Systems’’, in Communications of the ACM v31no11

(Nov 88) pp 1268–87

[340] D Cvrcek, ‘‘Counters, Freshness, and Implementation’’, Oct 2 2007, at

http://www.lightbluetouchpaper.org/

[341] F D’Addario, ‘‘Testing Security’s Effectiveness’’, in Security Manage-

ment Online October 2001, at http://www.securitymanagement.com/

library/Security D’Addario1001.html

[342] J Daemen, V Rijmen, ‘The Design of Rijndael: AES–The Advanced

Encryption Standard’, Springer (2002) ISBN 3-540-42580-2

[343] ‘‘Beating the credit card telephone fraudsters’’, in Daily Telegraph (9

Oct 1999), at http://www.telegraph.co.uk:80/

[344] G Danezis, Roger Dingledine, N Mathewson, ‘‘Mixmin-

ion: Design of a Type III Anonymous Remailer Protocol’’,

in IEEE Symposium on Security and Privacy (2003) pp 2–15; at

http://mixminion.net/miniondesign.pdf

[345] G Danezis, B Wittneben, ‘‘The Economics of Mass Surveillance’’, Fifth

Workshop on the Economics of Information Security (2006)

[346] G Danezis, R Clayton, ‘‘Introducing Trafﬁc Analysis’’, Jan 2007, in Digi-

tal Privacy: Theory, Technologies, and Practices, Taylor and Francis 2007,

at http://homes.esat.kuleuven.be/~gdanezis/TAIntro-book.pdf

[347] G Danezis, C Diaz, ‘‘Survey of Privacy Technology’’, 2007, at http://

homes.esat.kuleuven.be/~gdanezis/anonSurvey.pdf

[348] M Darman, E le Roux, ‘‘A new generation of terrestrial and satellite

microwave communication products for military networks’’, in

Electrical Communication (Q4 94) pp 359–364

[349] Two statements, made by the Data Protection Commissioners of EU and

EES countries and Switzerland, 20th International Conference on Data

Protection, Santiago de Compostela, 16–18 September 1998; available

at http://www.dataprotection.gov.uk/20dpcom.html

[350] Daubert v. Merrell Dow Pharmaceuticals, 113 S. Ct. 2786 (1993)

[351] J Daugman, ‘‘High Conﬁdence Visual Recognition of Persons by a Test

of Statistical Independence’’, in IEEE Transactions on Pattern Analysis

and Machine Intelligence v 15 no 11 (Nov 93) pp 1148–1161

920 Bibliography

[352] J Daugman, ‘Biometric decision landscapes’, Technical Report no TR482,

University of Cambridge Computer Laboratory.

[353] C Davies, R Ganesan, ‘‘BApasswd: A New Proactive Password Chec-

ker’’, in 16th National Computer Security Conference (1993), proceedings

published by NIST, pp 1–15

[354] DW Davies, WL Price, ‘Security for Computer Networks’ (John Wiley

and Sons 1984)

[355] G Davies, ‘A History of money from ancient times to the present day’,

University of Wales Press (1996) ISBN 0-7083-1351-5; related material

at http://www.ex.ac.uk/%7ERDavies/arian/llyfr.html

[356] H Davies, ‘‘Physiognomic access control’’, in Information Security

Monitor v 10 no 3 (Feb 95) pp 5–8

[357] D Davis, ‘‘Compliance Defects in Public-Key Cryptography’’, in Sixth

Usenix Security Symposium Proceedings (July 1996) pp 171–178

[358] D Davis, R Ihaka, P Fenstermacher, ‘‘Cryptographic Randomness from

Air Turbulence in Disk Drives’’ in Advances in Cryptology–Crypto 94

Springer LNCS v 839 pp 114–120

[359] J Davis, ‘‘Hackers Take Down the Most Wired Country in Europe’’, in

Wired, Aug 21 2007, at http://www.wired.com/politics/security/

magazine/15-09/ff estonia

[360] D Dean, EW Felten, DS Wallach, ‘‘Java Security: From HotJava to

Netscape and Beyond’’, in Proceedings of the 1996 IEEE Symposium on

Security and Privacy, IEEE Computer Society Press, pp 190–200

[361] C Deavours, D Kahn, L Kruh, G Mellen, B Winkel, ‘Cryptol-

ogy–Yesterday, Today and Tomorrow’, Artech House (1987), ISBN

0-89006-253-6

[362] C Deavours, D Kahn, L Kruh, G Mellen, B Winkel, ‘Selections from

Cryptologia–History, People and Technology’, Artech House (1997)

ISBN 0-89006-862-3

[363] C Deavours, L Kruh, ‘Machine Cryptography and Modern Cryptanalysis’,

Artech House (1985) ISBN 0-89006-161-0

[364] JF de Beer, ‘‘Constitutional Jurisdiction Over Paracopyright Laws’’, in

‘The Public Interest: The Future of Canadian Copyright Law’,Irwin

Law (2005)

[365] B Demoulin, L Kone, C Poudroux, P Degauque, ‘‘Electromagnetic

Radiation of Shielded Data Transmission Lines’’, in [481] pp 163–173

Bibliography 921

[366] I Denley, S Weston-Smith, ‘‘Implementing access control to protect the

conﬁdentiality of patient information in clinical information systems in

the acute hospital’’, in Health Informatics Journal v 4 nos 3–4 (Dec 1998)

pp 174–178

[367] I Denley, S Weston-Smith, ‘‘Privacy in clinical information systems in

secondary care’’ in British Medical Journal v 318 (15 May 1999)

pp 1328–1331

[368] DE Denning, ‘‘The Lattice Model of Secure Information Flow’’, in

Communications of the ACM v 19 no 5 pp 236–248

[369] DE Denning, ‘Cryptography and Data Security’, Addison-Wesley (1982)

ISBN 0-201-10150-5

[370] DE Denning, ‘Information Warfare and Security’, Addison-Wesley (1999)

ISBN 0-201-43303-6

[371] DE Denning, ‘‘Activism, Hacktivism, and Cyberterrorism: The Internet

as a Tool for Inﬂuencing Foreign Policy’’, InfowarCon 2000, at

http://www.nautilus.org/info-policy/workshop/papers/

denning.html

[372] DE Denning, PJ Denning, M Schwartz, ‘‘The tracker: a threat to

statistical database security’’, in ACM Transactions on Database Systems

v 4 no 1 (1979) pp 76–96

[373] DE Denning, PH MacDoran, ‘‘Location-Based Authentication:

Grounding Cyberspace for Better Security’’, in Computer Fraud and

Security Bulletin (Feb 96) pp 12–16

[374] DE Denning, J Schlorer, ‘‘Inference Controls for Statistical Databases’’,

in IEEE Computer v 16 no 7 (July 1983) pp 69–82

[375] Department of Defense, ‘Department of Defense Trusted Computer

System Evaluation Criteria’, DoD 5200.28-STD, December 1985

[376] Department of Defense, ‘A Guide to Understanding Covert Channel

Analysis of Trusted Systems’, NCSC-TG-030 (Nov 1993)

[377] Department of Defense, ‘Password Management Guideline’, CSC-STD-

002-85 (1985)

[378] Department of Defense, ‘A Guide to Understanding Data Remanence in

Automated Information Systems’, NCSC-TG-025 (1991)

[379] Department of Defense, ‘Technical Rationale behind CSC-STD-003-85:

computer security requirements’, CSC-STD-004-85 (1985)

922 Bibliography

[380] Department of Defense, News Transcript, Oct 20 2007, at http://

cryptome.org/af-squirm/af-squirm.htm

[381] Department of Justice, ‘Guidelines for Searching and Seizing Com-

puters’, 1994; at http://www.epic.org/security/computer search

guidelines.txt

[382] Y Desmedt, Y Frankel, ‘‘Threshold cryptosystems’’, in Advances in

Cryptology–Proceedings of Crypto 89, Springer LNCS v 435

pp 307–315

[383] J Dethloff, ‘‘Special Report: Intellectual Property Rights and Smart

Card Patents: The Past–The Present–The Future’’, in Smart Card

News (Feb 96) pp 36–38

[384] J Dibbell, ‘‘The Life of the Chinese Gold Farmer’’, in New York Times

Jun 17 2007; at http://www.nytimes.com/2007/06/17/magazine/

17lootfarmers-t.html?ex=1340769600&en=87f96d5d8676cbad&ei

=5124&partner=permalink&exprod=permalink

[385] W Difﬁe, ME Hellman, ‘‘New Directions in Cryptography’’, in IEEE

Transactions on information theory v 22 no 6 (Nov 76) pp 644–654

[386] W Difﬁe, ME Hellman, ‘‘Exhaustive cryptanalysis of the NBS Data

Encryption Standard’’, in Computer v 10 no 6 (June 77) pp 74–84

[387] W Difﬁe, S Landau, ‘Privacy on the Line–The Politics of Wiretapping and

Encryption’, MIT Press (1998) ISBN 0-262-04167-7

[388] E Dijkstra, ‘‘Solution of a problem in concurrent programming control’’,

in Communications of the ACM v 8 no 9 (1965) p 569

[389] Allana Dion, ‘‘Rapper Veriﬁed?’’, on Gridgrind Dec 8 2007; at

http://www.gridgrind.com/?p=229

[390] The Discount Long Distance Digest, at http://www.thedigest.com/

shame/

[391] D Dittrich, ‘Distributed Denial of Service (DDoS) Attacks/tools’,at

http://staff.washington.edu/dittrich/

[392] AK Dixit, ‘Lawlessness and Economics’, Princeton University Press, 2003

[393] RC Dixon, ‘Spread Spectrum Systems with Commercial Applications’,

Wiley (1994) ISBN 0-471-59342-7

[394] H Dobbertin, ‘‘Cryptanalysis of MD4’’, Journal of Cryptology v11

no 4 (1998) pp 253–270

Bibliography 923

[395] B Dole, S Lodin, E Spafford, ‘‘Misplaced Trust: Kerberos 4 Session

Keys’’, in Internet Society Symposium on Network and Distributed Sys-

tem Security, proceedings published by the IEEE, ISBN 0-8186-7767-8,

pp 60–70

[396] JR Douceur, ‘‘The Sybil Attack’’, IPTPS 2002, at www.cs.rice.edu/

Conferences/IPTPS02/101.pdf

[397] J Doward, ‘‘The friend of the stars who fell from grace’’, in The Observer

Aug 26 2007; at http://www.guardian.co.uk/media/2007/aug/26/

radio.television

[398] P Drahos, J Braithwaite, ‘Information Feudalism–Who Owns the Knowl-

edge Economy?’, Earthscan 2002

[399] S Drimer, ‘‘Banks don’t help ﬁght phishing’’, Mar 10 2006, Light Blue

Touchpaper; at http://www.lightbluetouchpaper.org/2006/03/

10/banks-dont-help-fight-phishing/

[400] S Drimer, ‘Volatile FPGA design security–a survey’, 2007, at

http://www.cl.cam.ac.uk/~sd410/papers/fpga security.pdf

[401] S Drimer, SJ Murdoch, ‘‘Keep your enemies close: Distance bounding

against smartcard relay attacks’’, in 16th USENIX Security Symposium

(2007), at http://www.cl.cam.ac.uk/~sd410/papers/sc relay.pdf

[402] IE Dror, D Charlton, AE P´

eron, ‘‘Contextual information renders

experts vulnerable to making erroneous identiﬁcations’’, in Forensic

Science International 156 (2006) 74–78

[403] IE Dror, D Charlton, ‘‘Why Experts Make Errors’’, in Journal of Forensic

Identiﬁcation v 56 no 4 (2006) pp 600–616; at http://users

.ecs.soton.ac.uk/id/biometrics.html

[404] IE Dror, ‘‘Don’t forget us humans’’, in The Times, July 31 2006; at

http://users.ecs.soton.ac.uk/id/biometrics.html

[405] I Drury, ‘‘Pointing the ﬁnger’’, in Security Surveyor v 27 no 5 (Jan 97)

pp 15–17; at http://users.ecs.soton.ac.uk/id/biometrics.html

[406] C Dyer, ‘‘Europe’s concern over UK data protection ’defects’

revealed’’, in The Guardian Oct 1 2007; at http://www.guardian

.co.uk/uk news/story/0,,2180729,00.html

[407] N Eagle, A Pentland, D Lazer, ‘‘Inferring Social Network Structure

using Mobile Phone Data’’, 2007, at http://reality.media.mit.edu/

pdfs/network structure.pdf

924 Bibliography

[408] W van Eck, ‘‘Electromagnetic Radiation from Video Display Units: An

Eavesdropping Risk? in Computers and Security v 4 (1985) pp 269–286

[409] P Eckersley, ‘‘Comcast is also Jamming Gnutella (and Lotus Notes?)’’,

EFF Deeplinks Blog Oct 20 2007, at http://www.eff.org/

deeplinks/2007/10/comcast-also-jamming-gnutella-and-lotus-

notes

[410] The Economist, ‘‘Digital rights and wrongs’’ (17/7/1999); see

www.economist.com

[411] The Economist, ‘‘Living in the global goldﬁsh bowl ’’, 18–24 Dec 1999,

Christmas special; see www.economist.com

[412] The Economist, ‘‘A price worth paying?’’, May 19 2005

[413] The Economist, ‘‘Cyberwarfare–Newly nasty’’, May 24 2007

[414] The Economist, ‘‘After smart weapons, smart soldiers’’, Oct 25 2007

[415] The Economist, ‘‘Getting the message, at last’’, Dec 13 2007, at http://

www.economist.com/opinion/displaystory.cfm?story id=10286400

[416] B Edelman, ‘‘Adverse Selection in Online ‘Trust’ Certiﬁcates’’, at Fifth

Workshop on the Economics of Information Security (2006); at http://

weis2006.econinfosec.org/

[417] J Edge, ‘‘IPv6 source routing: history repeats itself’’, May 7, 2007; at

http://lwn.net/Articles/232781/

[418] EDRI, FIPR and VOSN, ‘Response to the European commission consulta-

tion on the review of the ‘‘acquis communautaire’’ in the ﬁeld of copyright

and related rights’, Oct 2004, at http://www.edri.org/campaigns/

[419] A Edwards, ‘‘BOLERO, a TTP project for the Shipping Industry’’, in

Information Security Technical Report v 1 no 1 (1996) pp 40–45

[420] M van Eeten, JM Bauer, M de Bruijne, J Groenewegen, W Lenstra, ‘The

Economics of Malware: Security Decisions, Incentives and Externalities’

OECD Report, 2008

[421] M Eichin, J Rochlis, ‘‘With Microscope and Tweezers: An Analysis of

the Internet Virus of November 1988’’, in Proceedings of the 1989 IEEE

Symposium on Security and Privacy pp 326–343

[422] Electronic Frontier Foundation, http://www.eff.org

Bibliography 925

[423] Electronic Frontier Foundation, ‘Cracking DES: Secrets of Encryption

Research, Wiretap Politics, and Chip Design’, EFF (1998); ISBN 1-56592-

520-3; http://cryptome.org/cracking-des.htm

[424] Electronic Frontier Foundation, Felten, et al., v. RIAA, et al. at http://

www.eff.org/IP/DMCA/Felten vRIAA/

[425] Electronic Frontier Foundation, ‘‘DocuColor Tracking Dot Decoding

Guide’’, at http://w2.eff.org/Privacy/printers/docucolor/

[426] M Ellims, ‘‘Is Security Necessary for Safety?’’, in ESCAR 2006,

at http://www.pi-shurlok.com/uploads/documents/security and

safety.pdf

[427] JH Ellis, The History of Non-secret Encryption, 1987, at http://www

.jya.com/ellisdoc.htm

[428] C Ellison, B Schneier, ‘‘Ten Risks of PKI: What You’re Not Being Told

About Public Key Infrastructure’’, in Computer Security Journal v XIII

no 1 (2000); also at http://www.counterpane.com/pki-risks.html

[429] EMV documents available from EMVCo LLP at http://www

.emvco.com/

[430] ‘Enfopol Papiere’, Telepolis archiv special 1998/9, at http://www.heise

.de/tp/deutsch/special/enfo/default.html

[431] P Enge, T Walter, S Pullen, CD Kee, YC Chao, YJ Tsai, ‘‘Wide Area

Augmentation of the Global Positioning System’’, in Proceedings of the

IEEE v 84 no 8 (Aug 96) pp 1063–1088

[432] Electronic Privacy Information Center, http://www.epic.org

[433] EPIC, ‘Approvals for Federal Pen Registers and Trap and Trace Devices

1987–1998’,athttp://www.epic.org/privacy/wiretap/stats/

penreg.html

[434] EPIC, ‘Report of the Director of the Administrative Ofﬁce of the United

States Courts’,athttp://www.epic.org/privacy/wiretap/stats/

1999-report/wiretap99.pdf

[435] EPIC, ‘Wiretapping’,athttp://www.epic.org/privacy/wiretap/

[436] J Epstein, S Matsumoto, G McGraw, ‘‘Software Security and SOA:

Danger, Will Robinson!’’, in IEEE Security and Privacy, Jan/Feb 2006,

926 Bibliography

pp 80–83, at http://www.cigital.com/papers/download/bsi12-soa

.doc.pdf

[437] J Epstein, H Orman, J McHugh, R Pascale, M Branstad, A Marmor-

Squires, ‘‘A High Assurance Window System Prototype’’, in Journal of

Computer Security v 2 no 2–3 (1993) pp 159–190

[438] J Epstein, R Pascale, ‘‘User Interface for a High Assurance Windowing

System’’, in Ninth Annual Computer Security Applications Conference

(1993), proceedings published by the IEEE, ISBN 0-8186-4330-7,

pp 256–264

[439] T Escamilla, ‘Intrusion Detection–Network Security beyond the Firewall’,

Wiley (1998) ISBN 0-471-29000-9

[440] J Essinger, ‘ATM Networks–Their Organisation, Security and Future’,

Elsevier 1987

[441] A Etzioni, ‘The Limits of Privacy’, Basic Books (1999) ISBN 0-465-04089-6

[442] European Commission, ‘Impact assessment–amending Framework Deci-

sion 2002/475/JHA on combating terrorism’, Brussels, Nov 6 2007, SEC

(2007) 1424, at http://www.ipex.eu/ipex/webdav/site/myjahiasite/

groups/CentralSupport/public/2007/SEC 2007 1424/COM SEC

(2007)1424 EN.pdf

[443] European Parliament, ‘Development of surveillance technology and risk

of abuse of economic information’, Luxembourg (April 1999) PE 166.184/

Part 3/4, at http://www.gn.apc.org/duncan/stoa.htm

[444] European Union, ‘Directive on the protection of individuals with regard

to the processing of personal data and on the free movement of such data’,

Directive 95/46/EC, at http://www.privacy.org/pi/intl orgs/ec/

eudp.html

[445] European Union, ‘‘Draft Council Resolution on the lawful interception

of telecommunications in relation to new technologies’’ 6715/99

(15/3/1999), at http://www.fipr.org/polarch/enfopol19.html;for

background see http://www.fipr.org/polarch/

[446] European Union, ‘Directive on the retention of data generated or

processed in connection with the provision of publicly available elec-

tronic communications services or of public communications networks’,

2006/24/EC

[447] European Union, ‘‘Promoting Data Protection by Privacy Enhancing

Technologies (PETs)’’, COM(2007) 228 ﬁnal, Brussels, May 2nd 2007

Bibliography 927

[448] Eurosmart, ‘Protection Proﬁle–Smart Card Integrated Circuit With

Embedded Software’, 1999, at http://www.commoncriteriaportal.org/

[449] R Evans, D Leigh, ‘‘GM subsidiary paid conman for ‘blagged’ private

data, court told’’, The Guardian Apr 24, 2007; at http://www.guardian

.co.uk/crime/article/0,,2064180,00.html

[450] Facebook, Photos,athttp://www.facebook.com/help.php?page=7

[451] G Faden, ‘‘Reconciling CMW Requirements with Those of X11

Applications’’, in Proceedings of the 14th Annual National Computer

Security Conference (1991)

[452] M Fairhurst, ‘‘The Hedge End Experiment’’, in International Security

Review no 85 (Summer 94) p 20

[453] M Fairhurst, ‘‘Signature veriﬁcation revisited: promoting practical

exploitation of biometric technology’’, in Electronics and Communication

Engineering Journal v 9 no 6 (Dec 97) pp 273–280

[454] B Feder, ‘‘Face-Recognition Technology Improves’’, New York Times

Mar 14 2003; at http://www.nytimes.com/2003/03/14/technology/

14FACE.html

[455] Federal Committee on Statistical Methodology, ‘ Statistical Policy Work-

ing Paper 22’ (Revised 2005)–‘Report on Statistical Disclosure Limitation

Methodology’), at http://www.fcsm.gov/working-papers/spwp22.html

[456] Federal Trade Commission v Audiotex Connection, Inc., and others, at

http://www.ftc.gov/os/1997/9711/Adtxamdfcmp.htm

[457] Federal Trade Commission and Department of Commerce,

‘Electronic Signatures in Global and National Commerce Act–The Con-

sumer Consent Provision in Section 101(c)(1)(C)(ii) ’, June 2001, at

http://www.ftc.gov/os/2001/06/esign7.htm

[458] Federal Trade Commission, ‘ID Theft: When Bad Things Happen to

Your Good Name’,athttp://www.consumer.gov/idtheft/

[459] Federal Trade Commission, ‘ ChoicePoint Settles Data Security

Breach Charges; to Pay 10MillioninCivilPenalties,5 Million for Consumer

Redress’, Jan 26 2006, at http://www.ftc.gov/opa/2006/01/

choicepoint.shtm

[460] Federation of American Scientists, http://www.fas.org

[461] H Federrath, J Thees, ‘‘Schutz der Vertraulichkeit des Aufenthaltsorts

von Mobilfunkteilnehmern’’, in Datenschutz und Datensicherheit

(June 1995) pp 338–348

928 Bibliography

[462] P Fellwock (using pseudonym ‘Winslow Peck’), ‘‘U.S. Electronic

Espionage: A Memoir’’, in Ramparts v 11 no 2 (August 1972) pp 35–50; at

http://jya.com/nsa-elint.htm

[463] E Felten, ‘‘Facebook and the Campus Cops’’, Mar 20 2006, at http://

www.freedom-to-tinker.com/?p=994

[464] JS Fenton, ‘Information Protection Systems’, PhD Thesis, Cambridge

University, 1973

[465] N Ferguson, B Schneier, ‘‘A Cryptographic Evaluation of IPSEC’’, at

http://www.counterpane.com/ipsec.html

[466] D Ferraiolo, R Kuhn, ‘‘Role-Based Access Control’’, in 15th National

Computer Security Conference (1992), proceedings published by NIST,

pp 554–563

[467] D Ferraiolo, R Kuhn, R Chandramouli, ‘Role-Based Access Control’,

Artech House, 2007

[468] D Fewer, P Gauvin, A Cameron, ‘Digital Rights Management Technolo-

gies and Consumer Privacy–An Assessment of DRM Applications Under

Canadian Privacy Law’, September 2007, at www.cippic.ca

[469] A Fiat, M Naor, ‘‘Broadcast Encryption’’, in Crypto ’93, Springer LNCS

v 773 pp 480–491

[470] PFJ Fillery, AN Chandler, ‘‘Is lack of quality software a password to

information security problems?’’, in IFIP SEC 94 paper C8

[471] ‘‘Psychologists and banks clash over merits of photographs on cards’’,

in Financial Technology International Bulletin v 13 no 5 (Jan 96) pp 2–3

[472] D Fine, ‘‘Why is Kevin Lee Poulsen Really in Jail?’’, at http://www

.well.com/user/fine/journalism/jail.html

[473] G Fiorentini, S Pelzman, ‘The Economics of Organised Crime’, Cambridge

University Press 1995

[474] B Fischer, talk at Cryptologic History Symposium, NSA, October 1999;

reported in Cryptologia v 24 no 2 (Apr 2000) pp 160–167

[475] RA Fisher, ‘The Genetical Theory of Natural Selection’, Clarendon Press,

Oxford (1930); 2nd ed. Dover Publications, NY (1958)

[476] J Flanagan, ‘‘Prison Phone Phraud (or The RISKS of Spanish)’’, reporting

University of Washington staff newspaper,incomp.risks v 12.47; at

http://catless.ncl.ac.uk/Risks/20.69.html

Bibliography 929

[477] M Fleet, ‘‘Five face sentence over notes that passed ultraviolet tests’’, in

The Daily Telegraph (23/12/1999), available at http://www.telegraph

.co.uk:80/

[478] B Fletcher, C Roberts, K Risser, ‘‘The Design and Implementation of a

Guard Installation and Administration Framework’’, in Third Annual

SElinux Symposium,athttp://selinux-symposium.org/

[479] S Fluhrer, I Mantin, A Shamir, ‘‘Weaknesses in the Key Scheduling

Algorithm of RC4’’ in SAC 2001

[480] SN Foley, ‘‘Aggregation and separation as noninterference proper-

ties’’, in Journal of Computer Security v 1 no 2 (1992) pp 158–188

[481] Fondazione Ugo Bordoni, ‘Symposium on Electromagnetic Security for

Information Protection’, Rome, Italy, 21–22 November 1991

[482] S Forrest, SA Hofmeyr, A Somayaji, ‘‘Computer Immunology’’, in

Communications of the ACM v 40 no 10 (Oct 97) pp 88–96

[483] DS Fortney, JJ Lim, ‘‘A technical approach for determining the

importance of information in computerised alarm systems’’, in

Seventeenth National Computer Security Conference (1994), proceedings

published by NIST; pp 348–357

[484] The Foundation for Information Policy Research, http://www.fipr.org

[485] B Fox, ‘‘How to keep thieves guessing’’, in New Scientist (3rd June 95)

p18

[486] B Fox, ‘‘Do not adjust your set . . . we have assumed radio control’’, in

New Scientist 8 Jan 2000, at http://www.newscientist.com/ns/

20000108/newsstory6.html

[487] B Fox, ‘‘The pirate’s tale’’, in New Scientist 18 Dec 1999, at http://www

.newscientist.com/ns/19991218/thepirates.html

[488] D Fox, ‘‘IMSI-Catcher’’, in Datenschutz und Datensicherheit v21no9

(9/97) p 539

[489] D Foxwell, ‘‘Off-the-shelf, on to sea’’, in International Defense Review v

30 (Jan 97) pp 33–38

[490] D Foxwell, M Hewish, ‘‘GPS: is it lulling the military into a false sense

of security?’’, in Jane’s International Defense Review (Sep 98) pp 32–41

[491] LJ Fraim, ‘‘SCOMP: A Solution to the Multilevel Security Problem’’, in

IEEE Computer v 16 no 7 (July 83) pp 26–34

930 Bibliography

[492] T Frank, ‘‘Tougher TSA bomb tests raise stakes for screeners’’, in USA

Today Oct 18 2007, at http://www.usatoday.com/printedition/news/

20071018/a insidescreeners18.art.htm

[493] J Franks, P Hallam-Baker, J Hostetler, S Lawrence, P Leach, A Luotonen,

L Stewart, ‘‘HTTP Authentication: Basic and Digest Access

Authentication’’, RFC 2617

[494] T Fraser, ‘‘LOMAC: Low Water-Mark Integrity Protection for COTS

Environments’’, in Proceedings of the 2000 IEEE Symposium on Security

and Privacy, IEEE Computer Society Press, pp 230–245

[495] J Frizell, T Phillips, T Groover, ‘‘The electronic intrusion threat to

national security and emergency preparedness telecommunications:

an awareness document’’, in Seventeenth National Computer Security

Conference (1994); proceedings published by NIST, pp 378–399

[496] M Frost, ‘Spyworld: Inside the Canadian & American Intelligence

Establishments’, Diane Publishing Co (1994), ISBN 0-78815791-4

[497] ‘‘Banks ﬁngerprint customers to cut cheque fraud’’, in Fraud Watch

(1997) no 1 p 9

[498] ‘‘Chip cards reduce fraud in France’’, in Fraud Watch (1996) no 1 p 8

[499] ‘‘Counterfeit and cross border fraud on increase warning’, in Fraud

Watch (1996) no 1 pp 6–7

[500] ‘‘Finger minutiae system leaps the 1:100,000 false refusal barrier’’, in

Fraud Watch (1996) no 2 pp 6–9

[501] ‘‘Widespread card skimming causes European concern’’, in Fraud

Watch (1997) v 3 pp 1–2

[502] P Freiberger, M Swaine, ‘Fire in the Valley — the Making of the Personal

Computer’, McGraw-Hill (1999) ISBN 0-07-135892-7

[503] M Freiss, ‘Protecting Networks with Satan’, O’Reilly (1997) ISBN

1-56592-425-8

[504] AM Froomkin, ‘‘The Death of Privacy’’, in Stanford Law Review v52

pp 1461–1543, at http://www.law.miami.edu/~froomkin/articles/

privacy-deathof.pdf

[505] D Frye, ‘‘Open and Secure: Linux Today and Tomorrow’’, in 2007 Secu-

rity Enhanced Linux Symposium,athttp://selinux-symposium.org/

2007/agenda.php

Bibliography 931

[506] DA Fulghum, ‘‘Communications Intercepts Pace EP-3s’’, in Aviation

Week and Space Technology v 146 no 19 (5/5/97) pp 53–54

[507] Dr Fun, ‘Suddenly, just as Paul was about to clinch the job interview,

he received a visit from the Ghost of Usenet Postings Past’, 1996, at

http://www.ibiblio.org/Dave/Dr-Fun/df9601/df960124.jpg

[508] S Furber, ‘ARM System Architecture’, Addison-Wesley (1996); ISBN

0-210-40352-8

[509] M Galecotti, ‘‘Russia’s eavesdroppers come out of the shadows’’, in

Jane’s Intelligence Review v 9 no 12 (Dec 97) pp 531–535

[510] Sir F Galton, ‘‘Personal identiﬁcation and description,” in Nature

(21/6/1888) pp 173–177

[511] Sir F Galton, ‘Finger Prints’, Macmillan, 1892

[512] HF Gaines, ‘Cryptanalysis–a study of ciphers and their solution’, Dover,

ISBN 486-20097-3 (1939, 1956)

[513] T Gandy, ‘‘Brainwaves in fraud busting’’, Banking Technology (Dec

95/Jan 96) pp 20–24

[514] R Gardner, A Yasinsac, M Bishop, T Kohno, Z Hartley, J Kerski,

D Gainey, R Walega, E Hollander, M Gerke, ‘Software Review and

Security Analysis of the Diebold Voting Machine Software’, Floriday

State University, Jul 27 2007, at http://www.sait.fsu.edu/news/2007-

07-31.shtml

[515] S Garﬁnkel, ‘Database Nation’, O’Reilly and Associates (2000) ISBN

1-56592-653-6

[516] S Garﬁnkel, ‘Design Principles and Patterns for Computer Systems That

Are Simultaneously Secure and Usable’, PhD Thesis, MIT 2005, at

http://www.simson.net/thesis/

[517] S Garﬁnkel, G Spafford, ‘Practical Unix and Internet Security’, O’Reilly

and Associates (1996); ISBN 1-56592-148-8

[518] W Gates, W Buffett, ‘‘The Bill & Warren Show’’, in Fortune, 20/7/1998

[519] B Gellman, D Linzer, CD Leonnig, ‘‘Surveillance Net Yields Few

Suspects’’, in Washington Post, Feb 5 2006 p A01; at http://www

.washingtonpost.com/wp-dyn/content/article/2006/02/04/

AR2006020401373 pf.html

[520] General Accounting Ofﬁce, USA, ‘Medicare–Improvements Needed to

Enhance Protection of Conﬁdential Health Information’, GAO/HEHS-99-

140; http://www.gao.gov/AIndexFY99/abstracts/he99140.htm

932 Bibliography

[521] RM Gerecht, ‘‘The Counterterrorist Myth’’, in Atlantic Monthly,

Jul–Aug 2001, at http://www.theatlantic.com/doc/200107/gerecht

[522] E German, ‘‘Problem Idents’’, at http://onin.com/fp/problemidents

.html

[523] E German, ‘‘Legal Challenges to Fingerprints’’, at http://www.onin

.com/fp/daubert links.html

[524] A Gidari, JP Morgan, ‘‘Survey of State Electronic and Digital Signature

Legislative Initiatives’’, at http://www.ilpf.org/digsig/digrep.htm

[525] D Gifford, A Spector, ‘‘The CIRRUS Banking Network’’, in Communi-

cations of the ACM v 28 no 8 (Aug 1985) pp 797–807

[526] D Gilbert, ‘‘If only gay sex caused global warming’’, LA Times,July2,

2006; http://www.latimes.com/news/opinion/sunday/commentary/

la-op-gilbert2jul02,0,4254536.story?coll=la-sunday-

commentary

[527] M Gill, A Spriggs, ‘Assessing the impact of CCTV’,UKHomeOfﬁce

Research Study 292, at www.homeoffice.gov.uk/rds/pdfs05/

hors292.pdf

[528] J Gilmore, ‘‘Nacchio affects spy probe’’, in Denver Post Oct 20 2007;

cited in ‘‘NSA solicited illegal Qwest mass wiretaps right after Bush

inauguration’’, Cryptography List Oct 20 2007, at http://www

.mailarchive.com/cryptography%40metzdowd.com/msg08213.html

[529] T Gilovich, D Grifﬁn, D Kahneman, ‘Heuristics and Biases–The

Psychology of Intuitive Judgment’, Cambridge University Press 2002

[530] AA Giordano, HA Sunkenberg, HE de Pdero, P Stynes, DW Brown,

SC Lee, ‘‘A Spread-Spectrum Simulcast MF Radio Network’’, in IEEE

Transactions on Communications v TC-30 no 5 (May 1982) pp 1057–1070

[531] WN Goetzmann, ‘Financing Civilization’,http://viking.som.yale

.edu/will/finciv/chapter1.htm

[532] J Goguen, J Meseguer, ‘‘Security Policies and Security Models’’, in

Proceedings of the 1982 IEEE Computer Society Symposium on Research

in Security and Privacy pp 11–20

[533] I Goldberg, D Wagner, ‘‘Randomness and the Netscape browser’’, in

Dr Dobbs Journal no 243 (Jan 96) pp 66–70

[534] L Goldberg, ‘‘Recycled Cold-War Electronics Battle Cellular Telephone

Thieves’’, in Electronic Design v 44 no 18 (3 September 1996)

pp 41–42

Bibliography 933

[535] O Goldreich, ‘Foundations of Cryptography’, v 1 and 2, 2001 and 2004, at

http://www.wisdom.weizmann.ac.il/~oded/foc-book.html

[536] S Goldwasser, S Micali, ‘‘Probabilistic encryption’’, in J Comp Sys Sci v

28 (1984) pp 270–299

[537] D Gollmann, ‘Computer Security’, Wiley (1999); ISBN 0-471-97884-2

[538] D Gollmann, ‘‘What Is Authentication?’’, in Security Protocols (2000),

Springer LNCS 1796 pp 65–72

[539] L Gong, ‘Inside Java 2 Platform Security: Architecture, API Design, and

Implementation’, Addison-Wesley (1999); ISBN: 0-201-31000-7

[540] L Gong, DJ Wheeler, ‘‘A matrix key-distribution scheme’’, in Journal of

Cryptology v 2 no 1 (1990) pp 51–59

[541] R Gonggrijp, WJ Hengeveld, A Bogk, D Engling, H Mehnert, F Rieger,

P Scheffers, B Wels, ‘‘Nedap/Groenendaal ES3B voting computer–a

security analysis’’, Oct 2006, at http://www.wijvertrouwenstem-

computersniet.nl/Nedap-en

[542] D Goodin, ‘‘Anatomy of an eBay scam’’, in The Register, Mar 21 2007;

at http://www.theregister.co.uk/2007/03/21/ebay fraud anatomy/

[543] D Goodin, ‘‘Firefox leak could divulge sensitive info’’, in The Register,

Aug 13 2007; at http://www.theregister.co.uk/2007/08/13/firefox

remote leakage/

[544] D Goodin, ‘‘TJX agrees to pay banks $41 m to cover Visa losses’’, in The

Channel Register, Dec 3 2007; at http://www.channelregister.co.uk/

2007/12/03/tjx settlement agreement/print.html

[545] D Goodin, ‘‘Ukrainian eBay scam turns Down Syndrome man into

cash machine’’, in The Register Nov 8 2007, at http://www.theregister

.co.uk/2007/11/08/ebay victims track their mules/

[546] JI Gordon, ‘‘Copyright.Protection@Internet.net’’, in 3W Valparaiso Jour-

nal of Law and Technology v 1 (24/1/1999), at http://www.wvjolt.wvu

.edu/v3i1/gordon.htm

[547] KE Gordon, RJ Wong, ‘‘Conducting Filament of the Programmed

Metal Electrode Amorphous Silicon Antifuse’’, in Proceedings of

International Electron Devices Meeting, Dec 93; reprinted as pp 6-3 to

6-10, QuickLogic Data Book (1994)

[548] MF Grady, F Parisi, ‘The Law and economics of Cybersecurity’, Cam-

bridge University Press, 2006

934 Bibliography

[549] RM Graham, ‘‘Protection in an Information Processing Utility,’’ in

Communications of the ACM v 11 no 5 (May 1968) pp 365–369

[550] FT Grampp, RH Morris, ‘‘UNIX Operating System Security’’, AT&T

Bell Laboratories Technical Journal v 63 no 8 (Oct 84) pp 1649–1672

[551] S Granneman, ‘‘Electronic Voting Debacle’’, in The Register Nov 18

2003; at http://www.theregister.co.uk/2003/11/18/electronic

voting debacle/

[552] RD Graubart, JL Berger, JPL Woodward, ‘Compartmented Mode,

Workstation Evaluation Criteria, Version 1’, Mitre MTR 10953, 1991 (also

published by the Defense Intelligence Agency as document DDS-

2600-6243-91)

[553] J Gray, P Helland, P O’Neil, D Shasha, ‘‘The Dangers of Replication

and a Solution,’’ in SIGMOD Record v 25 no 2 (1996) pp 173–182

[554] J Gray, P Syverson, ‘‘A Logical Approach to Mulilevel Security of

Probabilistic Systems,’’ in Distributed Computing v 11 no 2 (1988)

[555] TC Greene, ‘‘Vista security overview: too little too late’’, in The Register

Feb 20 2007, at http://www.theregister.co.uk/2007/02/20/vista

security oversold/

[556] T Greening, ‘‘Ask and Ye Shall Receive: A Study in Social Engineering’’,

in SIGSAC Review v 14 no 2 (Apr 96) pp 9–14

[557] M Gregory, P Losocco, ‘‘Using the Flask Security Architecture to Facili-

tate Risk Adaptable Access Controls’’, in 2007 Security Enhanced Linux

Symposium,athttp://selinux-symposium.org/2007/agenda.php

[558] A Griew, R Currell, ‘A Strategy for Security of the Electronic Patient

Record’, Institute for Health Informatics, University of Wales, Aberyst-

wyth, March 1995

[559] V Groebner, J Peck, M Kyburz, ‘Who Are You?: Identiﬁcation, Deception,

and Surveillance in Early Modern Europe’, Zone Books, 2007

[560] J Gross, ‘‘Keeping Patients’ Details Private, Even From Kin’’, in New

York Times July 3 2007

[561] D Grover, ‘The protection of computer software–its technology and

applications’, British Computer Society / Cambridge University Press

(1992) ISBN 0-521-42462-3

Bibliography 935

[562] D Gruhl, W Bender, ‘‘Information Hiding to Foil the Casual

Counterfeiter’’, in Proceedings of the Second International Workshop

on Information Hiding (Portland, Apr 98), Springer LNCS v 1525

pp 1–15

[563] LC Guillou, M Ugon, JJ Quisquater, ‘‘The Smart Card–A Standardised

Security Device Dedicated to Public Cryptology’’, in [1171] pp 561–613

[564] R Gupta, SA Smolka, S Bhaskar, ‘‘On Randomization in Sequential and

Distributed Algorithms’’, in ACM Computing Surveys v 26 no 1 (March

94) pp 7–86

[566] P Gutmann, ‘‘Secure Deletion of Data from Magnetic and Solid-State

Memory’’, in Sixth USENIX Security Symposium Proceedings (July 1996)

pp 77–89

[567] P Gutmann, ‘‘Software Generation of Practically Strong Random

Numbers’’, in Seventh Usenix Security Symposium Proceedings (Jan 1998)

pp 243–257

[568] P Gutmann, ‘‘Data Remanence in Semiconductor Devices’’, in Usenix

Security Symposium (2001)

[569] P Gutmann, ‘‘Invalid banking cert spooks only one user in 300’’, Cryp-

tography List May 16 2005; at http://www.mail-archive.com/crypto-

graphy%40metzdowd.com/msg03852.html

[570] P Gutmann, ‘‘A Cost Analysis of Windows Vista Content Protection’’,

April 2007, at http://www.cs.auckland.ac.nz/~pgut001/pubs/vista

cost.html

[571] P Gutmann, ‘‘Commercial CAPTCHA-breakers for sale’’, Cryptography

List Oct 22 2007, at http://www.mail-archive.com/cryptography%

40metzdowd.com/msg08203.html; see also http://www.lafdc.com/

captcha/

[572] S Haber, WS Stornetta, ‘‘How to time-stamp a digital document’’, in

Journal of Cryptology v 3 no 2 (1991) pp 99–111

[573] S Haber, WS Stornetta, ‘‘Secure Names for Bit-Strings’’, in 4th ACM

Conference on Computer and Communications Security (1997) pp 28–35

936 Bibliography

[574] W Hackmann, ‘‘Asdics at war’’, in IEE Review v 46 no 3 (May 2000)

pp 15–19

[575] ‘‘Chris Carey Arrested In New Zealand’’, in Hack Watch News

(9/1/1999), at http://www.iol.ie/~kooltek/legal.html

[576] N Hager, ‘Secret Power–New Zealand’s Role in the International Spy

Network’, Craig Potton Publishing (1996) ISBN 0-908802-35-8

[577] JA Halderman, ‘‘Amazon’s MP3 Store Wisely Forgoes Watermarks’’,

Oct 2 2007, at http://www.freedom-to-tinker.com/?p=1207

[578] PS Hall, TK Garland-Collins, RS Picton, RG Lee, ‘Radar’, Brassey’s New

Battleﬁeld Weapons Systems and Technology Series (v 9), ISBN

0-08-037711-4

[579] Hall of Shame, at http://www.pluralsight.com/wiki/default.aspx/

Keith.HallOfShame; see also http://www.threatcode.com/admin

rights.htm

[580] H Handschuh, P Paillier, J Stern, ‘‘Probing attacks on tamper-resistant

devices’’, in Cryptographic Hardware and Embedded Systems–CHES 99,

Springer LNCS v 1717 pp 303–315

[581] R Hanley, ‘‘Millions in thefts plague New Jersey area’’, in New York

Times, February 9, 1981, late city ﬁnal edition, section A; p 1

[582] R Hanson, ‘‘Can wiretaps remain cost-effective?’’, in Communications

of the ACM v 37 no 12 (Dec 94) pp 13–15

[583] V Harrington, P Mayhew, ‘Mobile Phone Theft’,UKHomeOfﬁce

Research Study 235, January 2002

[584] MA Harrison, ML Ruzzo, JD Ullman, ‘‘Protection in Operating

Systems’’, in Communications of the ACM v 19 no 8 (Aug 1976)

pp 461–471

[585] A Hassey, M Wells, ‘‘Clinical Systems Security–Implementing the

BMA Policy and Guidelines’’, in [43] pp 79–94

[586] Health and Safety Executive, nuclear safety reports at http://www

.hse.gov.uk/nsd/, especially ‘HSE Team Inspection of the Control and

Supervision of Operations at BNFL’s Sellaﬁeld Site’,http://www.hse

.gov.uk/nsd/team.htm

[587] LJ Heath, ‘An Analysis of the Systemic Security Weaknesses of the US

Navy Fleet Broadcasting System 1967–1974, as Exploited by CWO John

Walker’, MSc Thesis, Georgia Tech, at http://www.fas.org/irp/

eprint/heath.pdf

Bibliography 937

[588] T Heim, ‘‘Outrage at 500,000 DNA database mistakes’’, Daily Telegraph,

Aug 28 2007

[589] N Heintze, ‘‘Scalable Document Fingerprinting’’, in Second USENIX

Workshop on Electronic Commerce (1996), ISBN 1-880446-83-9

pp 191–200

[590] S Helmers, ‘‘A Brief History of anon.penet.ﬁ–The Legendary Anony-

mous Remailer’’, CMC Magazine, Sep 1997; at http://www.december

.com/cmc/mag/1997/sep/helmers.html

[591] D Hencke, ‘‘Child beneﬁt workers kept out of loop on data security’’,

in The Guardian Dec 15 2007, at http://politics.guardian.co.uk/

homeaffairs/story/0,,2227999,00.html

[592] E Henning, ‘‘ The Stamp of Incompetence’’, c’t magazine, Sep 3 2007; at

http://www.heise-security.co.uk/articles/95341

[593] Sir ER Henry, ‘Classiﬁcation and Uses of Finger Prints’ George Rutledge

& Sons, London, 1900

[594] I Herbert, ‘‘No evidence against man in child porn inquiry who ‘killed

himself’ ’’, in The Independent Oct 1 2005, at http://news.independent

.co.uk/uk/legal/article316391.ece

[595] Herodotus, ‘Histories’; Book 1 123.4, Book 5 35.3 and Book 7 239.3

[596] ‘‘Interview with David Herson - SOGIS’’, September 25, 1996, in

Ingeniørennet,athttp://www.ing.dk/redaktion/herson.htm

[597] A Herzberg, M Jakobsson, S Jarecki, H Krawczyk, M Yung, ‘‘Proactive

Public Key and Signature Systems’’, 4th ACM Conference on Computer

and Communications Security (1997) pp 100–110

[598] RA Hettinga, ‘‘Credit Card Fraud Higher. Credit Card Fraud Lower’’,

in nettime (22/3/2000), at http://www.nettime.org/nettime

.w3archive/200003/msg00184.html

[599] M Hewish, ‘‘Combat ID advances on all fronts’’, in International Defense

Review v 29 (Dec 96) pp 18–19

[600] Hewlett-Packard, ‘IA-64 Instruction Set Architecture Guide’,athttp://

devresource.hp.com/devresource/Docs/Refs/IA64ISA/index.html

[601] TS Heydt-Benjamin, DV Bailey, K Fu, A Juels, T OHare, ‘‘Vulnerabilities

in First-Generation RFID-enabled Credit Cards’’, in Proceedings of

Eleventh International Conference on Financial Cryptography and Data

Security, 2007

938 Bibliography

[602] HM Heys, ‘‘A Tutorial on Linear and Differential Cryptanalysis’’, in

Cryptologia v XXVI no 3 (Jul 2002) pp 189–221; at www.engr.mun.ca/

~howard/PAPERS/ldc tutorial.ps

[603] M Hickley, ‘‘Taliban tapping British troops’ mobiles to taunt soldiers’

families’’, in Daily Mail, Aug 22 2007

[604] HJ Highland ‘‘Electromagnetic Radiation Revisited’’, in Computers &

Security v5 (1986) 85–93 and 181–184

[605] HJ Highland, ‘‘Perspectives in Information Technology Security’’, in

Proceedings of the 1992 IFIP Congress, ‘Education and Society’, IFIP A-13

v II (1992) pp 440–446

[606] TF Himdi, RS Sandhu, ‘‘Lattice-Based Models for Controlled Sharing

of Conﬁdential Information in the Saudi Hajj System’’, in 13th Annual

Computer Security Applications Conference, San Diego, California,

December 8–12 1997; proceedings published by the IEEE Computer

Society, ISBN 0-8186-8274-4; pp 164–174

[607] Eric von Hippel, ‘‘Open Source Software Projects as User Innovation

Networks’’, Open Source Software Economics 2002 (Toulouse)

[608] Jack Hirshleifer, ‘‘Privacy: Its Origin, Function and Future’’, in Journal

of Legal Studies v 9 (Dec 1980) pp 649–664

[609] Jack Hirshleifer, ‘‘From weakest-link to best-shot: the voluntary

provision of public goods’’, in Public Choice v 41, (1983) pp 371–386

[610] Jack Hirshleifer, ‘Economic behaviour in Adversity’,Universityof

Chicago Press, 1987

[611] T Hobbes, ‘Leviathan, or The Matter, Forme and Power of a Common

Wealth Ecclesiasticall and Civil, commonly called Leviathan’ (1651)

[612] J Hoffman, ‘‘Implementing RBAC on a Type Enforced System’’, in 13th

Annual Computer Security Applications Conference, San Diego, Califor-

nia, December 8–12 1997; proceedings published by the IEEE Computer

Society, ISBN 0-8186-8274-4; pp 158–163

[613] P Hoffmann, ‘‘Microsoft Windows Root Certiﬁcate Security Issues’’,

2007, at http://www.proper.com/root-cert-problem/

[614] S Hoffmann, ‘‘Salesforce.com Responds To Phishing Scams’’, on CNN,

Nov 8 2007, at http://www.crn.com/security/202804065

Bibliography 939

[615] G Hogben, ‘‘Security Issues and Recommendations for Online Social

Networks’’, ENISA Position Paper, Oct 2007

[616] G Hoglund, G McGraw, ‘Exploiting Software–How to Break Code’,

Addison Wesley 2004

[617] G Hoglund, G McGraw, ‘Exploiting Online Games–Cheating Massively

Distributed Systems’, Addison-Wesley 2007

[618] P Hollinger, ‘‘Single language for barcode Babel’’, in Financial Times

(25/7/2000) p 15

[619] C Holloway, ‘‘Controlling the Use of Cryptographic Keys’’, in

Computers and Security v 14 no 7 (95) pp 587–598

[620] DI Hopper, ‘‘Authorities Sue Adult Web Sites’’, in Washington Post

(23/8/2000); at http://www.washingtonpost.com/

[621] G Horn, B Preneel, ‘‘Authentication and Payment in Future Mobile

Systems’’, in ESORICS 98, Springer LNCS v 1485, pp 277–293; journal

version in Journal of Computer Security v 8 no 2–3 (2000) pp 183–207

[622] JD Horton, R Harland, E Ashby, RH Cooper, WF Hyslop, DG

Nickerson, WM Stewart, OK Ward, ‘‘The Cascade Vulnerability

Problem’’, in Journal of Computer Security v 2 no 4 (93) pp 279–290

[623] V Hougham, ‘‘Sociological Skills Used in the Capture of Saddam

Hussein’’, in Footnotes (Jul/Aug 2005), at http://www.asanet.org/

footnotes/julyaugust05/fn3.html

[624] House of Commons Health Committee, ‘The ELectronic Patient Record’,

6th Report of Session 2006–7, at http://www.publications

.parliament.uk/pa/cm200607/cmselect/cmhealth/422/422.pdf

[625] House of Lords Science and Technology Committee, ‘Personal Internet

Security, 5th Report of Session 2006–7

[626] JD Howard, ‘An Analysis Of Security Incidents On The Internet

1989–1995’, PhD thesis (1997), Carnegie Mellon University, at

http://www.cert.org/research/JHThesis/Start.html

[627] M Howard, D LeBlanc, ‘Writing Secure Code’, (second edition),

Microsoft Press 2002, ISBN 0-7356-1722-8

[628] D Howell, ‘‘Counterfeit technology forges ahead’’, in The Daily

Telegraph (22/3/1999), available at http://www.telegraph.co.uk:80/

940 Bibliography

[629] A Huang, ‘Hacking the Xbox–An Introduction to Reverse Engineering’,

No Starch Press (2003)

[630] Q Hu, JY Yang, Q Zhang, K Liu, XJ Shen, ‘‘An automatic seal imprint

veriﬁcation approach’’, in Pattern Recognition v 28 no 8 (Aug 95)

pp 251–266

[631] G Huber, ‘‘CMW Introduction’’, in ACM SIGSAC v 12 no 4 (Oct 94)

pp 6–10

[632] N Htoo-Mosher, R Nasser,N Zunic, J Straw, ‘‘E4 ITSEC Evaluation of

PRISM on ES/9000 Processors’’, in 19th National Information Systems

Security Conference (1996), proceedings published by NIST, pp 1–11

[633] M Hypponen, ‘‘Malware goes mobile’’, in Scientiﬁc American Nov 2006

pp 70–77

[634] ‘‘Role of Communications in Operation Desert Storm’’, in IEEE

Communications Magazine (Special Issue) v 30 no 1 (Jan 92)

[635] ‘‘New England shopping mall ATM scam copied in UK’’, in Information

Security Monitor v 9 no 7 (June 94) pp 1–2

[636] ‘‘Pink Death Strikes at US West Cellular’’, in Information Security

Monitor v 9 no 2 (Jan 94) pp 1–2

[637] Independent Security Evaluators Inc., ‘‘Content Protection for Optical

Media’’, May 2005, at www.securityevaluators.com/eval/spdc aacs

2005.pdf

[638] Information Systems Audit and Control Association, ‘Control Objectives

for Information and related Technology’,athttp://www.isaca.org/

cobit.htm

[639] Information Systems Audit and Control Association, ‘Exam Preparation

Materials available from ISACA’,athttp://www.isaca.org/cert1.htm

[640] International Atomic Energy Authority (IAEA), ‘The Physical Protec-

tion of Nuclear Material and Nuclear Facilities’, INFCIRC/225/Rev 4,

http://www.iaea.org/Publications/Documents/Infcircs/1999/

infcirc225r4c/rev4 content.html

[641] IBM, ‘IBM 4758 PCI Cryptographic Coprocessor–CCA Basic Services Ref-

erence and Guide, Release 1.31 for the IBM 4758-001, available through

http://www.ibm.com/security/cryptocards/

[642] IEE Electronics and Communications Engineering Journal v12no3(June

2000)–special issue on UMTS

Bibliography 941

[643] IEEE Carnahan Conference,http://www.carnahanconference.com/

[644] IEEE Spectrum, special issue on nuclear safekeeping, v 37 no 3 (Mar

2000)

[645] ‘‘Ex-radio chief ‘masterminded’ TV cards scam’’, in The Independent

17/2/1998; see also ‘‘The Sinking of a Pirate’’, Sunday Independent,

1/3/1998

[646] Intel Corporation, ‘Intel Architecture Software Developer’s Manual–

Volume 1: Basic Architecture’, Order number 243190 (1997)

[647] Intel Corporation and others, ‘Advanced Access Content System

(AACS)–Technical Overview (informative)’, July 21 2004, at http://www

.aacsla.com/home

[648] International Electrotechnical Commission, ‘Digital Audio Interface’,

IEC 60958, Geneva, February 1989

[649] T Iwata, K Kurosawa, ‘‘OMAC: One-Key CBC MAC’’, in Fast Software

Encryption (2003) Springer LNCS v 2887 pp 129–153

[650] C Jackson, DR Simon, DS Tan, A Barth, ‘‘An Evaluation of Extended

Validation and Picture-in-Picture Phishing Attacks’’, USEC 2007;at

www.usablesecurity.org/papers/jackson.pdf

[651] I Jackson, personal communication

[652] L Jackson, ‘‘BT forced to pay out refunds after free calls fraud’’, in The

Sunday Telegraph (9/2/1997); at http://www.telegraph.co.uk:80/

[653] TN Jagatic, NA Johnson, M Jakobsson, F Menczer, ‘‘Social Phishing’’,

in Communications of the ACM v 50 no 10 (Oct 2007) pp 94–100

[654] G Jagpal, ‘Steganography in Digital Images’, undergraduate thesis,

Selwyn College, Cambridge University, 1995

[655] AK Jain, R Bolle, S Pankanti, ‘Biometrics–Personal Identiﬁcation in

Networked Society’, Kluwer (1999); ISBN 0-7923-8346-1

[656] AK Jain, L Hong, S Pankanti, R Bolle, ‘‘An Identity-Authentication

System Using Fingerprints’’, in Proceedings of the IEEE v85no9(Sep

97) pp 1365–1388

[657] S Jajodia, W List, G McGregor, L Strous (editors), ‘Integrity and Internal

Control in Information Systems–Volume 1: Increasing the conﬁdence in

information systems’, Chapman & Hall (1997) ISBN 0-412-82600-3

942 Bibliography

[658] M Jakobsson, ‘‘Modeling and Preventing Phishing Attacks’’, in Financial

Cryptography 2005,atwww.informatics.indiana.edu/markus/

papers/phishing jakobsson.pdf

[659] M Jakobsson, S Myers, ‘Phishing and Countermeasures’, Wiley 2007

[660] M Jakobsson, Z Ramzan, ‘Crimeware’, Addison-Wesley 2008

[661] M Jay, ‘‘ACPO’s intruder policy — underwritten?’’, in Security Surveyor

v 26 no 3 (Sep 95) pp 10–15

[662] D Jedig, ‘‘Security by example’’, 2006, at http://syneticon.net/

support/security/security-by-example.html

[663] N Jefferies, C Mitchell, M Walker, ‘‘A Proposed Architecture for

Trusted Third Party Services’’, in Cryptography: Policy and Algorithms,

Springer LNCS v 1029 pp 98–104

[664] R Jenkins, ‘‘Hole-in-wall thief used MP3 player’’, in The Times

Nov 15 2006; at http://www.timesonline.co.uk/article/0,,29389-

2453590,00.html

[665] A Jerichow, J M ¨

uller, A Pﬁtzmann, B Pﬁtzmann, M Waidner, ‘‘Real-

Time Mixes: a Bandwidth-Efﬁcient Anonymity Protocol’’, in IEEE

Journal on Special Areas in Communications v 16 no 4 (May 98)

pp 495–509

[666] John Young Architect, http://www.jya.com

[667] K Johnson, ‘‘One Less Thing to Believe In: Fraud at Fake Cash

Machine’’, in New York Times 13 May 1993 p 1

[668] RG Johnston, ARE Garcia, ‘‘Vulnerability Assessment of Security

Seals’’, in Journal of Security Administration v 20 no 1 (June 97)

pp 15–27; the Vulnerability Assessment Team’s papers are at

http://pearl1.lanl.gov/seals/, backed up at http://www.cl.cam

.ac.uk/~rja14/preprints/Johnston/for non-US readers

[669] P Jones, ‘‘Protection money’’, in Computer Business Review v4no12

(Dec 96) pp 31–36

[670] RV Jones, ‘Most Secret War’, Wordsworth Editions (1978,1998) ISBN

1-85326-699-X

[671] RV Jones, ‘Reﬂections on Intelligence’, Octopus (1989) ISBN

0-7493-0474-X

Bibliography 943

[672] J Jonsson, B Kaliski, ‘‘Public-Key Cryptography Standards (PKCS) #1:

RSA Cryptography Speciﬁcations Version 2.1’’, RFC 3447

[673] A Jøsang, K Johannesen, ‘‘Authentication in Analogue Telephone

Access Networks’’, in Pragocrypt 96, proceedings published by CTU

Publishing House, Prague, ISBN 80-01-01502-5; pp 324–336

[674] Dorothy Judd v Citibank, 435 NYS, 2d series, pp 210–212, 107

Misc.2d 526

[675] MY Jung, ‘‘Biometric Market and Industry Overview’’, IBG,Dec8

2005; at http://events.wcoomd.org/files/style%20elements/17-

Jung-IBG%20-%20Biometric%20Market%20and%20Industry%

20Overview.pdf

[676] D Kahn, ‘The Codebreakers’, Macmillan (1967)

[677] D Kahn, ‘Seizing the Enigma’, Houghton Mifﬂin (1991); ISBN

0-395-42739-8

[678] D Kahn, ‘‘Soviet Comint in the Cold War’’, in Cryptologia vXXIIno1

(Jan 98) pp 1–24

[679] D Kahneman, ‘‘Maps of Bounded Rationality: a Perspective on Intuitive

Judgment and Choice’’, Nobel Prize Lecture, 2002

[680] B Kaliski, ‘‘PKCS #7: Cryptographic Message Syntax Version 1.5’’, RFC

2315

[681] JB Kam, GI Davida, ‘‘A Structured Design of Substitution-Permutation

Encryption Network’’, in Foundations of Secure Computation, Academic

Press (1978)

[682] M Kam, G Fielding, R Conn, ‘‘Writer Identiﬁcation by Professional

Document Examiners’’, in Journal of Forensic Sciences v 42 (1997)

pp 778–786

[683] M Kam, G Fielding, R Conn, ‘‘Effects of Monetary Incentives on Per-

formance of Nonprofessionals in Document Examination Proﬁciency

Tests’’, in Journal of Forensic Sciences v 43 (1998) pp 1000–1004

[684] MS Kamel, HC Shen, AKC Wong, RI Campeanu, ‘‘System for the

recognition of human faces’’, in IBM Systems Journal v 32 no 2 (1993)

pp 307–320

[685] MH Kang, IS Moskowitz, ‘‘A Pump for Rapid, Reliable, Secure Com-

munications’’, in 1st ACM Conference on Computer and Communications

944 Bibliography

Security, 3–5/11/93, Fairfax, Virginia; Proceedings published by the

ACM, ISBN 0-89791-629-8, pp 118–129

[686] MH Kang, JN Froscher, J McDermott, O Costich, R Peyton, ‘‘Achieving

Database Security through Data Replication: The SINTRA Prototype’’,

in 17th National Computer Security Conference (1994) pp 77–87

[687] MH Kang, IS Moskowitz, DC Lee, ‘‘A Network Pump’’, in IEEE

Transactions on Software Engineering v 22 no 5 (May 96) pp 329–338

[688] MH Kang, IS Moskowitz, B Montrose, J Parsonese, ‘‘A Case Study of

Two NRL Pump Prototypes’’, in 12th Annual Computer Security Appli-

cations Conference, San Diego CA, December 9–13 1996; proceedings

published by the IEEE, ISBN 0-8186-7606-X, pp 32–43

[689] MH Kang, JN Froscher, IS Moskowitz, ‘‘An Architecture for Multilevel

Secure Interoperability’’, in 13th Annual Computer Security Applications

Conference, San Diego, California, December 8–12 1997; proceedings

published by the IEEE Computer Society, ISBN 0-8186-8274-4;

pp 194–204

[690] CS Kaplan, ‘‘Privacy Plan Likely to Kick Off Debate’’, in New York

Times (28 July 2000), at http://www.nytimes.com/

[691] MH Kang, IS Moskowitz, S Chincheck, ‘‘The Pump: A Decade of

Covert Fun’’, at 21st Annual Computer Security Applications Conference

(2005)

[692] PA Karger, VA Austell, DC Toll, ‘‘A New Mandatory Security Policy

Combining Secrecy and Integrity’’, IBM Research Report RC 21717

(97406) 15/3/2000

[693] PA Karger, RR Schell, ‘‘Thirty Years Later’: Lessons from the Multics

Security Evaluation’’, at ACSAC 2002 pp 119–126

[694] S Karp, ‘‘Facebook’s Public Search Listing Is Problematic for Users’’, in

Digitalmediawire Sep 5 2007, at http://www.dmwmedia.com/news/

2007/09/06/facebook-s-public-search-listing-is-problematic-

for-users

[695] F Kasiski, ‘Die Geheimschriften und die Dechiffrier-Kunst’, Mittler &

Sohn, Berlin (1863)

[696] ‘KASUMI Speciﬁcation’, ETSI/SAGE v 1 (23/12/1999), at http://www

.etsi.org/dvbandca/

[697] S Katzenbeisser, FAP Petitcolas, ‘Information hiding–Techniques for

steganography and digital watermarking’, Artech House (2000) ISBN

1-58053-035-4

Bibliography 945

[698] C Kaufman, R Perlman, M Speciner, ‘Network Security–Private

Communication in a Public World’, Prentice Hall 1995; ISBN

0-13-061466-1

[699] DT Keitkemper, SF Platek, KA Wolnik, ‘‘DNA versus ﬁngerprints’’, in

Journal of Forensic Sciences v 40 (1995) p 534

[700] GC Kelling, C Coles, ‘Fixing Broken Windows: Restoring Order and

Reducing Crime in Our Communities’ Martin Kessler Books (1996)

[701] L Kelly, T Young, in Computing Jan 25 2007; at http://www.vnunet

.com/computing/news/2173365/uk-firms-naive-usb-stick

[702] J Kelsey, B Schneier, D Wagner, ‘‘Protocol Interactions and the

Chosen Protocol Attack’’, in Security Protocols–Proceedings of the 5th

International Workshop (1997) Springer LNCS v 1361 pp 91–104

[703] J Kelsey, B Schneier, D Wagner, C Hall, ‘‘Cryptanalytic Attacks on

Pseudorandom Number Generators’’, in Fifth International Workshop

on Fast Software Encryption (1998), Springer LNCS v 1372 pp 168–188

[704] J Kelsey, B Schneier, D Wagner, C Hall, ‘‘Side Channel Cryptanalysis

of Product Ciphers,’’ in ESORICS 98, Springer LNCS v 1485 pp 97–110

[705] R Kemp, N Towell, G Pike, ‘‘When seeing should not be believing:

Photographs, credit cards and fraud’’, in Applied Cognitive Psychology

v 11 no 3 (1997) pp 211–222

[706] R Kemmerer, ‘‘Shared Resource Matrix Methodology: An Approach

to Identifying Storage and Timing Channels’’, in IEEE Transactions on

Computer Systems v 1 no 3 (1983) pp 256–277

[707] R Kemmerer, C Meadows, J Millen, ‘‘Three Systems for Cryptographic

Protocol Analysis’’, in Journal of Cryptology v 7 no 2 (Spring 94)

pp 79–130

[708] MG Kendall, B Babington-Smith, ‘‘Randomness and Random

Sampling Numbers’’, part 1 in Journal of the Royal Statistical Society

v 101 pp 147–166; part 2 in Supplement to the Journal of the Royal

Statistical Society,v6no1pp51–61

[709] T Kendall, ‘‘Pornography, Rape, and the Internet’’, at The Economics of

the Software and Internet Industries (Softint 2007), at http://people

.clemson.edu/~tkendal/internetcrime.pdf

[710] ST Kent, MI Millett, ‘Who Goes There? Authentication Through the Lens

of Privacy’, National Research Council 2003; at http://www.nap.edu/

catalog.php?record id=10656

946 Bibliography

[711] JO Kephardt, SR White, ‘‘Measuring and Modeling Computer Virus

Prevalence’’, in Proceedings of the 1993 IEEE Symposium on Security

and Privacy pp 2–15

[712] JO Kephardt, SR White, DM Chess, ‘‘Epidemiology of computer

viruses’’, in IEEE Spectrum v 30 no 5 (May 93) pp 27–29

[713] A Kerckhoffs, ‘‘La Cryptographie Militaire’’, in Journal des Sciences Mil-

itaires, 9 Jan 1883, pp 5–38; http://www.cl.cam.ac.uk/users/fapp2/

kerckhoffs/

[714] OS Kerr, ‘‘Computer Records and the Federal Rules of Evidence’’, in

USA Bulletin (Mar 2001), at http://www.usdoj.gov/criminal/

cybercrime/usamarch2001 4.htm

[715] PJ Kerry, ‘‘EMC in the new millennium’’, in Electronics and Communi-

cation Engineering Journal v 12 no 2 pp 43–48

[716] D Kesdogan, H Federrath, A Jerichow, ‘‘Location Management

Strategies Increasing Privacy in Mobile Communication’’, in 12th

International Information Security Conference (1996), Samos, Greece;

proceedings published by Chapman & Hall, ISBN 0-412-78120-4,

pp 39–48

[717] J Kilian, P Rogaway, ‘‘How to protect DES Against Exhaustive Key

Search’’, in Advances in Cryptology–Crypto 96 Springer LNCS v 1109

pp 252–267

[718] J King, ‘‘Bolero — a practical application of trusted third party

services’’, in Computer Fraud and Security Bulletin (July 95) pp 12–15

[719] Kingpin, ‘‘iKey 1000 Administrator Access and Data Compromise’’, in

BugTraq (20/7/2000), at http://www.L0pht.com/advisories.html

[720] DV Klein, ‘‘Foiling the Cracker; A Survey of, and Improvements to

Unix Password Security’’, Proceedings of the USENIX Security Work-

shop. (1990);http://www.deter.com/unix/

[721] RL Klevans, RD Rodman, ‘Voice Recognition’, Artech House (1997);

ISBN 0-89006-927-1

[722] HM Kluepfel, ‘‘Securing a Global Village and its Resources: Baseline

Security for Interconnected Signaling System # 7 Telecommunications

Networks’’, in First ACM Conference on Computer and Communications

Security (1993), proceedings published by the ACM , ISBN 0-89791-629-

8, pp 195–212; later version in IEEE Communications Magazine v32

no 9 (Sep 94) pp 82–89

Bibliography 947

[723] N Koblitz, ‘A Course in Number Theory and Cryptography’, Springer

Graduate Texts in Mathematics no 114 (1987), ISBN 0-387-96576-9

[724] N Koblitz, A Menezes, ‘‘Another Look at ‘Provable Security’ ’’, in

Journal of Cryptology v 20 no 1 (2007) pp 3–37

[725] ER Koch, J Sperber, ‘Die Datenmaﬁa’, Rohwolt Verlag (1995) ISBN

3-499-60247-4

[726] M Kochanski, ‘‘A Survey of Data Insecurity Devices’’, in Cryptologia v

IX no 1 pp 1–15

[727] P Kocher, ‘‘Timing Attacks on Implementations of Difﬁe-Hellman,

RSA, DSS, and Other Systems’’, in Advances in Cryptology–Crypto 96

Springer LNCS v 1109 pp 104–113

[728] P Kocher, ‘‘Differential Power Analysis’’, in Advances in Cryptology–

Crypto 99 Springer LNCS v 1666 pp 388–397; a brief version was

presented at the rump session of Crypto 98

[729] P Kocher, ‘‘Design and Validation Strategies for Obtaining Assurance

in Countermeasures to Power Analysis and Related Attacks’’, at FIPS

Physical Security Workshop, Hawaii 2005; at http://csrc.nist.gov/

groups/STM/cmvp/documents/fips140-3/physec/papers/physe-

cpaper09.pdf

[730] KJ Koelman, ‘‘A Hard Nut to Crack: The Protection of Technological

Measures’’, in European Intellectual Property Review (2000) pp 272–288;

at http://www.ivir.nl/Publicaties/koelman/hardnut.html

[731] T Kohno, A Stubbleﬁeld, AD Rubin, DS Wallach, ‘‘Analysis of an

Electronic Voting System’’, Johns Hopkins TR 2003-19; also published

in IEEE Symposium on Security and Privacy (2004)

[732] S Kokolakis, D Gritzalis, S Katsikas, ‘‘Generic Security Policies for

Health Information Systems’’, in Health Informatics Journal v 4 nos 3–4

(Dec 1998) pp 184–195

[733] O K¨

ommerling, MG Kuhn, ‘‘Design Principles for Tamper-Resistant

Smartcard Processors’’, in Usenix Workshop on Smartcard Technology,

proceedings published by Usenix (1999), ISBN 1-880446-34-0 pp 9–20

[734] A Kondi, R Davis, ‘‘Software Encryption in the DoD’’, in 20th National

Information Systems Security Conference (1997), proceedings published

by NIST, pp 543–554

948 Bibliography

[735] LD Koontz, VC Melvin, ‘‘Health Information Technology–Efforts Con-

tinue but Comprehensive Privacy Approach Needed for National Strat-

egy’’, GAO, 2007; at http://www.gao.gov/new.items/d07988t.pdf

[736] BJ Koops, ‘Crypto Law Survey’,athttp://rechten.uvt.nl/koops/

cryptolaw/; see also his thesis ‘The Crypto Controversy: A Key Conﬂict

in the Information Society’

[737] C Kopp, ‘‘Electromagnetic Bomb–Weapon of Electronic Mass

Destruction’’, at http://www.abovetopsecret.com/pages/ebomb.html

[738] DP Kormann, AD Rubin, ‘‘Risks of the Passport Single Signon Proto-

col’’, in Computer Networks (July 2000); at http://avirubin.com/vita

.html

[739] M Kotadia, ‘‘Citibank e-mail looks phishy: Consultants’’, Zdnet Nov 9

2006; at http://www.zdnet.com.au/news/security/soa/Citibank-e-

mail-looks-phishy-Consultants/0,130061744,339272126,00.htm

[740] KPHO, ‘‘Sodomized Ex-McDonald’s Employee Wins $6.1 M’’, KPHO,

Oct 6 2007; at http://www.kpho.com/news/14277937/detail.html

[741] H Krawczyk, M Bellare, R Canetti, ‘HMAC: Keyed-Hashing for Message

Authentication’, RFC 2104 (Feb 1997), at http://www.faqs.org/rfcs/

rfc2104.html

[742] B Krebs, ‘‘Just How Bad Is the Storm Worm?’’, in The Washington Post

Oct 1 2007; at http://blog.washingtonpost.com/securityfix/

2007/10/the storm worm maelstrom or te.html

[743] B Krebs, ‘‘Salesforce.com Acknowledges Data Loss’’, in The Washington

Post Nov 6 2007; at http://blog.washingtonpost.com/securityfix/

2007/11/salesforcecom acknowledges dat.html

[744] S Krempl, ‘‘Lauschangriff am Geldautomaten’’, in Der Spiegel Jan 8

1999; at http://web.archive.org/web/20001031024042/http://www

.spiegel.de/netzwelt/technologie/0,1518,13731,00.html

[745] HM Kriz, ‘‘Phreaking recognised by Directorate General of France

Telecom’’, in Chaos Digest 1.03 (Jan 93)

[746] I Krsul, EH Spafford, ‘‘Authorship analysis: identifying the author of a

program’’, in Computers and Security v 16 no 3 (1996) pp 233–257

[747] D K ¨

ugler, ‘‘ ‘Man in the Middle’ Attacks on Bluetooth’’, in Financial

Cryptography 2004, Springer LNCS v 2742 pp 149–161

Bibliography 949

[748] MG Kuhn, ‘‘Cipher Instruction Search Attack on the Bus-Encryption

Security Microcontroller DS5002FP’’, in IEEE Transactions on Comput-

ers v 47 no 10 (Oct 1998) pp 1153–1157

[749] MG Kuhn, private communication

[750] MG Kuhn, ‘‘Optical Time-Domain Eavesdropping Risks of CRT

Displays’’ in IEEE Symposium on Security and Privacy (2002)

[751] MG Kuhn, ‘‘An Asymmetric Security Mechanism for Navigation

Signals’’, in Information Hiding 2004 Springer LNCS 3200 pp 239–252

[752] MG Kuhn, ‘‘Electromagnetic Eavesdropping Risks of Flat-Panel Dis-

plays’’, in PET 2004,athttp://www.cl.cam.ac.uk/~mgk25/pet2004-

fpd.pdf

[753] MG Kuhn, RJ Anderson, ‘‘Soft Tempest: Hidden Data Transmission

Using Electromagnetic Emanations’’, in Proceedings of the Second Inter-

national Workshop on Information Hiding (Portland, Apr 98), Springer

LNCS v 1525 pp 126–143

[754] R Kuhn, P Edfors, V Howard, C Caputo, TS Philips, ‘‘Improving Public

Switched Network Security in an Open Environment’’, in Computer,

August 1993, pp 32–35

[755] S Kumar, C Paar, J Pelzl, G Pfeiffer, M Schimmler, ‘‘Breaking Ciphers

with COPACOBANA–A Cost-Optimized Parallel Code Breaker’’, in

CHES 2006 and at http://www.copacobana.org/

[756] J Kuo, ‘‘Storm Drain’’, in Anti-Malware Engineering Team blog,Sep20

2007, at http://blogs.technet.com/antimalware/default.aspx

[757] GD Kutz, G Aloise, JW Cooney, ‘NUCLEAR SECURITY–Actions

Taken by NRC to Strengthen Its Licensing Process for Sealed Radioactive

Sources Are Not Effective’, GAO Report GAO-07-1038T, July 12, 2007

[758] Greg L, ‘‘ID Theft, RMT and Lineage’’, Terra Nova Jul 2007, at http://

terranova.blogs.com/terra nova/2006/07/id theft rmt nc

.html#more

[759] ‘L0phtCrack 2.52 for Win95/NT’,athttp://www.l0pht.com/

l0phtcrack/

[760] J Lacy, SR Quackenbush, A Reibman, JH Snyder, ‘‘Intellectual Property

Protection Systems and Digital Watermarking’’, in Proceedings of the

Second International Workshop on Information Hiding (Portland, Apr 98),

Springer LNCS v 1525 pp 158–168

[761] RJ Lackey, DW Upmal, ‘‘Speakeasy: The Military Software Radio’’, in

IEEE Communications Magazine v 33 no 5 (May 95) pp 56–61

950 Bibliography

[762] P Ladkin, ‘‘Flight Control System Software Anomalies’’, comp.risks

v 24 no 03, Aug 31 2005, at http://www.mail-archive.com/risks@csl

.sri.com/msg00319.html

[763] Lamarr/Antheil Patent Story Home Page, http://www.ncafe.com/

chris/pat2/index.html; contains US patent no 2,292,387 (HK Markey

et al., Aug 11 1942)

[764] G Lambourne, ‘The Fingerprint Story’, Harrap (1984) ISBN

0-245-53963-8

[765] L Lamont, ‘‘And the real Lotto winner is ...that man at the cash regis-

ter’’, Sydney Morning Herald, May 3 2007, at http://www.smh.com.au/

articles/2007/05/02/1177788228072.html

[766] L Lamport, ‘‘Time, Clocks and the Reordering of Events in a Distributed

System’’, in Communications of the ACM v 21 no 7 (July 1978)

pp 558–565

[767] L Lamport, R Shostak, M Pease, ‘‘The Byzantine Generals Problem’’,

in ACM Transactions on Programming Languages and Systems v4no3

(1982) pp 382–401

[768] B Lampson, ‘‘A Note on the Conﬁnement problem’’, in Communications

of the ACM v 16 no 10 (Oct 1973) pp 613–615

[769] P Lamy,J Martinho,T Rosa, MP Queluz, ‘‘Content-Based Watermarking

for Image Authentication’’, in Proceedings of the Third International

Workshop on Information Hiding (1999), Springer LNCS v 1768

pp 187–198

[770] S Landau, S Kent, C Brooks, S Charney, D Denning, W Difﬁe, A Lauck,

D Miller, P Neumann, D Sobel, ‘‘Codes, Keys and Conﬂicts: Issues in

US Crypto Policy’’, Report of the ACM US Public Policy Committee,

June 1994

[771] M Landler, ‘‘Fine-Tuning For Privacy, Hong Kong Plans Digital ID’’,

in New York Times, Feb 19 2002; at http://www.nytimes.com/2002/

02/18/technology/18KONG.html

[772] R Landley, ‘‘Son of DIVX: DVD Copy Control’’, Motley Fool,http://

www.fool.com/portfolios/rulemaker/2000/rulemaker000127.htm

[773] P Landrock, ‘‘Roles and Responsibilities in BOLERO’’, in TEDIS EDI

trusted third parties workshop (1995), proceedings published as ISBN

84-7653-506-6, pp 125–135

Bibliography 951

[774] CE Landwehr, AR Bull, JP McDermott, WS Choi, ‘A Taxonomy of

Computer Program Security Flaws, with Examples’, US Navy Report

NRL/FR/5542-93-9591 (19/11/93)

[775] D Lane, ‘‘Where cash is king’’, in Banking Technology, Oct 92, pp 38–41

[776] J Leake, ‘‘Workers used forged passes at Sellaﬁeld’’, in Sunday Times

(2/4/2000) p 6

[777] S LeBlanc, KE Register, ‘Constant Battles: Why We Fight’, St Martin’s,

2003

[778] HC Lee, RE Guesslen (eds), ‘Advances in Fingerprint Technology’,

Elsevier (1991) ISBN 0-444-01579-5

[779] D Leigh, ‘‘Crackdown on ﬁrms stealing personal data’’, in The Guardian

Nov 15 2006; at http://www.guardian.co.uk/crime/article/0,,

1948016,00.html

[780] AK Lenstra, HW Lenstra, ‘The development of the number ﬁeld sieve’,

Springer Lecture Notes in Mathematics v 1554 (1993) ISBN

0-387-57013-6

[781] AK Lenstra, E Tromer, A Shamir, W Kortsmit, B Dodson, J Hughes,

P Leyland, ‘‘Factoring estimates for a 1024-bit RSA modulus’’, in

Asiacrypt 2003, Springer LNCS 2894 pp 331–346

[782] K Leonard, ‘‘Face Recognition Technology: Security Enhancements v.

Civil Rights’’, 2001 B.C. Intell. Prop. & Tech. F. 120301, at http://www

.bc.edu/bc org/avp/law/st org/iptf/headlines/content/

2001120301.html

[783] D Leppard, P Nuki, ‘‘BA staff sell fake duty-free goods’’, in Sunday

Times Sep 12 1999; at http://home.clara.net/brescom/Documents/

BA FAkes.htm

[784] L Lessig, ‘Code and Other Laws of Cyberspace’, Basic Books (2000); ‘Code:

Version 2.0’, Basic Books (2006); at http://www.lessig.org/

[785] L Lessig, ‘Free Culture: The Nature and Future of Creativity’, Penguin

(2005); at http://www.lessig.org/

[786] NG Leveson, ‘Safeware–System Safety and Computers’, Addison-Wesley

(1994) ISBN 0-201-11972-2

[787] S Levitt, SJ Dubner, ‘Freakonomics: A Rogue Economist Explores the

Hidden Side of Everything,’ William Morrow, 2005

952 Bibliography

[788] A Lewcock, ‘‘Bodily Power’’, in Computer Business Review v6no2(Feb

98) pp 24–27

[789] O Lewis, ‘‘Re: News: London nailbomber used the Net’’, post to

ukcrypto mailing list, 5/6/2000, archived at http://www.chiark

.greenend.org.uk/mailman/listinfo/ukcrypto

[790] Lexmark International, Inc., vs Static Control Components, Inc., US

Court of Appeals (6th Circuit), Oct 26 2004, at www.eff.org/legal/

cases/Lexmark vStatic Control/20041026 Ruling.pdf

[791] J Leyden, ‘‘Russian bookmaker hackers jailed for eight years’’, in The

russian bookmaker hackers jailed/

[792] J Leyden, ‘‘Thai police crack credit card wiretap scam’’, in The Register

Aug 4 2006, at http://www.theregister.co.uk/2006/08/04/thai

wiretap scam/

[793] J Leyden, ‘‘Hacked to the TK Maxx’’, in The Register Jan 19 2007; at

http://www.theregister.co.uk/2007/01/19/tjx hack alert/

[794] J Leyden, ‘‘Italy tops global wiretap league’’, in The Register,Mar7

2007; at http://www.theregister.co.uk/2007/03/07/wiretap

trends ss8/

[795] J Leyden, ‘‘Feds told they need warrants for webmail’’, in The Register

June 19 2007; at http://www.theregister.co.uk/2007/06/19/webmail

wiretaps appeal/

[796] J Leyden, ‘‘MySpace phishing scam targets music fans’’, in The Register,

Oct 14 2006; at http://www.theregister.co.uk/2006/10/14/

myspace phishing scam/

[797] J Leyden, ‘‘Program Names govern admin rights in Vista, in The

vista program naming oddness/

[798] CC Lin, WC Lin, ‘‘Extracting facial features by an inhibiting mechanism

based on gradient distributions’’, in Pattern Recognition v29no12(Dec

96) pp 2079–2101

[799] R Linde, ‘‘Operating Systems Penetration,’’ National Computer Confer-

ence, AFIPS (1975) pp 361–368

Bibliography 953

[800] JPMG Linnartz, ‘‘The ‘Ticket’ Concept for Copy Control Based on

Embedded Signalling’’, Fifth European Symposium on Research in Com-

puter Security (ESORICS 98), Springer LNCS 1485 pp 257–274

[801] E Linos, E Linos, G Colditz, ‘‘Screening programme evaluation applied

to airport security’’, British Medical Journal v 335, Dec 22 2007

pp 1290–1292; http://www.bmj.com/cgi/content/full/335/7633/1290

[802] J Linsky and others, ‘Bluetooth–simple pairing whitepaper’,from

www.bluetooth.com

[803] JPMG Linnartz, M van Dijk, ‘‘Analysis of the Sensitivity Attack

Against Electronic Watermarks in Images’’, in [97] pp 258–272

[804] D Litchﬁeld, C Anley, J Heasman, B Grindlay, ‘The Database Hacker’s

Handbook: Defending Database Servers’, Wiley 2005

[805] B Littlewood, ‘‘Predicting software reliability’’, in Philosophical

Transactions of the Royal Society of London A327 (1989), pp 513–527

[806] WF Lloyd, ‘Two Lectures on the Checks to Population’, Oxford Univer-

sity Press (1833)

[807] Lockheed Martin, ‘‘Covert Surveillance using Commercial Radio and

Television Signals’’, at http://silentsentry.external.lmco.com

[808] L Loeb, ‘Secure Electronic Transactions–Introduction and technical Refer-

ence’, Artech House (1998) ISBN 0-89006-992-1

[809] London School of Economics & Political Science, ‘The Identity Project–

An assessment of the UK Identity Cards Bill & its implications’, 2005, at

www.lse.ac.uk/collections/pressAndInformationOffice/PDF/

IDreport.pdf

[810] J Long, Google Hacking Database,athttp://johnny.ihackstuff.com/

ghdb.php

[811] D Longley, S Rigby, ‘‘An Automatic Search for Security Flaws in Key

Management’’, Computers & Security v 11 (March 1992) pp 75–89

[812] PA Loscocco, SD Smalley, PA Muckelbauer, RC Taylor, SJ Turner,

JF Farrell, ‘‘The Inevitability of Failure: The Flawed Assumption

of Security in Modern Computing Environments’’, in 20th National

Information Systems Security Conference, proceedings published by

NIST (1998 pp 303–314)

954 Bibliography

[813] PA Loscocco, SD Smalley, ‘‘Integrating Flexible Support for Security

Policies into the Linux Operating System’’, in Proceedings of the

FREENIX Track: 2001 USENIX Annual Technical Conference (FREENIX

’01) (June 2001). See also NSA SELinux site: http://www.nsa.gov/

selinux

[814] JR Lott, ‘More Guns, Less Crime: Understanding Crime and Gun-Control

Laws’, University of Chicago Press 2000

[815] J Loughry, DA Umphress, ‘‘Information leakage from optical emana-

tions’’, in ACM Transactions on Information and System Security v5no

3 (Aug 2002) pp 262–289

[816] WW Lowrance, ‘Privacy and Health Research’,ReporttotheUS

Secretary of Health and Human Services (May 1997)

[817] M Ludwig, ‘The Giant Black Book of Computer Viruses’, American Eagle

Publishers (1995) ISBN 0-929408-10-1

[818] J Luk`

aˇ

s, J Fridrich, M Goljan, ‘‘Digital ‘bullet scratches’ for images’’, in

ICIP 05;athttp://www.ws.binghamton.edu/fridrich/Research/

ICIP05.pdf

[819] M Lyu, ‘Software Reliability Engineering’, IEEE Computer Society Press

(1995), ISBN 0-07-039400-8

[820] D Mackett, ‘‘A Pilot on Airline Security’’, in Hot Air, July 16 2007,

at http://hotair.com/archives/2007/07/16/a-pilot-on-

airline-security/

[821] Macnn, ‘‘iPhone unlock ﬁrm threatened by AT&T’’, Aug 25 2007, at

http://www.macnn.com/articles/07/08/25/iphone.unlock.firm

.threat/

[822] B Macq, ‘‘Special Issue–Identiﬁcation and protection of Multimedia Infor-

mation’’,Proceedings of the IEEE v 87 no 7 (July 1999)

[823] W Madsen, ‘‘Airline passengers to be subject to database monitoring’’,

in Computer Fraud and Security Bulletin (Mar 97) pp 7–8

[824] W Madsen, ‘‘Crypto AG: The NSA’s Trojan Whore?’’, in Covert Action

Quarterly (Winter 1998), at http://www.mediafilter.org/caq/

cryptogate/

[825] W Madsen, ‘‘Government-Sponsored Computer Warfare and Sabo-

tage’’, in Computers and Security v 11 (1991) pp 233–236

[826] M Maes, ‘‘Twin Peaks: The Histogram Attack on Fixed Depth Image

Watermarks’’, in Proceedings of the Second International Workshop on

Information Hiding (1998), Springer LNCS v 1525 pp 290–305

Bibliography 955

[827] M Magee, ‘‘HP inkjet cartridges have built-in expiry dates–Carly’s

cunning consumable plan’’, The Inquirer, 29 April 2003, at http://

www.theinquirer.net/?article=9220

[828] K Maguire, ‘‘Muckraker who feeds off bins of the famous’’, in The

Guardian (27/7/2000), at http://www.guardianunlimited.co.uk/

Labour/Story/0,2763,347535,00.html

[829] S Maguire, Debugging the Development Process, Microsoft Press, ISBN

1-55615-650-2 p 50 (1994)

[830] F Main, ‘‘Your phone records are for sale’’, Chicago Sun-Times,Jan

5 2006, at http://blogs.law.harvard.edu/jim/2006/01/08/your-

phone-records-are-for-sale-fbi-as-reported-in-the-chicago-

sun-times/

[831] D Maio, D Maltoni, ‘‘Direct Gray-Scale Minutiae Detection in

Fingerprints’’, in IEEE Transactions on Pattern Analysis and Machine

Intelligence v 19 no 1 (Jan 97) pp 27–40

[832] D Maltoni, D Maio, AK Jain, S Prabhakar, ‘Handbook of Fingerprint

Recognition’, Springer-Verlag New York, 2003

[833] S Mangard, E Oswald, T Popp, ‘Power Analysis Attacks–Revealing the

Secrets of Smartcards’, Springer 2007

[834] T Mansﬁeld, G Kelly, D Chandler, J Kane, ‘Biometric Product Testing

Final Report, Issue 1.0, 19 March 2001, National Physical Laboratory; at

www.cesg.gov.uk/site/ast/biometrics/media/BiometricTest-

Reportpt1.pdf

[835] J Markoff, ‘What the Dormouse Said: How the 60s Counterculture Shaped

the Personal Computer’, Viking Adult (2005)

[836] L Marks, Between Silk and Cyanide–a Codemaker’s War 1941–1945,

Harper Collins (1998) ISBN 0-68486780-X

[837] L Martin, ‘‘Using Semiconductor Failure Analysis Tools for Security

Analysis’’, FIPS Physical Security Workshop, Hawaii 2005; at http://

csrc.nist.gov/groups/STM/cmvp/documents/fips140-3/physec/

papers/physecpaper11.pdf

[838] S Mason, ‘Electronic Evidence–Disclosure, Discovery and Admissibility’,

LexisNexis Butterworths (2007)

[839] M Mastanduno, ‘‘Economics and Security in Statecraft and Scholar-

ship’’, International Organization v 52 no 4 (Autumn 1998)

[840] ‘‘Reducing the Price of Convenience’’, B Masuda, International Security

Review no 82 (Autumn 93) pp 45–48

956 Bibliography

[841] JM Matey, O Naroditsky, K Hanna, R Kolczynski, DJ LoIacono,

S Mangru, M Tinker, TM Zappia, WY Zhao, ‘‘Iris on the Move:

Acquisition of Images for Iris recognition in Less Constrained

Environments’’, in Proc IEEE v 94 no 11 (Nov 2006) pp 1936–1947

[842] SA Mathieson. ‘‘Gone phishing in Halifax–UK bank sends out

marketing email which its own staff identify as a fake’’, in Infosecurity

News, Oct 7 2005, at http://www.infosecurity-magazine.com/news/

051007 halifax email.htm

[843] M Matsui, ‘‘Linear Cryptanalysis Method for DES Cipher’’, in Advances

in Cryptology — Eurocrypt 93, Springer LNCS v 765 pp 386–397

[844] M Matsui, ‘‘New Block Encryption Algorithm MISTY’’, in Fourth

International Workshop on Fast Software Encryption (1997), Springer

LNCS v 1267 pp 54–68

[845] T Matsumoto, H Matsumoto, K Yamada, S Hoshino, ‘‘Impact of

Artiﬁcial ‘Gummy’ Fingers on Fingerprint Systems’’ Proceedings of

SPIE v 4677, Optical Security and Counterfeit Deterrence Techniques

IV, 2002

[846] R Matthews, ‘‘The power of one’’, in New Scientist (10/7/1999)

pp 26–30; at http://www.newscientist.com/ns/19990710/

thepowerof.html

[847] V Maty´

as, ‘‘Protecting the identity of doctors in drug prescription

analysis’’, in Health Informatics Journal v 4 nos 3–4 (Dec 1998)

pp 205–209

[848] J Maynard Smith, G Price, ‘‘The Logic of Animal Conﬂict’’, in Nature v

146 (1973) pp 15–18

[849] D Mazi`

eres, MF Kaashoek, ‘‘The Design, Implementation and

Operation of an Email Pseudonym Server’’, in Proceedings of the

5th ACM Conference on Computer and Communications Security (1998),

http://www.pdos.lcs.mit.edu/~dm

[850] J McCormac. ‘European Scrambling Systems–The Black Book’, version 5

(1996), Waterford University Press, ISBN 1-873556-22-5

[851] D McCullagh, ‘‘U.S. to Track Crypto Trails’’, in Wired, 4/5/2000,

at http://www.wired.com/news/politics/0,1283,36067,00.html;

statistics at http://www.uscourts.gov/wiretap99/contents.html

[852] D McCullagh, R Zarate, ‘‘Scanning Tech a Blurry Picture’’, in Wired,

Feb 16 2002; at http://www.wired.com/politics/law/news/2002/02/

50470

Bibliography 957

[853] K McCurley, Remarks at IACR General Meeting. Crypto 98,Santa

Barbara, Ca., Aug 1998

[854] D McCullough, ‘‘A Hook-up Theorem for Multi-Level Security’’, in

IEEE Transactions on Software Engineering v 16 no 6 (June 1990)

pp 563–568

[855] P McDaniel, K Butler, W Enck, H Hursti, S McLaughlin, P Traynor, MA

Blaze, A Aviv, P ˇ

Cern´

y, S Clark, E Cronin, G Shah, M Sherr, A Vigna,

R Kemmerer, D Balzarotti, G Banks, M Cova, V Felmetsger, W Robert-

son, F Valeur, JL Hall, L Quilter, ‘EVEREST: Evaluation and Validation

of Election-Related Equipment, Standards and Testing’, Final Report, Dec

7, 2007; at http://www.sos.state.oh.us/sos/info/EVEREST/

14-AcademicFinalEVERESTReport.pdf

[856] AD McDonald, MG Kuhn, ‘‘StegFS: A Steganographic File System for

Linux’’, in [1022] pp 463–477

[857] D MacEoin, ‘The hijacking of British Islam–How extremist literature is

subverting mosques in the UK’, Policy Exchange (2007)

[858] G McGraw, ‘Software Security–Building Security In’, Addison-Wesley,

2006

[859] G McGraw, EW Felten, ‘Java Security’, Wiley (1997) ISBN 0-471-17842-X

[860] J McGroddy, HS Lin, ‘A Review of the FBI’s Trilogy Information Tech-

nology Modernization Program’, National Academies Press, 2004, at

http://www7 .nationalacademies.org/cstb/pub fbi.html

[861] J McHugh, ‘‘An EMACS Based Downgrader for the SAT’’ in Computer

and Network Security, IEEE Computer Society Press (1986) pp 228–237

[862] D McGrew, J Viega, ‘‘The Galois/Counter Mode of Operation (GCM)’’,

Submission to NIST Modes of Operation Process, January 2004;

updated May 2005

[863] J McLean, ‘‘The Speciﬁcation and Modeling of Computer Security’’, in

Computer v 23 no 1 (Jan 1990) pp 9–16

[864] J McLean, ‘‘Security Models,’’ in Encyclopedia of Software Engineering,

John Wiley & Sons (1994)

[865] J McLean, ‘‘A General Theory of Composition for a Class of ’Possibilis-

tic’ Properties,’’, in IEEE Transactions on Software Engineering v22no1

(Jan 1996) pp 53–67

958 Bibliography

[866] I McKie, ‘‘Total Vindication for Shirley McKie!’’ (23/6/2000), at

http://onin.com/fp/mckievindication.html

[867] I McKie, M Russell, ‘Shirley McKie–The Price of Innocence’, Birlinn,

2007; ISBN 1-84150-575-0

[868] J McMillan, ‘‘Mobile Phones Help Secure Online Banking’’, in PC

World, Sep 11 2007; at http://www.pcworld.com/printable/article/

id,137057/printable.html

[869] J McNamara, ‘‘The Complete, Unofﬁcial TEMPEST Information Page’’,

at http://www.eskimo.com/~joelm/tempest.html

[870] B McWilliams, ‘‘Sex Sites Accused of Gouging Visitors with Phone

Scam’’, in InternetNews.com (7/4/2000), at http://www

.internetnews.com/bus-news/print/0,,3 337101,00.html

[871] J Meek, ‘‘Robo Cop’’, in The Guardian, June 13 2002, at http://www

.guardian.co.uk/Archive/Article/0,4273,4432506,00.html

[872] AJ Menezes, PC van Oorschot, SA Vanstone, ‘Handbook of Applied

cryptography’, CRC Press (1997); ISBN 0-8493-8523-7; also available

online at http://www.cacr.math.uwaterloo.ca/hac/

[873] CG Menk, ‘‘System Security Engineering Capability Maturity Model

and Evaluations: Partners within the Assurance Framework’’, in 19th

National Information Systems Security Conference (1996) pp 76–88

[874] J Mercer, ‘‘Document Fraud Deterrent Strategies: Four Case Studies’’,

in Optical Security and Counterfeit Deterrence Techniques II (1998),

IS&T (The Society for Imaging Science and Technology) and SPIE

(The International Society for Optical Engineering) v 3314 ISBN

0-8194-2754-3, pp 39–51

[875] R Mercuri, ‘‘Physical Veriﬁability of Computer Systems’’, 5th Interna-

tional Computer Virus and Security Conference (March 1992)

[876] R Mercuri, ‘Electronic Vote Tabulation Checks & Balances’, PhD Thesis,

U Penn, 2000; see http://www.notablesoftware.com/evote.html

[877] TS Messergues, EA Dabish, RH Sloan, ‘‘Investigations of Power

Analysis Attacks on Smartcards’’, in Usenix Workshop on Smartcard

Technology, pp 151–161

[878] E Messmer, ‘‘DOD looks to put pizzazz back in PKI’’, Network World

Aug 15 2005; at http://www.networkworld.com/news/2005/081505-

pki.html?nl

Bibliography 959

[879] ‘‘Gamer may face jail for £1m racket’’, Metro Oct 25 2007, at http://

www.metro.co.uk/news/article.html?in article id=72887&in page

id=34; see also ‘‘Neil Higgs, aka Mr Modchips, ‘guilty’ ’’, Oct 25 2007, at

http://www.p2pnet.net/story/13780

[880] CH Meyer and SM Matyas, ‘Cryptography: A New Dimension in

Computer Data Security’, Wiley, 1982

[881] R Meyer-Sommer, ‘‘Smartly analyzing the simplicity and the power of

simple power analysis on Smartcards’’, in Workshop on Cryptographic

Hardware and Embedded Systems (2000); Springer LNCS v 1965

pp 78–92

[882] J Micklethwait, A Wooldridge, ‘The Witch Doctors–What the man-

agement gurus are saying, why it matters and how to make sense of it’,

Random House (1997) ISBN 0-7493-2645-X

[883] Microsoft Inc, ‘Architecture of Windows Media Rights Manager’,May

2004, at http://www.microsoft.com/windows/windowsmedia/

howto/articles/drmarchitecture.aspx

[884] Microsoft Inc, ‘‘Sony DRM Rootkit’’, Nov 12 2005, at http://

blogs.technet.com/antimalware/archive/2005/11/12/414299.aspx

[885] Microsoft Inc, ‘Understanding and Conﬁguring User Account Control in

Windows Vista’, Dec 2007, at http://technet2.microsoft.com/

WindowsVista/en/library/00d04415-2b2f-422c-b70e-

b18ff918c2811033.mspx

[886] A Midgley, ‘‘R.I.P. and NHSNet’’, post to ukcrypto mailing list, 1/7/

2000, archived at http://www.cs.ucl.ac.uk/staff/I.Brown/

archives/ukcrypto/

[887] S Mihm, ‘A Nation of Counterfeiters’, Harvard 2007

[888] S Milgram, ‘Obedience to Authority: An Experimental View’,

HarperCollins, (1974, reprinted 2004)

[889] J Millen, ‘‘A Resource Allocation Model for Denial of Service

Protection’’, in Journal of Computer Security v 2 no 2–3 (1993)

pp 89–106

[890] B Miller, ‘‘Vital Signs of Security’’, in IEEE Spectrum (Feb 94) pp 22–30

[891] GA Miller, ‘‘The Magical Number Seven, Plus or Minus Two: Some

Limits on our Capacity for Processing Information’’, in Psychological

Review v 63 (1956) pp 81–97

[892] ML Miller, IJ Cox, JA Bloom, ‘‘Watermarking in the Real World: An

Application to DVD’’ in Sixth ACM International Multimedia Conference

960 Bibliography

(1998); Workshop notes published by GMD–Forschungszentrum

Informationstechnik GmbH. as v 41 of GMD Report, pp 71–76

[893] JR Minkel, ‘‘Conﬁrmed: The U.S. Census Bureau Gave Up Names of

Japanese-Americans in WW II’’, in Scientiﬁc American Mar 30 2007, at

http://www.sciam.com/article.cfm?articleID=A4F4DED6-E7F2-

99DF-32E46B0AC1FDE0FE&sc=I100322

[894] SF Mires, ‘‘Production, Distribution, and Use of Postal Security Devices

and Information-Based Indicia’’, Federal Register v 65 no 191 Oct 2, 2000

pp 58682–58698; at http://www.cs.berkeley.edu/~tygar//papers/

IBIP/Production PSD.pdf

[895] KD Mitnick, Congressional testimony, as reported by AP (03/02/00);

see also http://www.zdnet.com/zdnn/stories/news/0,4586,

2454737,00.htmland http://news.cnet.com/category/0-1005-200-

1562611.html

[896] KD Mitnick, ‘The Art of Deception: Controlling the Human Element of

Security’, John Wiley and Sons (2002)

[897] Mobile Payment Forum, ‘Risks and Threats Analysis and Security Best

Practices–Mobile 2-Way Messaging Systems’ (Dec 2002), at http://

www.mobilepaymentforum.org/documents/Risk and Threats

Analysis and Security Best Practices Mobile 2Way Messaging

December 2002.pdf

[898] B Moghaddam, A Pentland, ‘‘Probabilistic Visual Learning for Object

Representation’’, in IEEE Transactions on Pattern Analysis and Machine

Intelligence v 19 no 7 (July 97) pp 696–710

[899] U M¨

oller, L Cottrell, P Palfrader, L Sassaman, ‘‘Mixmaster Protocol–

Version 2’’, IETF draft (2003) at http://www.abditum.com/mix-

master-spec.txt

[900] ‘‘Card fraud nets Esc6 billion’, F Mollet, Cards International (22/9/95)

[901] E Montegrosso, ‘‘Charging and Accounting Mechanisms’’ (3G TR

22.924 v 3.1.1), from Third Generation Partnership Project,at

http://www.3gpp.org/TSG/Oct status list.htm

[902] J Moore, ‘‘Hacking Friendster, Part 1’’, Feb 5 2004, at http://more

.theory.org/archives/000106.html; ‘‘Hacking Social Networks Part

2: Don’t Search Private Data’’, Feb 10 2004 at http://more.theory.org/

archives/000110.html

Bibliography 961

[903] SW Moore, RJ Anderson, R Mullins, G Taylor, J Fournier, ‘‘Balanced

Self-Checking Asynchronous Logic for Smart Card Applications’’, in

Microprocessors and Microsystems Journal v 27 no 9 (Oct 2003)

pp 421–430

[904] R Morris, ‘‘A Weakness in the 4.2BSD Unix TCP/IP Software’’, Bell

Labs Computer Science Technical Report no 117, February 25, 1985; at

http://www.cs.berkeley.edu/~daw/security/seq-attack.html

[905] R Morris, Invited talk, Crypto 95

[906] R Morris, K Thompson, ‘‘Password security: A case history’’, in

Communications of the ACM v 22 no 11 (November 1979) pp 594–597

[907] DP Moynihan, ‘Secrecy–The American Experience’, Yale University

Press (1999) ISBN 0-300-08079-4

[908] C Mueller, S Spray, J Grear, ‘‘The Unique Signal Concept for

Detonation Safety in Nuclear Weapons’’, Sand91-1269, UC-706.

Available via National Technical Information Service

[909] J Mueller, Overblown–How Politicians and the Terrorism Industry

Inﬂate National Security Threats, and Why we Believe Them, Simon and

Schuster 2006

[910] P Mukherjee, V Stavridou, ‘‘The Formal Speciﬁcation of Safety

Requirements for Storing Explosives’’, in Formal Aspects of Computing

v 5 no 4 (1993) pp 299–336

[911] T Mulhall, ‘‘Where Have All The Hackers Gone? A Study in Motivation,

Deterrence and Crime Displacement’’, in Computers and Security v16

no 4 (1997) pp 277–315

[912] S Mullender (ed), ‘Distributed Systems’, Addison-Wesley (1993); ISBN

0-201-62427-3

[913] SJ Murdoch, ‘‘Browser storage of passwords: a risk or opportunity?’’,

Apr 18 2006 in Light Blue Touchpaper;athttp://www

.lightbluetouchpaper.org/2006/04/18/browser-storage-of

-passwords-a-risk-or-opportunity/

[914] SJ Murdoch, ‘‘Hot or Not: Revealing Hidden Services by their Clock

Skew’’, in 13th ACM Conference on Computer and Communications

Security. 2006

[915] SJ Murdoch, ‘‘Chip & PIN relay attacks’’, at http://www

.lightbluetouchpaper.org/2007/02/06/chip-pin-relay-attacks/

962 Bibliography

[916] SJ Murdoch, ‘Covert channel vulnerabilities in anonymity systems’,

PhD Thesis, Cambridge 2007

[917] SJ Murdoch, ‘‘Embassy email accounts breached by unencrypted

passwords’’, Sep 10 2007; at http://www.lightbluetouchpaper.org/

2007/09/10/

[918] SJ Murdoch, RJ Anderson, ‘‘Shifting Borders’’, in Index on censorship

Dec 18 2007; at http://www.cl.cam.ac.uk/~sjm217/papers/index07-

borders.pdf

[919] SJ Murdoch, G Danezis, ‘‘Low-Cost Trafﬁc Analysis of Tor’’, in IEEE

Symposium on Security and Privacy (2005), at http://www.cl.cam.ac

.uk/users/sjm217/papers/oakland05torta.pdf

[920] SJ Murdoch, Piotr Zieli ´

nski, ‘‘Sampled Trafﬁc Analysis by Internet-

Exchange-Level Adversaries’’, at PET 2007; at http://www.cl.cam

.ac.uk/~sjm217/

[921] JC Murphy, D Dubbel, R Benson, ‘‘Technology Approaches to Currency

Security’’, in Optical Security and Counterfeit Deterrence Techniques II

(1998), IS&T (The Society for Imaging Science and Technology) and

SPIE (The International Society for Optical Engineering) v 3314 ISBN

0-8194-2754-3, pp 21–28

[922] K Murray, ‘‘Protection of computer programs in Ireland’’, in Computer

Law and Security Report v 12 no 3 (May/June 96) pp 57–59

[923] Major General RFH Nalder, ‘History of the Royal Corps of Signals’,

published by the Royal Signals Institution (1958)

[924] Shishir Nagaraja and Ross Anderson, ‘‘The Topology of Covert Con-

ﬂict’’, Fifth Workshop on the Economics of Information Security (2006)

[925] E Nakashima, ‘‘Verizon Says It Turned Over Data Without Court

Orders’’, in The Washington Post Oct 16 2007 p A01; at http://www

.washingtonpost.com/wp-dyn/content/article/2007/10/15/

AR2007101501857.html

[926] E Nakashima, ‘‘A Story of Surveillance–Former Technician ‘Turning In’

AT&T Over NSA Program’’, in The Washington Post Nov 7 2007 p D01;

at http://www.washingtonpost.com/wp-dyn/content/article/2007/

11/07/AR2007110700006.html

[927] E Nakashima, ‘‘FBI Prepares Vast Database Of Biometrics–$1 Billion

Project to Include Images of Irises and Faces’’, in The Washington Post

Bibliography 963

Dec 22 2007, p A01; at http://www.washingtonpost.com/wp-dyn/

content/article/2007/12/21/AR2007122102544.html

[928] A Narayanan, V Shmatikov, ‘‘How To Break Anonymity of the Netﬂix

Prize Dataset’’ (Nov 2007) at http://arxiv.org/abs/cs/0610105

[929] M Nash, ‘‘MS Security VP Mike Nash Replies’’, on Slashdot Jan 26 2006,

at http://interviews.slashdot.org/interviews/06/01/26/131246

.shtml

[930] National Audit Ofﬁce, ‘Minister of Defence: Combat Identiﬁcation’, 2002;

at www.nao.gov.uk/publications/nao reports/01-02/0102661.pdf

[931] Wikipedia, Napster,http://en.wikipedia.org/wiki/Napster

[932] M Nash, R Kennett, ‘‘Implementing Security policy in a Large

Defence Procurement’’ in 12th Annual Computer Security Applications

Conference, San Diego CA, December 9–13 1996; proceedings published

by the IEEE, ISBN 0-8186-7606-X; pp 15–23

[933] National Information Infrastructure Task Force, ‘Options for Promoting

Privacy on the National Information Infrastructure’ (April 1997), at

http://www.iitf.nist.gov/ipc/privacy.htm

[934] National Institute of Standards and Technology, archive of publications

on computer security, http://csrc.nist.gov/publications/

history/index.html

[935] National Institute of Standards and Technology, ‘Common Criteria for

Information Technology Security Evaluation’, Version 2.0 / ISO IS 15408

(May 1998); Version 3.1 (Sep 2006–Sep 2007), at http://www

.commoncriteriaportal.org

[936] National Institute of Standards and Technology, ‘Data Encryption Stan-

dard (DES)’ FIPS 46-3, Nov 1999 incorporating upgrade to triple DES, at

http://csrc.nist.gov/publications/fips/fips46-3/fips46-3.pdf

[937] National Institute of Standards and Technology, ‘Escrowed Encryption

Standard’, FIPS 185, Feb 1994

[938] National Institute of Standards and Technology, ‘Security Requirements

for Cryptographic Modules’ (11/1/1994), at http://www.itl.nist.gov/

fipspubs/0-toc.htm#cs

[939] National Institute of Standards and Technology, ‘SKIPJACK and KEA

Algorithms’, 23/6/98, http://csrc.nist.gov/encryption/skipjack-

kea.htm

964 Bibliography

[940] National Institute of Standards and Technology, ‘Advanced Encryption

Standard’, FIPS 197, Nov 26, 2001

[941] National Institute of Standards and Technology, ‘Digital Signature

Standard (DSS)’, FIPS 186-2, Jan 2000, with change notice Oct 2001

[942] National Institute of Standards and Technology, ‘Digital Signature

Standard (DSS)’, FIPS 186-3, draft, Mar 2006

[943] National Institute of Standards and Technology, ‘PBX Vulnerability

Analysis–Finding Holes in Your PBX Before Somebody Else Does’,Spe-

cial Publication 800-24, at http://csrc.nist.gov/publications/

PubsSPs.html

[944] National Institute of Standards and Technology, ‘Recommendation

for Block Cipher Modes of Operation’, Special Publication 800-38A 2001

Edition, at http://csrc.nist.gov/CryptoToolkit/modes/

[945] National Institute of Standards and Technology, ‘Recommendation for

Block Cipher Modes of Operation: The CMAC Mode for Authentication’,

Special Publication 800-38B, May 2005

[946] National Institute of Standards and Technology, ‘Recommendation for

Block Cipher Modes of Operation: The CCM Mode for Authentication and

Conﬁdentiality’, Special Publication 800-38C, May 2004

[947] National Institute of Standards and Technology, ‘Recommendation

for Block Cipher Modes of Operation: Galois/Counter Mode (GCM) and

GMAC’ NIST Special Publication 800-38D, November 2007

[948] National Institute of Standards and Technology, ‘Recommendation for

Key Management–Part 1: General (Revised), Special Publication 800-57,

May 2006

[949] National Institute of Standards and Technology, ‘Announcing request for

Candidate Algorithm Nominations for a New Cryptographic Hash Algo-

rithm (SHA-3) Family’,inFederal Register v 72 no 212, Nov 2 2007,

pp 62212–20

[950] National Research Council, ‘Cryptography’s Role in Securing the Infor-

mation Society’, National Academy Press (1996) ISBN 0-309-05475-3

[951] National Research Council, ‘For the Record: Protecting Electronic Health

Information’, National Academy Press (1997) ISBN 0-309-05697-7

[952] National Security Agency, ‘The NSA Security Manual’,athttp://

www.cl.cam.ac.uk/ftp/users/rja14/nsaman.tex.gz

Bibliography 965

[953] National Statistics, ‘‘Protocol on Data Access and Conﬁdentiality’’,

at http://www.statistics.gov.uk

[954] P Naur, B Randell, ‘Software Engineering–Report on a Conference’,

NATO Scientiﬁc Affairs Division, Garmisch 1968

[955] R Neame, ‘‘Managing Health Data Privacy and Security’’, in [43]

pp 225–232

[956] GC Necula, P Lee, ‘‘Safe, Untrusted Agents Using Proof-Carrying

Code’’, in Mobile Agents and Security, ISBN 3-540-64792-9, pp 61–91

[957] RM Needham, ‘‘Denial of Service: An Example’’, in Communications of

the ACM v37no11(Nov94)pp42–46

[958] RM Needham, ‘‘Naming’’, in [912], pp 318–127

[959] RM Needham, ‘‘The Hardware Environment’’, in Proceedings of the

1999 IEEE Symposium on Security and Privacy, IEEE Computer Society

Press, p 236

[960] RM Needham, MD Schroeder, ‘‘Using Encryption for Authentication

in Large Networks of Computers’’, in Communications of the ACM v21

no 12 (Dec 78) pp 993–999

[961] A Neewitz, ‘‘Defenses lacking at social network sites’’, Security Focus

Dec 31 2003, at http://www.securityfocus.com/news/7739

[962] P Neumann, ‘Computer Related Risks’, Addison-Wesley (1995); ISBN

0-201-55805-X

[963] P Neumann, Principled Assuredly Trustworthy Composable Architectures,

CHATS Project ﬁnal report (2004), at http://www.csl.sri.com/users/

neumann/

[964] New South Wales Supreme Court, ‘‘RTA v. Michell (New South Wales

Supreme Court, 3/24/2006)’’, reported in http://www.thenewspaper

.com/news/10/1037.asp

[965] MEJ Newman, ‘‘The structure and function of complex networks’’, in

SIAM Review v 45 no 2 (2003) pp 167–256

[966] MEJ Newman, ‘‘Modularity and community structure in networks’’, in

Proc. Natl. Acad. Sci. USA v 103 pp 8577–8582 (2006); at http://arxiv

.org/abs/physics/0602124

[967] Richard Newman, Sherman Gavette, Larry Yonge, RJ Anderson,

‘‘Protecting Domestic Power-line Communications’’, in Symposium On

Usable Privacy and Security 2006 pp 122–132

966 Bibliography

[968] O Newman, ‘Defensible Space: People and Design in the Violent City’,

MacMillan 1972

[969] J Newton, ‘‘Countering the counterfeiters’’, in Cards International

(21/12/94) p 12

[970] J Newton, ‘Organised Plastic Counterfeiting’, Her Majesty’s Stationery

Ofﬁce (1996), ISBN 0-11-341128-6

[971] N Nisan, T Roughgarden, E Tardos, VV Vazirani, ‘Algorithmic

Mechanism Design’, CUP 2007

[972] DA Norman, ‘‘Cautious Cars and Cantankerous Kitchens: How

Machines Take Control’’, at http://www.jnd.org/; chapter 1 of The

Design of Future Things (due 2008)

[973] R Norton-Taylor ‘‘Titan Rain–how Chinese hackers targeted White-

hall’’, in The Guardian, Sep 5 2007 p 1; at http://www.guardian.co.uk/

technology/2007/sep/04/news.internet

[974] R v Ipswich Crown Court ex parte NTL Ltd, [2002] EWHC 1585

(Admin), at http://www.cyber-rights.org/documents/ntl case.htm

[975] R v Paul Matthew Stubbs, [2006] EWCA Crim 2312 (12 October 2006), at

http://www.bailii.org/cgi-bin/markup.cgi?doc=/ew/cases/EWCA/

Crim/2006/2312.html

[976] Nuclear Regulatory Commission, www.nrc.gov

[977] H Nugent, ‘‘Adulterers who call 118 118 for an affair’’, in The Times,

May 27 2006; at http://www.timesonline.co.uk/article/0,,2-

2198924.html

[978] F Oberholzer, K Strumpf, ‘‘The Effect of File Sharing on Record

Sales–An Empirical Analysis’’, June 2004; journal version F Oberholzer-

Gee, K Strumpf, ‘‘The Effect of File Sharing on Record Sales: An

Empirical Analysis’’, Journal of Political Economy v 115 (2007) pp 1–42

[979] AM Odlyzko, ‘The history of communications and its implications for the

Internet’,athttp://www.dtc.umn.edu/~odlyzko/doc/networks.html

[980] AM Odlyzko, ‘‘Smart and stupid networks: Why the Internet is like

Microsoft’’, ACM netWorker, Dec 1998, pp 38–46, at http://www

.acm.org/networker/issue/9805/ssnet.html

[981] AM Odlyzko, ‘‘Privacy, economics, and price discrimination on the

Internet’’, in ICEC ’03: Proceedings of the 5th international conference on

Bibliography 967

electronic commerce, pp 355–366; at http://www.dtc.umn.edu/

~odlyzko/doc/networks.html

[982] AM Odlyzko, ‘‘Pricing and Architecture of the Internet: Historical

Perspectives from Telecommunications and Transportation’’, TPRC

2004,athttp://www.dtc.umn.edu/~odlyzko/doc/networks.html

[983] N Okuntsev, ‘Windows NT Security’, R&D Books (1999); ISBN

0-87930-473-1

[984] Open Net Initiative, ‘Internet Filtering in China in 2004-2005: A Coun-

try Study’, April 14, 2005, at www.opennetinitiative.net

[985] Open Net Initiative, ‘China (including Hong Kong)’, Country report

2006, at www.opennetinitiative.net

[986] Open Net Initiative, ‘Pulling the Plug’, Oct 2007, at www.opennetinitia-

tive.net

[987] Open Rights Group, ‘May 2007 Election Report–Findings of the Open

Rights Group Election Observation Mission in Scotland and England’,at

http://www.openrightsgroup.org/e-voting-main

[988] R Oppliger, ‘Internet and Intranet Security’, Artech House (1998) ISBN

0-89006-829-1

[989] Oracle Inc., ‘Unbreakable: Oracle’s Commitment to Security’, Oracle

White Paper, Feb 2002, at http://www.oracle.com/technology/

deploy/security/pdf/unbreak3.pdf

[990] M Orozco, Y Asfaw, A Adler, S Shirmohammadi, A El Saddik,

‘‘Automatic Identiﬁcation of Participants in Haptic Systems’’, in 2005

IEEE Instrumentation and Measurement Technology Conference, Ottawa,

pp 888–892, at http://www.sce.carleton.ca/faculty/adler/

publications/publications.html

[991] Organization for Economic Cooperation and Development, ‘Guidelines

for the Protections of Privacy and Transborder Flow of Personal Data’,

OECD Doc no C(80)58 (1981), at http://www.oecd.org//dsti/sti/it/

secur/prod/PRIV-EN.HTM

[992] J Osen, ‘‘The Cream of Other Men’s Wit: Plagiarism and Misappropri-

ation in Cyberspace’’, in Computer Fraud and Security Bulletin (11/97)

pp 13–19

[993] M Ossman, ‘‘WEP: Dead Again’’, in Security Focus: Part 1, Dec 14 2004, at

http://www.securityfocus.com/infocus/1814, and part 2, Mar 8 2005,

at http://www.securityfocus.com/infocus/1824

968 Bibliography

[994] DA Osvik, A Shamir, E Tromer, ‘‘Cache attacks and countermeasures:

the case of AES,’’ in RSA Conference Cryptographers Track 2006, LNCS

3860, pp 1–20

[995] Out-law News, ‘‘SWIFT broke data protection law, says Working

Party’’, Nov 27 2006, at http://www.out-law.com/page-7518

[996] Out-law News, ‘‘SWIFT will stop some US processing in 2009’’,

Oct 15 2007, at http://www.out-law.com/page-8548;

[997] A Ozment, S Schechter, ‘‘Bootstrapping the Adoption of Internet

Security Protocols’’, at Fifth Workshop on the Economics of Information

Security Security, 2006; at http://www.cl.cam.ac.uk/~jo262/

[998] A Ozment, S Schechter, ‘‘Milk or Wine: Does Software Security

Improve with Age?’’ in 15th Usenix Security Symposium (2006)

[999] D Page, ‘Theoretical Use of Cache Memory as a Cryptanalytic Side-

Channel’, Technical Report CSTR-02-003, University of Bristol,

June 2002

[1000] L Page, ‘‘Thai insurgents move to keyless-entry bombs’’, in The Register

Apr 27 2007, at http://www.theregister.co.uk/2007/04/27/ied ew

carries on

[1001] G Pahl, W Beitz, Konstruktionslehre’; translated as ‘Engineering Design:

A Systematic Approach’, Springer 1999

[1002] S Pancho, ‘‘Paradigm shifts in protocol analysis’’, in Proceedings of the

1999 New Security Paradigms Workshop, ACM (2000), pp 70–79

[1003] A Papadimoulis, ‘‘Wish-It-Was Two-Factor’’, Sep 20 2007, at http://

worsethanfailure.com/Articles/WishItWas-TwoFactor-.aspx

[1004] DJ Parker, ‘‘DVD Copy Protection: An Agreement At Last?–Protecting

Intellectual Property Rights In The Age Of Technology’’, in Tape/

Disc Magazine (Oct 96) http://www.kipinet.com/tdb/tdb oct96/

feat protection.html

[1005] DJ Parker, ‘Fighting Computer crime–A New Framework for Protecting

Information’, Wiley (1998) ISBN 0-471-16378-3

[1006] A Pasick, ‘‘FBI checks gambling in Second Life virtual world’’, Reuters,

Apr 4 2007, at http://www.reuters.com/article/technologyNews/

idUSN0327865820070404?feedType=RSS

[1007] J Pastor,‘‘CRYPTOPOST–A cryptographic application to mail process-

ing’’, in Journal of Cryptology v 3 no 2 (Jan 1991) pp 137–146

Bibliography 969

[1008] B Patterson, letter to Communications of the ACM v 43 no 4 (Apr 2000)

pp 11–12

[1009] R Paul, ‘‘Leaked Media Defender e-mails reveal secret government

project’’, Ars Technica Sep 16 2007, at http://arstechnica.com/

news.ars/post/20070916-leaked-media-defender-e-mails-reveal-

secret-government-project.html

[1010] LC Paulson, ‘‘Inductive analysis of the Internet protocol TLS’’, in

ACM Transactions on Computer and System Security v 2 no 3 (1999)

pp 332–351; also at http://www.cl.cam.ac.uk/users/lcp/papers/

protocols.html

[1011] V Paxson, ‘‘An Analysis of Using Reﬂectors for Distributed Denial-

of-Service Attacks’’, in Computer Communication Review v31no3,July

2001, at http://www.icir.org/vern/

[1012] B Pease, A Pease, ‘ Why Men Don’t Listen and Women Can’t Read Maps:

How We’re Different and What to Do about It’, Broadway Books 2001

[1013] TP Pedersen, ‘‘Electronic Payments of Small Amounts’’, in Security

Protocols (1996), Springer LNCS v 1189 pp 59–68

[1014] J Pereira, ‘‘Breaking the Code: How Credit-Card Data Went Out

Wireless Door’’, in The Wall Street Journal, May 4 2007, p A1

[1015] A Perrig, ‘A Copyright Protection Environment for Digital Images’,

Diploma thesis, ´

Ecole Polytechnique F´

ed´

erale de Lausanne (1997)

[1016] P Pesic, ‘‘The Clue to the Labyrinth: Francis Bacon and the Decryption

of Nature’’, in Cryptologia v XXIV no 3 (July 2000) pp 193–211

[1017] M Peters, ‘‘MTN moves to prevent SIM card swap fraud’’, IOL,Dec30

2007, at http://www.iol.co.za/index.php?set id=1&click id=79&

art id=vn20071230080257431C811594&newslett=1&em=

169205a1a20080102ah

[1018] I Peterson, ‘‘From Counting to Writing’’, MathLand Archives,

http://www.maa.org/mathland/mathland 224.html

[1019] FAP Petitcolas, RJ Anderson, MG Kuhn, ‘‘Attacks on Copyright

Marking Systems’’, in Proceedings of the Second International Workshop

on Information Hiding (1998), Springer LNCS v 1525 pp 219–239

[1020] FAP Petitcolas, RJ Anderson, MG Kuhn, ‘‘Information Hiding–A

Survey’’, in Proceedings of the IEEE v 87 no 7 (July 1999) pp 1062–1078

970 Bibliography

[1021] H Petroski, ‘To Engineer is Human’, Barnes and Noble Books (1994)

ISBN 1-56619502-0

[1022] A Pﬁtzmann, Proceedings of the Third International Workshop on Infor-

mation Hiding (1999), Springer LNCS v 1768

[1023] B Pﬁtzmann, ‘‘Information Hiding Terminology’’, in Proceedings of the

First International Workshop on Information Hiding (1996), Springer

LNCS v 1174 pp 347–350

[1024] Z Phillips, ‘‘Security Theater’’, in Government Executive Aug 1, 2007, at

http://www.govexec.com/features/0807-01/0807-01s3.htm

[1025] GE Pickett, ‘‘How do you select the ‘right’ security feature(s) for your

company’s products?’’, in Optical Security and Counterfeit Deterrence

Techniques II (1998), IS&T (The Society for Imaging Science and Tech-

nology) and SPIE (The International Society for Optical Engineering) v

3314, ISBN 0-8194-2754-3, pp 52–58

[1026] RL Pickholtz, DL Schilling, LB Milstein, ‘‘Theory of Spread Spectrum

Communications–A Tutorial’’, in IEEE Transactions on Communications

v TC-30 no 5 (May 1982) pp 855–884

[1027] RL Pickholtz, DB Newman, YQ Zhang, M Tatebayashi, ‘‘Security

Analysis of the INTELSAT VI and VII Command Network’’, in IEEE

Proceedings on Selected Areas in Communications v 11 no 5 (June 1993)

pp 663–672

[1028] L Pinault, ‘Consulting Demons’, Collins 2000

[1029] RA Poisel, ‘Modern Communications Jamming Principles and Techniques’,

Artech House 2003; ISBN 158053743X

[1030] D Polak, ‘‘GSM mobile network in Switzerland reveals location of its

users’’, in Privacy Forum Digest v 6 no 18 (31/12/1997), at http://

www.vortex.com/privacy/priv.06.18

[1031] Politech mailing list, at http://www.politechbot.com/

[1032] B Pomeroy, S Wiseman, ‘‘Private Desktops and Shared Store’’, in

Computer Security Applications Conference, Phoenix, Arizona, (1998);

proceedings published by the IEEE, ISBN 0-8186-8789-4, pp190–200

[1033] GJ Popek, RP Goldberg, ‘‘Formal Requirements for Virtualizable Third

Generation Architectures’’, in Communications of the ACM v17no7

(July 1974) pp 412–421

Bibliography 971

[1034] B Poser, ‘‘The Provenzano Code’’, in Language Log, Apr 21, 2006; at

http://itre.cis.upenn.edu/~myl/languagelog/archives/003049

.html

[1035] Richard Posner, ‘‘An Economic Theory of Privacy’’, in Regulation

(1978) pp 19–26

[1036] Richard Posner, ‘‘Privacy, Secrecy and Reputation’’ in Buffalo Law

Review v 28 no 1 (1979)

[1037] K Poulsen, ‘‘ATM Reprogramming Caper Hits Pennsylvania’’, in

Wired, July 12 2007, at http://blog.wired.com/27bstroke6/2007/07/

atm-reprogrammi.html

[1038] S Poulter, ‘‘Phone ﬁrm’s whistleblower says his life has been made a

misery’’, in The Daily Mail Jun 21 2007; at http://www.dailymail.co

.uk/pages/live/articles/news/news.html?in article id=

463593&in page id=1770

[1039] J Preece, H Sharp, Y Rogers, ‘Interaction design: beyond human-computer

interaction’, Wiley (2002)

[1040] B Preneel, PC van Oorschot, ‘‘MDx-MAC and Building Fast MACs

from Hash Functions’’, in Advances in Cryptology–Crypto 95, Springer

LNCS v 963 pp 1–14

[1041] RS Pressman, ‘Software Engineering: A Practitioner’s Approach’,

McGraw-Hill (5th edition, 2000) ISBN 0-073-65578-3

[1042] V Prevelakis, D Spinellis, ‘‘The Athens Affair’’, IEEE Spectrum,

July 2007, at http://www.spectrum.ieee.org/print/5280

[1043] G Price, ‘The Interaction Between Fault Tolerance and Security’, Technical

Report no 214, Cambridge University Computer Laboratory

[1044] WR Price, ‘‘Issues to Consider When Using Evaluated Products to

Implement Secure Mission Systems’’, in Proceedings of the 15th National

Computer Security Conference, National Institute of Standards and

Technology (1992) pp 292–299

[1045] H Pringle, ‘‘The Cradle of Cash’’, in Discover v 19 no 10 (Oct 1998);

http://www.discover.com/oct issue/cradle.html

[1046] C Prins, ‘‘Biometric Technology Law’’, in The Computer Law and

Security Report v 14 no 3 (May/Jun 98) pp 159–165

[1047] Privacy Commissioner of Canada, ‘‘Inadequate security safeguards led

to TJX breach, Commissioners say’’, Sep 25 2007, at http://www

.privcom.gc.ca/media/nr-c/2007/nr-c 070925 e.asp

972 Bibliography

[1048] The Privacy Exchange, http://www.privacyexchange.org/

[1049] A Pruneda, ‘‘Windows Media Technologies: Using Windows Media

Rights Manager to Protect and Distribute Digital Media’’, MSDN

Magazine, Dec 2001, at http://msdn.microsoft.com/msdnmag/issues/

01/12/DRM/

[1050] Public Lending Right (PLR), at http://www.writers.org.uk/guild/

Crafts/Books/PLRBody.html

[1051] Public Record Ofﬁce, ‘Functional Requirements for Electronic Record

Management Systems’, November 1999, http://www.pro.gov.uk/

recordsmanagement/eros/invest/reference.pdf

[1052] RD Putnam, ‘Bowling Alone: the Collapse and Revival of American Com-

munity’, Simon & Schuster, 2000

[1053] T Pyszczynski, S Solomon, J Greenberg, ‘In the Wake of 9/11–the

Psychology of Terror’, American Psychological Association 2003

[1054] JJ Quisquater, D Samyde, ‘‘ElectroMagnetic Analysis (EMA): Measures

and Counter-Measures for Smart Cards’’ in International Conference on

Research in Smart Cards, Springer LNCS v 2140 pp 200–210

[1055] Rain Forest Puppy, ‘‘Issue disclosure policy v1.1’’, at http://www

.wiretrip.net/rfp/policy.html

[1056] W Rankl, W Efﬁng, ‘Smartcard Handbook’, Wiley (1997), ISBN 0-471-

96720-3; translated from the German ‘Handbuch der Chpkarten’, Carl

Hanser Verlag (1995), ISBN 3-446-17993-3

[1057] ES Raymond, ‘‘The Case of the Quake Cheats’’, 27/12/1999, at http://

www.tuxedo.org/~esr/writings/quake-cheats.html

[1058] ES Raymond, ‘The Cathedral and the Bazaar’,athttp://www.tuxedo

.org/~esr/writings/cathedral-bazaar/

[1059] ES Raymond, ‘The Magic Cauldron’, June 1999, at http://www.tuxedo

.org/~esr/writings/magic-cauldron/magic-cauldron.html

[1060] J Reason, ‘Human Error’, Cambridge University Press 1990

[1061] SM Redl, MK Weber, MW Oliphant, ‘GSM and Personal Communications

Handbook’, Artech House (1998) ISBN 0-89006-957-3

[1062] MG Reed, PF Syverson, DM Goldschlag, ‘‘Anonymous Connections and

Onion Routing’’, in IEEE Journal on Special Areas in Communications

v 16 no 4 (May 98) pp 482–494

Bibliography 973

[1063] T Reid, ‘‘China’s cyber army is preparing to march on America, says

Pentagon’’, in The Times Sep 7 2007; at http://technology

.timesonline.co.uk/tol/news/tech and web/the web/article

2409865.ece

[1064] ‘‘Mystery of Levy tax phone calls’’, C Reiss, Evening Standard

July 5 2000 p 1; also at http://www.thisislondon.com/

[1065] MK Reiter, ‘‘A Secure Group Membership Protocol’’, IEEE Transactions

on Software Engineering v22no1(Jan96)pp31–42

[1066] MK Reiter, MK Franklin, JB Lacy, RA Wright, ‘‘The Omega Key Manage-

ment Service’’, 3rd ACM Conference on Computer and Communications

Security (1996) pp 38–47

[1067] M Reiter, AD Rubin, ‘‘Anonymous web transactions with Crowds’’, in

Communications of the ACM v 42 no 2 (Feb 99) pp 32–38

[1068] J Reno, http://www.cnn.com/2000/US/05/25/security.breaches.01/

index.html

[1069] Reporters without Borders, ‘Handbook for Bloggers and Cyber-dissidents’,

2005, at http://www.rsf.org/rubrique.php3?id rubrique=542

[1070] E Rescorla, ‘SSL and TLS–Designing and Building Secure Systems’,

Addison-Wesley 2000

[1071] E Rescorla, ‘‘Is Finding Security Holes a Good Idea?’’, Third Workshop

on the Economics of Information Security (2004)

[1072] Reuters, ‘‘No Surveillance Tech for Tampa’’, in Wired Aug 21 2003, at

http://www.wired.com/politics/law/news/2003/08/60140

[1073] Reuters, ‘‘Nissan warns U.S. cellphones can disable car keys’’,

May 24 2007, at http://www.reuters.com/article/technologyNews/

idUSN2424455020070524?feedType=RSS&rpc=22

[1074] D Richardson, ‘Techniques and Equipment of Electronic Warfare’,

Salamander Books, ISBN 0-8601-265-8

[1075] LW Ricketts, JE Bridges, J Miletta, ‘EMP Radiation and Protection

Techniques’, Wiley 1975

[1076] M Ridley, ‘The Red Queen: Sex and the Evolution of Human Nature’,

Viking Books (1993); ISBN 0-1402-4548-0

[1077] RL Rivest, A Shamir, ‘‘PayWord and MicroMint: Two Simple Micro-

payment Schemes’’, in Security Protocols (1996), Springer LNCS v 1189

pp 69–87

974 Bibliography

[1078] RL Rivest, A Shamir, L Adleman, ‘‘A Method for Obtaining Digital

Signatures and Public-Key Cryptosystems’’, in Communications of the

ACM v 21 no 2 (Feb 1978) pp 120–126

[1079] MB Robinson, ‘‘The Theoretical Development of ‘CPTED’: 25 years of

Responses to C. Ray Jeffery’’, in Advances in Criminological Theory v8;

at http://www.acs.appstate.edu/dept/ps-cj/vitacpted2.html

[1080] AR Roddy, JD Stosz, ‘‘Fingerprint Features — Statistical Analysis and

System Performance Estimates’’, in Proceedings of the IEEE v85no9

(Sep 97) pp 1390–1421

[1081] R Rohozinski, M Mambetalieva, ‘‘Election Monitoring in Kyrgyzstan’’,

2005, Open Net Initiative,athttp://opennet.net/special/kg/

[1082] SJ Root, ‘Beyond COSO–Internal Control to Enhance Corporate Gover-

nance’, Wiley 1998

[1083] N Rosasco, D Larochelle, ‘‘How and Why More Secure Technologies

Succeed in Legacy Markets: Lessons from the Success of SSH’’, in WEIS

2003;athttp://www.cl.cam.ac.uk/~rja14/econsec.html

[1084] J Rosen, ‘‘A Watchful State’’, in New York Times, Oct 7 2001 p 38

[1085] B Ross, C Jackson, N Miyake, D Boneh, JC Mitchell, ‘‘Stronger Password

Authentication Using Browser Extensions’’, in Proceedings of the 14th

Usenix Security Symposium, 2005;athttp://crypto.stanford.edu/

PwdHash/

[1086] DE Ross, ‘‘Two Signatures’’, in comp.risks v 20.81: http://catless

.ncl.ac.uk/Risks/20.81.html

[1087] ‘‘Card fraud plummets in France’’, M Rowe, Banking Technology (May

94) p 10

[1088] T Rowland, ‘‘Ringing up the wrong numbers’’, in The Guardian May 18

2006; at http://www.guardian.co.uk/media/2006/may/18/newmedia

.technology

[1089] The Royal Society, ‘Strategy options for the UK’s separated plutonium’,

Sep 27 2007, at http://royalsociety.org/document.asp?latest=

1&id=7080

[1090] WW Royce, ‘‘Managing the development of Large Software Systems:

Concepts and Techniques’’, in Proceedings IEEE WESCON (1970)

pp 1–9

Bibliography 975

[1091] A Rubin, ‘‘Bugs in Anonymity Services’’, BugTraq, 13 Apr 1999; at

http://www.securityportal.com/list-archive/bugtraq/1999/Apr/

0126.html

[1092] HH Rubinovitz, ‘‘Issues Associated with Porting Applications to the

Compartmented Mode Workstation’’, in ACM SIGSAC v12no4(Oct

94) pp 2–5

[1093] RA Rueppel, ‘Analysis and Design of Stream Ciphers’, Springer-Verlag

(1986) ISBN 0-387-16870-2

[1094] RA Rueppel, ‘‘Criticism of ISO CD 11166 Banking: Key Management by

Means of Asymmetric Algorithms’’, in Proceedings of 3rd Symposium of

State and Progress of Research in Cryptography, Fondazione Ugo Bor-

doni, Rome 1993, pp 191–198

[1095] R Rufﬁn, ‘‘Following the Flow of Funds’’ in Security Management

(July 1994) pp 46–52

[1096] J Rushby, B Randell, ‘‘A Distributed Secure System’’, in IEEE Computer

v 16 no 7 (July 83) pp 55–67

[1097] B Russell, Answer to parliamentary question, Hansard 10 Jun 2003

column 762W, at http://www.publications.parliament.uk/pa/

cm200203/cmhansrd/vo030610/text/30610w13.htm

[1098] D Russell, GT Gangemi, ‘Computer Security Basics’, Chapter 10:

TEMPEST, O’Reilly & Associates (1991), ISBN 0-937175-71-4

[1099] J Rutkowska, ‘‘ Running Vista Every Day!’’, Invisible Things Blog,Feb

2007; at http://theinvisiblethings.blogspot.com/2007/02/

running-vista-every-day.html

[1100] DR Safford, DL Schales, DK Hess, ‘‘The TAMU Security Package: An

Ongoing Response to Internet Intruders in an Academic Environ-

ment’’, in Usenix Security 93, pp 91–118

[1101] Salon, ‘‘The computer virus turns 25’’, Jul 12 2007; at http://machinist

.salon.com/blog/2007/07/12/virus birthday/index.html

[1102] JD Saltzer, MD Schroeder, ‘‘The Protection of Information in Computer

Systems’’, in Proceedings of the IEEE v 63 no 9 (Mar 1975) pp 1278–1308

[1103] RG Saltzman, ‘‘Assuring Accuracy, Integrity and Security in National

Elections: The Role of the U.S. Congress’’, in Computers, Freedom and

Privacy (1993); at http://www.cpsr.org/conferences/cfp93/saltman

.html

976 Bibliography

[1104] J Saltzman, M Daniel, ‘‘Man freed in 1997 shooting of ofﬁcer–Judge

gives ruling after ﬁngerprint revelation’’, in The Boston Globe Jan 24

2004, at http://www.truthinjustice.org/cowans2.htm

[1105] T Sammes, B Jenkinson, Forensic Computing–A Practitioner’s Guide’,

Springer (2000); ISBN 1-85233-299-9

[1106] R Samuels, S Stich, L Faucher, ‘‘Reason and Rationality’’, in Handbook

of Epistemology (Kluwer, 1999); at http://ruccs.rutgers.edu/

Archive-Folder/Research%20Group/Publications/Reason/

ReasonRationality.htm

[1107] P Samuelson, ‘‘Copyright and digital libraries’’, in Communications of

the ACM v 38 no 4, April 1995

[1108] P Samuelson, ‘‘Intellectual Property Rights and the Global Information

Economy’’, in Communications of the ACM v39no1(Jan96)pp23–28

[1109] P Samuelson, ‘‘The Copyright Grab’’, at http://uainfo.arizona.edu/

~weisband/411 511/copyright.html

[1110] Pam Samuelson and Suzanne Scotchmer, ‘‘The Law and Economics of

Reverse Engineering’’, Yale Law Journal (2002)

[1111] D Samyde, SP Skorobogatov, RJ Anderson, JJ Quisquater, ‘‘On a

New Way to Read Data from Memory’’, in IEEE Security in Storage

Workshop (2002) pp 65–69

[1112] RS Sandhu, S Jajodia, ‘‘Polyinstantiation for Cover Stories’’, in Computer

Security — ESORICS 92, LNCS v 648 pp 307–328

[1113] SANS Institute, ‘‘Consensus List of The Top Ten Internet Security

Threats’’, at http://www.sans.org/, Version 1.22 June 19, 2000

[1114] G Sandoval, ‘‘Glitches let Net shoppers get free goods’’, in CNET

News.com, July 5 2000; at http://news.cnet.com/news/0-1007-200-

2208733.html

[1115] PF Sass, L Gorr, ‘‘Communications for the Digitized Battleﬁeld of the

21st Century’’, in IEEE Communications v33no10(Oct95)pp86–95

[1116] W Schachtman, ‘‘How Technology Almost Lost the War: In Iraq, the

Critical Networks Are Social–Not Electronic’’, in Wired, Dec 15 2007, at

http://www.wired.com/politics/security/magazine/15-12/ff

futurewar?currentPage=all

[1117] M Schaefer, ‘‘Symbol Security Condition Considered Harmful’’, in

Proceedings of the 1989 IEEE Symposium on Security and Privacy,

pp 20–46

Bibliography 977

[1118] RR Schell, ‘‘Computer Security: The Achilles’ Heel of the Electronic Air

Force?’’, in Air University Review, v 30 no 2 (Jan–Feb 1979) pp 16–33

[1119] RR Schell, PJ Downey, GJ Popek, ‘Preliminary notes on the design of

secure military computer systems’, Electronic Systems Division, Air Force

Systems Command (1/1/1973) MCI-73-1; at http://seclab.cs

.ucdavis.edu/projects/history/papers/sche73.pdf

[1120] DL Schilling, ‘Meteor Burst Communications: Theory and Practice’, Wiley

(1993) ISBN 0-471-52212-0

[1121] DC Schleher, ‘Electronic Warfare in the Information Age’, Artech House

(1999) ISBN 0-89006-526-8

[1122] D Schmandt-Besserat, ‘How Writing Came About’, University of Texas

Press (1996): ISBN: 0-29277-704-3, http://www.dla.utexas.edu/depts/

lrc/numerals/dsb1.html

[1123] ZE Schnabel, ‘‘The estimation of the total ﬁsh population in a lake’’,in

American Mathematical Monthly v 45 (1938) pp 348–352

[1124] PM Schneider, ‘‘Datenbanken mit genetischen Merkmalen von

Straft¨

atern’’, in Datenschutz und Datensicherheit v 22 (6/1998)

pp 330–333

[1125] B Schneier, ‘Applied Cryptography’, Wiley (1996); ISBN 0-471-12845-7

[1126] B Schneier, ‘‘Why Computers are Insecure’’, in comp.risks v 20.67:

http://catless.ncl.ac.uk/Risks/20.67.html

[1127] B Schneier, ‘Secrets and Lies : Digital Security in a Networked World’,

Wiley (2000); ISBN 0-471-25311-1

[1128] B Schneier, ‘‘Semantic Attacks: The Third Wave of Network Attacks’’, in

Crypto-Gram Newsletter October 15, 2000 at http://www.schneier.com/

crypto-gram-0010.html

[1129] B Schneier, ‘Beyond Fear: Thinking Sensibly about Security in an Uncer-

tain World’, Copernicus Books (2003)

[1130] B Schneier, ‘‘Real-World Passwords’’, in Crypto-Gram Newsletter Dec

14, 2006; at http://www.schneier.com/blog/archives/2006/12/

realworld passw.html

[1131] B Schneier, ‘‘Choosing Secure Passwords’’, Aug 7 2007; at http://

www.schneier.com/blog/archives/2007/08/asking for pass.html

[1132] B Schneier, ‘‘Secure Passwords Keep You Safer, in Crypto-Gram Newslet-

ter Jan 11, 2007; at http://www.schneier.com/blog/archives/2007/

01/choosing secure.html

978 Bibliography

[1133] B Schneier, ‘‘The Psychology of Security’’, RSA Conference (2007), at

http://www.schneier.com/essay-155.html

[1134] B Schneier, ‘‘The Nugache Worm/Botnet’’, Dec 31 2007, at http://

www.schneier.com/blog/archives/2007/12/the nugache wor.html

[1135] B Schneier, D Banisar, ‘The Electronic Privacy Papers–Documents on the

Battle for Privacy in the Age of Surveillance’, Wiley (1997) ISBN

0-471-12297-1

[1136] B Schneier, A Shostack, ‘‘Breaking up is Hard to Do: Modeling

Security Threats for Smart Cards,’’ in USENIX Workshop on Smart Card

Technology 1999, pp 175–185, at http://www.schneier.com/paper-

smart-card-threats.html

[1137] M Schnyder, ‘‘Datenﬂuesse im Gesundheitswesen’’, in Symposium f¨

Datenschutz und Informationssicherheit, Zuerich, Oct 98

[1138] RA Scholtz, ‘‘Origins of Spread-Spectrum Communications’’, in IEEE

Transactions on Communications v TC-30 no 5 (May 1982) pp 822–854

[1139] MD Schroeder, ‘Cooperation of Mutually Suspicious Subsystems in a

Computer Utility’, MIT PhD Thesis, September 1972, also available as

Project MAC Technical Report MAC TR-104, available on the web as

http://hdl.handle.net/ncstrl.mit lcs/MIT/LCS/TR-104

[1140] M Scorgie, ‘‘Untapped sources for accountants’’ in Genizah

Fragments (The Newsletter of Cambridge University’s Taylor-

Schechter Genizah Research Unit) no 29 (April 1995), at

http://www.lib.cam.ac.uk/Taylor-Schechter/GF/GF29.html

[1141] Beale Screamer, ‘‘Microsoft DRM - Technical description’’ and support-

ing documents, on Cryptome.org, Oct 23 2001; at http://cryptome.org/

beale-sci-crypt.htm

[1142] W Seltzer, M Anderson, ‘‘Census Conﬁdentiality under the Second

War Powers Act (1942-1947),’’ Annual Meeting of the Population

Association of America, Mar 30 2007, New York; at Ofﬁcial Statistics

and Statistical Conﬁdentiality: Recent Writings and Essential Documents,

at http://www.uwm.edu/%7Emargo/govstat/integrity.htm

[1143] R Senderek, ‘Key-Experiments–How PGP Deals With Manipulated Keys’,

at http://senderek.de/security/key-experiments.html

[1144] D Senie, ‘‘Changing the Default for Directed Broadcasts in Routers’’,

RFC 2644, at http://www.ietf.org/rfc/rfc2644.txt

[1145] Chandak Sengoopta, ‘Imprint of the Raj’, Pan Macmillan 2004

Bibliography 979

[1146] A Shamir, ‘‘How to share a secret’’, in Communications of the ACM v22

no 11 (Nov 1979) pp 612–613

[1147] A Shamir, ‘‘Identity-based cryptosystems and signature schemes’’, in

Proceedings of Crypto 1984, Springer LNCS v 196, pp 47–53

[1148] A Shamir, ‘‘Research Announcement: Microprocessor Bugs Can Be

Security Disasters’’, Nov 2007, at http://cryptome.org/bug-

attack.htm

[1149] MI Shamos, ‘‘Electronic Voting - Evaluating the Threat’’, in Computers,

Freedom and Privacy (1993); at http://www.cpsr.org/conferences/

cfp93/shamos.html

[1150] MI Shamos, ‘‘Paper v. Electronic Voting Records–An Assessment’’, in

Computers, Freedom & Privacy (Apr 2004), at http://euro.ecom.cmu

.edu/people/faculty/mshamos/paper.htm

[1151] M Sherr, E Cronin, S Clark, M Blaze, ‘‘Signaling vulnerabilities in

wiretapping systems’’, IEEE Security and Privacy v 3 no 6 (Nov/Dec

2005) pp 13–25

[1152] O Sibert, PA Porras, R Lindell, ‘‘An Analysis of the Intel 80x86 Security

Architecture and Implementations’’ in IEEE Transactions on Software

Engineering v 22 no 5 (May 96) pp 283–293

[1153] H Simon, ‘The Sciences of the Artiﬁcial’, 3rd ed, MIT Press, 1996

[1154] Y Shachmurove, G Fishman, S Hakim, ‘‘The burglar as a rational

economic agent,’’ Technical Report CARESS Working Paper 97-07, U

Penn University of Pennsylvania Center for Analytic Research in

Economics and the Social Sciences, June 1997

[1155] G Shah, A Molina, M Blaze, ‘‘Keyboards and Covert Channels’’, in 15th

USENIX Security Symposium 2006, at http://www.crypto.com/papers/

[1156] Y Shaked, A Wool, ‘‘Cracking the Bluetooth PIN’’, 2005, at http://www

.eng.tau.ac.il/~yash/shaked-wool-mobisys05/index.html

[1157] CE Shannon, ‘‘A Mathematical Theory of Communication’’, in Bell

Systems Technical Journal v 27 (1948) pp 379–423, 623–656

[1158] CE Shannon, ‘‘Communication theory of secrecy systems’’, in Bell

Systems Technical Journal v 28 (1949) pp 656–715

[1159] C Shapiro, H Varian, ‘Information Rules’, Harvard Business School

Press (1998), ISBN 0-87584-863-X

[1160] P Shekelle, SC Morton, EB Keeler, JK Wang, BI Chaudhry, SY Wu,

WA Majica, M Maglione, EA Roth, C Rolon, D Valentine, R Shanman, SJ

980 Bibliography

Newberry, Costs and Beneﬁts of Health Information Technology, DHHS

June 2006; at http://aspe.hhs.gov/daltcp/reports/2006/HITcb.htm

[1161] M Sherr, E Cronin, S Clark, MA Blaze, ‘‘Signaling vulnerabilities in

wiretapping systems’’, IEEE Security and Privacy, Nov/Dec 2005, at

http://www.crypto.com/papers/wiretapping/

[1162] D Sherwin, ‘‘Fraud–the Unmanaged Risk’’, in Financial Crime Review

v 1 no 1 (Fall 2000) pp 67–69

[1163] S Sheye, ‘‘SSL CLient Certiﬁcates–Not Securing the Web’’, in Cryp-

tomathic NewsOnInk Quarterly Newsletter (Nov 2006), at http://www

.cryptomathic.com/Admin/Public/DWSDownload.aspx?File=%

2fFiles%2fFiler%2fNewsletters%2fNewsOnInk Nov 2006.pdf

[1164] JF Shoch, JA Hupp, ‘‘The ‘Worm’ Programs–Early Experience

with a Distributed Computation’’, Comm ACM v 25 no 3 (1982)

pp 172–180

[1165] PW Shor, ‘‘Algorithms for Quantum Computers’’, in 35th Annual

Symposium on the Foundations of Computer Science (1994), proceedings

published by the IEEE, ISBN 0-8186-6580-7, pp 124–134

[1166] A Shostack, P Syverson, ‘‘What Price Privacy? (and why identity

theft is about neither identity nor theft)’’, in Economics of Information

Security, Kluwer Academic Publishers, 2004, Chapter 11

[1167] V Shoup, ‘‘OAEP Reconsidered’’, IBM Z ¨

urich, Switzerland, September

18, 2001; at http://www.shoup.net/papers/oaep.pdf

[1168] Luther Simjian–Inventor of the Week,athttp://web.mit.edu/invent/

iow/simjian.html

[1169] GJ Simmons, ‘‘The Prisoners’ Problem and the Subliminal Channel’’,

in Proceedings of CRYPTO ’83, Plenum Press (1984) pp 51–67

[1170] GJ Simmons, ‘‘How to Insure that Data Acquired to Verify Treaty

Compliance are Trustworthy’’, GJ Simmons, Proceedings of the IEEE v

76 no 5 (1988; reprinted as a chapter in [1171])

[1171] GJ Simmons (ed), ‘Contemporary Cryptology–The Science of

Information Integrity’, IEEE Press (1992) ISBN 0-87942-277-7

[1172] GJ Simmons, ‘‘A Survey of Information Authentication’’, in [1171]

pp 379–439

[1173] GJ Simmons, ‘‘An Introduction to Shared Secret and/or Shared Control

Schemes and Their Application’’, in [1171] pp 441–497

Bibliography 981

[1174] GJ Simmons, invited talk at the 1993 ACM Conference on Computer and

Communications Security, Fairfax, Virginia, Nov 3–5, 1993

[1175] GJ Simmons, ‘Subliminal Channels; Past and Present’, European

Transactions on Telecommunications v 5 no 4 (Jul/Aug 94) pp 459–473

[1176] GJ Simmons, ‘‘The History of Subliminal Channels’’, in IEEE Journal

on Selcted Areas in Communications v 16 no 4 (April 1998) pp 452–462

[1177] R Singel, ‘‘Encrypted E-Mail Company Hushmail Spills to Feds’’, in

Wired Nov 7 2007 at http://blog.wired.com/27bstroke6/2007/11/

encrypted-e-mai.html

[1178] R Singel, ‘‘Point, Click ... Eavesdrop: How the FBI Wiretap Net

Operates’’, in Wired Aug 29 2007 at http://www.wired.com/politics/

security/news/2007/08/wiretap

[1179] A Sipress, ‘‘Tracking Trafﬁc by Cell Phone; Md., Va. to Use Transmis-

sions to Pinpoint Congestion’’, in Washington Post (22/12/1999) p A01,

at http://www.washingtonpost.com/

[1180] KS Siyan, J Casad, J Millecan, D Yarashus, P Tso, J Shoults, ‘Windows

NT Server 4–Professional Reference’, New Riders Publishing (1996)

[1181] SP Skorobogatov, ‘‘Copy Protection in Modern Microcontrollers’’, at

http://www.cl.cam.ac.uk/~sps32/mcu lock.html

[1182] SP Skorobogatov, ‘Low temperature data remanence in static RAM’,

Cambridge University Technical Report UCAM-CL-TR-536 (June 2002),

at http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-536.html

[1183] SP Skorobogatov, ‘Semi-invasive attacks–A new approach to hardware

security analysis’, PhD Thesis, 2004; University of Cambridge Technical

Report 630, 2005; at http://www.cl.cam.ac.uk/techreports/UCAM-

CL-TR-630.html

[1184] SP Skorobogatov, ‘‘Data Remanence in Flash Memory Devices’’, in

Cryptographic Hardware and Embedded Systems Workshop (CHES-2005),

Springer LNCS 3659 pp 339–353

[1185] SP Skorobogatov, ‘‘Optically Enhanced Position-Locked Power

Analysis’’, in CHES 2006 pp 61–75

[1186] SP Skorobogatov, ‘‘Tamper resistance and physical attacks’’, at Summer

School on Cryptographic Hardware, Side-Channel and Fault Attacks,June

12–15, 2006, Louvain-la-Neuve, Belgium; slides at http://www.cl.cam

.ac.uk/~sps32

982 Bibliography

[1187] SP Skorobogatov, RJ Anderson, ‘‘Optical Fault Induction Attacks’’, in

Cryptographic Hardware and Embedded Systems Workshop (CHES 2002),

Springer LNCS v 2523 pp 2–12; at http://www.cl.cam.ac.uk/~sps32

[1188] B Skyrms, ‘Evolution of the Social Contract’ Cambridge University Press

(1996)

[1189] P Slovic, ML Finucane, E Peters, DG MacGregor, ‘‘Rational Actors or

Rational Fools? Implications of the Affect Heuristic for Behavioral

Economics’’, at http://www.decisionresearch.org/pdf/dr498v2.pdf;

revised version of ‘‘The Affect Heuristic’’ in Heuristics and Biases: The

Psychology of Intuitive Judgment, Cambridge University Press (2002)

pp 397–420

[1190] Smartcard Standards, http://www.cardwerk.com/smartcards/

smartcard standards.aspx

[1191] ‘‘Plastic Card Fraud Rises in the UK’’, in Smart Card News v6no3

(Mar 97) p 45

[1192] A Smith, ‘An Inquiry into the Nature and Causes of the Wealth of

Nations’, 1776; at http://www.econlib.org/LIBRARY/Smith/smWN.html

[1193] RE Smith, ‘‘Constructing a high assurance mail guard’’, in Seventeenth

National Computer Security Conference, 11–14 October, Baltimore,

Maryland; proceedings published by NIST (1994) pp 247–253

[1194] RM Smith, ‘‘Problems with Web Anonymizing Services’’ (15/4/1999),

at http://www.tiac.net/users/smiths/anon/anonprob.htm

[1195] S Smith, S Weingart, ‘Building a High-Performance, Programmable Secure

Coprocessor’, IBM Technical report RC 21102, available through http:

//www.ibm.com/security/cryptocards/

[1196] P Smulders, ‘‘The Threat of Information Theft by Reception of

Electromagnetic Radiation from RS-232 Cables’’, in Computers &

Security v 9 (1990) pp 53–58

[1197] C Soghoian, ‘‘Go Fish: Is Facebook Violating European Data Protection

Rules?’’, on Slight Paranoia June 26 2007, at http://paranoia.dubfire

.net/2007/06/go-fish-is-facebook-violating-european.html

[1198] A Solomon, ‘‘A brief history of PC virsuses’’, in Computer Fraud and

Security Bulletin (Dec 93) pp 9–19

[1199] A Solomon, Seminar given at Cambridge University Computer

Laboratory, 30th May 2000

Bibliography 983

[1200] D Solove, ‘‘A Taxonomy of Privacy’’, in University of Pennsylvania Law

Review v 154 no 3 (2006) pp 477–560; at http://papers.ssrn.com/

abstract id=667622

[1201] D Solove, ‘The future of reputation–gossip, rumor and privacy in the

Internet’, Caravan, 2007

[1202] P Sommer, ‘‘Intrusion Detection and Legal Proceedings’’, at Recent

Advances in Intrusion Detection (RAID) 1998, at http://www.zurich

.ibm.com/~dac/Prog RAID98/Full Papers/Sommer text.pdf

[1203] DX Song, D Wagner, XQ Tian, ‘‘Timing analysis of keystrokes and SSH

timing attacks,’’ in Proceedings of 10th USENIX Security Symposium

(2001)

[1204] R v Department of Health, ex parte Source Informatics: [2000] 2

WLR 940

[1205] South West Thames Regional Health Authority, ‘Report of the Inquiry

into the London Ambulance Service’ (1993), at http://www.cs.ucl.ac

.uk/staff/A.Finkelstein/las.html

[1206] E Spafford, ‘‘The Internet worm program: an analysis’’, in Computer

Communications Review v19no1(Jan89)pp17–57

[1207] EH Spafford, ‘‘OPUS: Preventing Weak Password Choices’’, in

Computers and Security v 11 no 3 (1992) pp 273–278

[1208] M Specter, ‘‘Do ﬁngerprints lie? The gold standard of forensic

evidence is now being challenged’’, New York Times, May 27, 2002; at

http://www.michaelspecter.com/ny/2002/2002 05 27 fingerprint

.html

[1209] R Spencer, S Smalley, P Loscocco, M Hibler, D Andersen, J Lepreau,

‘‘The Flask Security Architecture: System Support for Diverse Security

Policies,’’ in Proceedings of the 8th USENIX Security Symposium (1999)

pp 123–139

[1210] ‘‘Tip von Urmel’’, in Spiegel Magazine no 38 (11/9/95)

[1211] J Spolsky, ‘‘Does Issuing Passports Make Microsoft a Country?’’ at

http://joel.editthispage.com/stories/storyReader$139

[1212] ‘‘Your car radio may be revealing your tastes’’, in St Petersburg Times

(31/1/2000), at http://www.sptimes.com/News/013100/Technology/

Your car radio may be.shtml

[1213] S Stamm, Z Ramzan, M Jakobsson, ‘‘Drive-By Pharming’’, Indiana

University Department of Computer Science Technical Report

TR641, 2006

984 Bibliography

[1214] M Stamp, RM Low, ‘Applied Cryptanalysis’, Wiley 2007

[1215] T Standage, ‘The Victorian Internet’, Phoenix Press (1999), ISBN 0-75380-

703-3

[1216] D Standeford, ‘‘Case Could Signal Weakening Of Digital Rights

Management in Europe’’, in Intellectual Property Watch, June 4 2007, at

http://www.ip-watch.org/weblog/index.php?p=639&res=1600

ff&print=0

[1217] F Stajano, personal communication

[1218] F Stajano, RJ Anderson, ‘‘The Resurrecting Duckling: Security Issues

in Ad-Hoc Wireless Networks’’, in ‘Security Protocols–7th International

Workshop’, Springer LNCS 1796 pp 172–182

[1219] F Stajano, RJ Anderson, ‘‘The Cocaine Auction Protocol–On the Power

of Anonymous Broadcast’’, in [1022] pp 434–447

[1220] S Staniford, D Moore, V Paxson, N Weaver, ‘‘The Top Speed of Flash

Worms’’, in WORM04,atwww.icir.org/vern/papers/topspeed-

worm04.pdf

[1221] ‘‘Computer Chip Usage in Toner Cartridges and Impact on the

Aftermarket: Past, Current and Future’’, Static Control, Inc., formerly

at http://www.scc-inc.com/special/oemwarfare/whitepaper/

default.htm, retrieved via www.archive.org

[1222] WA Steer, ‘‘VideoDeCrypt’’, at http://www.ucl.ac.uk/~ucapwas/vdc/

[1223] P Stein, P Feaver, ‘‘Assuring Control of Nuclear Weapons’’, CSIA

occasional paper number 2, Harvard University 1987

[1224] J Steiner, BC Neuman, JI Schiller, ‘‘Kerberos: An Authentication

Service for Open Network Systems’’, in USENIX (Winter 1988); version

5in‘RFC 1510: The Kerberos Network Authentication Service (V5)’;at

http://sunsite.utk.edu/net/security/kerberos/

[1225] N Stephenson, ‘Snow Crash’, Bantam Doubleday Dell (1992), ISBN

0-553-38095-8

[1226] DR Stinson, ‘Cryptography–Theory and Practice’, CRC Press (1995);

ISBN 0-8493-8521-0

[1227] ‘Watching Them, Watching Us - UK CCTV Surveillance Regulation

Campaign’,athttp://www.spy.org.uk/

[1228] R Strehle, ‘Verschl¨

usselt–Der Fall Hans B¨

uhler’, Werd Verlag (1994)

ISBN 3-85932-141-2

Bibliography 985

[1229] R Stross, ‘‘How to Lose Your Job on Your Own Time’’, in New York

Times Dec 30 2007; at http://www.nytimes.com/2007/12/30/business/

30digi.html?ex=1356670800&en=bafd771bdcae2594&ei=5124&

partner=permalink&exprod=permalink

[1230] A Stubbleﬁeld, J Ioannidis, A Rubin, ‘‘Using the Fluhrer, Mantin, and

Shamir Attack to Break WEP’’, in ISOC 2002

[1231] K Stumper, ‘‘DNA-Analysen und ein Recht auf Nichtwissen’’, in

Datenschutz und Datensicherheit v 19 no 9 (Sep 95) pp 511–517

[1232] Suetonius (Gaius Suetonius Tranquillus), ‘Vitae XII Caesarum’, trans-

lated into English as ‘History of twelve Caesars’ by Philemon Holland,

1606; Nutt (1899)

[1233] D Sutherland, ‘‘A Model of Information’’, in 9th National Computer

Security Conference (1986)

[1234] M Sutton, ‘‘How Prevalent Are SQL Injection Vulnerabilities?’’ Michael

Sutton’s Blog, Sep 26 2006, at http://portal.spidynamics.com/blogs/

msutton/archive/2006/09/26/How-Prevalent-Are-SQL-Injection-

Vulnerabilities 3F00 .aspx

[1235] L Sweeney, ‘‘Weaving Technology and Policy Together to Maintain

Conﬁdentiality’’, in Journal of Law, Medicine and Ethics v25no2–3

(1997) pp 98–110

[1236] F Swiderski, W Snyder, ‘Threat Modeling’, Microsoft Press 2004

[1237] P Swire, ‘‘Efﬁcient Conﬁdentiality for Privacy, Security, and Conﬁden-

tial Business Information’’, Brookings-Wharton Papers on Financial

Services (2003), at http://ssrn.com/abstract=383180

[1238] P Swire, ‘‘A Theory of Disclosure for Security and Competitive Reasons:

Open Source, Proprietary Software, and Government Agencies’’, in

Houston Law Review v 42 no 5 (Jan 2006) pp 101–148; at http://ssrn

.com/abstract id=842228

[1239] Symantec, ‘Symantec Internet Security Threat Report–Trends for Jan-

uary–June 07 v 12, Sep 2007, at www.symantec.com/threatreport/

[1240] Symposium On Usable Privacy and Security,http://cups.cs.cmu.edu/

soups/2007/

[1241] C Tavris, E Aronson, ‘Mistakes were made–but not by me’, Harcourt

2007

[1242] J Taylor, MR Johnson, CG Crawford, ‘DVD Demystiﬁed’, Third edition,

McGraw-Hill 2006

986 Bibliography

[1243] J Tehranian, ‘‘An Unhurried View of Copyright Reform: Bridging the

Law/Norm Gap’’, 2007 Utah Law Review,atwww.turnergreen.com/

publications/Tehranian Infringement Nation.pdf

[1244] S Tendler, N Nuttall, ‘‘Hackers run up £1m bill on Yard’s phones’’, in The

Times, 5 Aug 1996; at http://www.the-times.co.uk/

[1245] E Tews, RP Weinmann, A Pyshkin, ‘‘Breaking 104 bit WEP in less than

60 seconds’’, Cryptology ePrint archive, Apr 2007; at http://eprint

.iacr.org/2007/120.pdf

[1246] L Thalheim, J Krissler, PM Ziegler, ‘‘Body Check–Biometric Access

Protection Devices and their Programs Put to the Test’’, c’t magazine,

Nov 2002 p 114, at http://www.heise.de/ct/english/02/11/114/

[1247] K Thompson, ‘‘Reﬂections on Trusting Trust’’, in Communications of the

ACM v 27 no 8 (Aug 84) pp 761–763; at http://www.acm.org/classics/

sep95/

[1248] R Thompson, ‘‘Google Sponsored Links Not Safe’’, Exploit Prevention

Labs Apr 24 2007, at http://explabs.blogspot.com/2007/04/google-

sponsored-links-not-safe.html; see also J Richards, ‘‘Hackers hijack

Google AdWords’’, The Times, Apr 27 2007, http://technology

.timesonline.co.uk/tol/news/tech and web/article1714656.ece

[1249] J Ticehurst, ‘‘Barclays online bank suffers another blow’’ (11/8/2000),

at http://www.vnunet.com/News/1108767

[1250] TimeWarner, ‘‘Carmine Caridi, Motion Picture Academy Member

Who Handed Over His Awards Screeners for Illegal Duplication,

Ordered to Pay $300,000 to Warner Bros. Entertainment Inc.’’, Nov 23

2004, at http://www.timewarner.com/corp/newsroom/pr/0,20812,

832500,00.html

[1251] AZ Tirkel, GA Rankin, RM van Schyndel, WJ Ho, NRA Mee,

CF Osborne, ‘‘Electronic Watermark’’, in Digital Image Computing,

Technology and Applications (DICTA 93) McQuarie University (1993)

pp 666–673

[1252] The TJX Companies, Inc., ‘Form 10-k’, ﬁled with SEC, at http://www.sec

.gov/Archives/edgar/data/109198/000095013507001906/

b64407tje10vk.htm

[1253] MW Tobias, ‘Locks, Safes and Security–An International Police Reference’

(second edition, 2000) ISBN 978-0-398-07079-3

[1254] MW Tobias, ‘‘Opening locks by bumping in ﬁve seconds or less: is it

really a threat to physical security?’’, 2006, at www.security.org

Bibliography 987

[1255] MW Tobias, ‘‘Bumping of locks–legal issues in the United States’’, at

www.security.org

[1256] MW Tobias, ‘‘The Medeco M3 Meets the Paper Clip: Is the security of

this lock at risk?’’ (2007), at www.security.org

[1257] C Tomlinson, ‘Rudimentary Treatise on the Construction of Locks’, 1853

(excerpt), at http://www.deter.com/unix/papers/treatise locks

.html

[1258] TT Tool, ‘The MIT Lock Picking Manual’, 1991; at http://people.csail

.mit.edu/custo/MITLockGuide.pdf

[1259] Transactional Records Access Clearinghouse, ‘TRACFBI’,athttp://

trac.syr.edu/tracfbi/index.html

[1260] A Travis, ‘‘Voice ID device to track failed asylum seekers’’, in The

Guardian Mar 10 2006; at http://www.guardian.co.uk/uk news/story/

0,,1727834,00.html

[1261] I Traynor, ‘‘DNA database agreed for police across EU’’, in The

Guardian, June 13 2007; at http://www.guardian.co.uk/

international/story/0,,2101496,00.html

[1262] M Trombly, ‘‘Visa issues 10 ’commandments’ for online merchants’’, in

Computerworld (11/8/2000), at http://www.computerworld.com/cwi/

story/0,1199,NAV47 STO48487,00.html

[1263] E Tromer, ‘Hardware-Based Cryptanalysis’, PhD Thesis, Weizmann

Institute of Science (2007), at http://www.wisdom.weizmann.ac

.il/~tromer/papers/tromer-phd-dissertation.pdf

[1264] C Troncoso, G Danezis, E Kosta, B Preneel, ‘‘PriPAYD: Privacy Friendly

Pay-As-You-Drive Insurance’’, in Workshop on Privacy in the Electronic

Society (2007), at https://www.cosic.esat.kuleuven.be/

publications/article-944.pdf

[1265] JD Tygar, BS Yee, N Heintze, ‘‘Cryptographic Postage Indicia’’, in

ASIAN 96 (Springer-Verlag LNCS v 1179) pp 378–391, at www.cs

.berkeley.edu/~tygar/papers/Cryptographic Postage Indicia/

CMU-CS-96-113.pdf

[1266] R Uhlig, ‘‘BT admits staff could have ﬁddled system to win Concorde

trip’’, in The Daily Telegraph (23/7/1997), at http://www.telegraph.co

.uk:80/

[1267] ukcrypto mailing list, at http://www.chiark.greenend.org.uk/

mailman/listinfo/ukcrypto

[1268] Underwriters’ Laboratories, http://www.ul.com

988 Bibliography

[1269] J Ungoed-Thomas, A Lorenz, ‘‘French play dirty for £1bn tank deal’’,

in Sunday Times (6/8/2000) p 5

[1270] United Kingdom Government, ‘e-commerce@its.best.uk’,athttp://www

.e-envoy.gov.uk/2000/strategy/strategy.htm

[1271] US Army, ‘TM 31-210 Improvised Munitions Handbook’, 1969, at http://

cryptome.org/tm-31-210.htm

[1272] ‘United States Code’–US Federal Law, online for example at http://

www4.law.cornell.edu/uscode/

[1273] United States Court of Appeals, District of Columbia Circuit, United

States Telecom Association v. Federal Communications Commission and

United States of America, no 99-1442, 15/8/2000, at http://pacer

.cadc.uscourts.gov/common/opinions/200008/99-1442a.txt

[1274] UK Passport Service, ‘Biometrics Enrolment Trial Report’, May 2005; at

www.passport.gov.uk/downloads/UKPSBiometrics Enrolment

Trial Report.pdf

[1275] UPI newswire item, Oklahoma distribution, November 26, 1983, Tulsa,

Oklahoma

[1276] NA Van House, ‘‘Flickr and Public Image-Sharing: Distant Closeness

and Photo Exhibition’’, at CHI 2007 pp 2717–2722

[1277] L van Hove, ‘‘Electronic Purses: (Which) Way to Go?’’, in First Monday v

5 no 7 (June 2000) at http://firstmonday.org/issues/issue5 7/hove/

[1278] P Van Oorschot, M Wiener, ‘‘Parallel Collision Search with Application

to Hash Functions and Discrete Logarithms’’, Second ACM Conference

on Computer and Communications Security; proceedings published by

the ACM, ISBN 0-89791-732-4, pp 210–218

[1279] R van Renesse, ‘Optical Document Security’ (second edition), Artech

House (1997) ISBN 0-89006-982-4

[1280] R van Renesse, ‘‘Verifying versus falsifying banknotes’’, in Optical

Security and Counterfeit Deterrence Techniques II (1998), IS&T (The Soci-

ety for Imaging Science and Technology) and SPIE (The International

Society for Optical Engineering) v 3314 ISBN 0-8194-2754-3, pp 71–85

[1281] H van Vliet, ‘Software Engineering–Principles and Practice’, Wiley

(second edition, 2000) ISBN 0-471-97508-7

Bibliography 989

[1282] R van Voris, ‘‘Black Box Car Idea Opens Can of Worms’’, in Law news

Network (4/6/99), at http://www.lawnewsnetwork.com/stories/

A2024-1999Jun4.html

[1283] G Vanneste, J Degraeve, ‘‘Initial report on security requirements’’,

in [92]

[1284] HR Varian, ‘Intermediate Microeconomics–A Modern Approach’ (ﬁfth

edition), Norton (1999), ISBN 0-393-97370-0

[1285] HR Varian, ‘‘Managing Online Security Risks’’, in The New York Times,

1 June 2000; at http://www.nytimes.com/library/financial/

columns/060100econ-scene.html

[1286] HR Varian, ‘‘New Chips Can Keep a Tight Rein on Customers’’, New

York Times July 4 2002, at http://www.nytimes.com/2002/07/04/

business/04SCEN.html

[1287] S Vaudenay, ‘‘FFT-Hash-II is not yet Collision-Free’’, Laboratoire d’Infor-

matique de l’Ecole Normale Sup´

erieure report LIENS-92-17

[1288] V Varadharajan, N Kumar, Y Mu, ‘‘Security Agent Based Distributed

Authorization: An Approach’, in 20th National Information Systems

Security Conference, proceedings published by NIST (1998) pp 315–328

[1289] H Varian, ‘‘Economic Aspects of Personal Privacy’’, in Privacy and

Self-Regulation in the Information Age, National Telecommunications

and Information Administration report, 1996

[1290] H Varian, ‘‘Managing Online Security Risks’’, Economic Science

Column, The New York Times, June 1, 2000

[1291] H Varian, ‘‘New chips and keep a tight rein on consumers, even after

they buy a product’’, New York Times, July 4 2002

[1292] H Varian, ‘‘System Reliability and Free Riding’’, in Economics of

Information Security, Kluwer 2004 pp 1–15

[1293] H Varian, Keynote address to the Third Digital Rights Management

Conference, Berlin, Germany, January 13, 2005

[1294] W Venema, ‘‘Murphy’s Law and Computer Security’’, in Usenix

Security 96 pp 187–193

[1295] J Vijayan, ‘‘HIPAA audit at hospital riles health care IT’’, Computer-

world, June 15 2007; at http://www.computerworld.com/action/

article.do?command=viewArticleBasic&articleId=9024921

990 Bibliography

[1296] J Vijayan, ‘‘Retail group takes a swipe at PCI, puts card companies ‘on

notice’ ’’, Computerworld Oct 4 2007; at http://computerworld.com/

action/article.do?command=viewArticleBasic&articleId=

9040958&intsrc=hm list

[1297] N Villeneuve, ‘‘DNS tampering in China’’, Jul 10 2007, at http://www

.nartv.org/2007/07/10/dns-tampering-in-china/

[1298] B Vinck, ‘‘Security Architecture’’ (3G TS 33.102 v 3.2.0), from Third Gen-

eration Partnership Project,athttp://www.3gpp.org/TSG/Oct status

list.htm

[1299] B Vinck, ‘‘Lawful Interception Requirements’’(3G TS 33.106 v 3.0.0),

from Third Generation Partnership Project,athttp://www.3gpp.org/

TSG/Oct status list.htm

[1300] VISA International, ‘Integrated Circuit Chip Card–Security Guidelines

Summary, version 2 draft 1, November 1997

[1301] A Viterbi, ‘‘Spread spectrum communications–myths and realities’’, in

IEEE Communications Magazine v 17 no 3 (May 1979) pp 11–18

[1302] PR Vizcaya, LA Gerhardt, ‘‘A Nonlinear Orientation Model for Global

Description of Fingerprints’’, in Pattern Recognition v 29 no 7 (July 96)

pp 1221–1231

[1303] L von Ahn, personal communication, 2006

[1304] L von Ahn, M Blum, NJ Hopper, J Langford, ‘‘CAPTCHA: Using Hard

AI Problems For Security’’, Advances in Cryptology–Eurocrypt 2003,

Springer LNCS v 2656 pp 294–311

[1305] D Wagner, B Schneier, J Kelsey, ‘‘Cryptanalysis of the Cellular Message

Encryption Algorithm’’, in Advances in Cryptology–Crypto 95, Springer

LNCS v 1294 pp 527–537

[1306] D Wagner, ‘‘Cryptanalysis of Some Recently-Proposed Multiple

Modes of Operation’’, in Fifth International Workshop on Fast Software

Encryption (1998), Springer LNCS v 1372 pp 254–269

[1307] D Wagner, I Goldberg, M Briceno, ‘‘GSM Cloning’’, at http://www

.isaac.cs.berkeley.edu/isaac/gsm-faq.html; see also http://www

.scard.org/gsm/

[1308] D Wagner, B Schneier, ‘‘Analysis of the SSL 3.0 Protocol’’, in Second

USENIX Workshop on Electronic Commerce (1996), pp 29–40; at http://

www.counterpane.com

Bibliography 991

[1309] M Waldman, AD Rubin, LF Cranor, ‘‘Publius: A robust, tamper-

evident, censorship-resistant, web publishing system’’, in 9th USENIX

Security Symposium (2000) pp 59–72

[1310] M Walker, ‘‘On the Security of 3GPP Networks’’, Invited talk at

Eurocrypt 2000, at http://www.ieee-security.org/Cipher/

ConfReports/2000/CR2000-Eurocrypt.html

[1311] G Walsh, ‘Review of Policy relating to Encryption Technologies’ (1996), at

http://www.efa.org.au/Issues/Crypto/Walsh/

[1312] KG Walter, WF Ogden, WC Rounds, FT Bradshaw, SR Ames, DG

Shumway, ‘Models for Secure Computer Systems’, Case Western Reserve

University, Report no 1137 (31/7/1973, revised 21/11/1973)

[1313] KG Walter, WF Ogden, WC Rounds, FT Bradshaw, SR Ames, DG

Shumway, ‘Primitive Models for Computer Security’, Case Western

Reserve University, Report no ESD–TR–74–117 (23/1/1974); at

http://www.dtic.mil

[1314] E Waltz, ‘Information Warfare–Principles and Operations’, Artech House

(1998) ISBN 0-89006-511-X

[1315] XY Wang, DG Feng, XJ Lai, HB Yu, ‘‘Collisions for Hash Functions

MD4, MD5, HAVAL-128 and RIPEMD’’, IACR Cryptology ePrint

Archive Report 2004/199, at http://eprint.iacr.org/2004/199

[1316] XY Wang, YQL Yin, HB Yu, ‘‘ Collision Search Attacks on SHA1’’, Feb

13 2005, at http://www.infosec.sdu.edu.cn/sha-1/shanote.pdf

[1317] XY Wang, HB Yu, ‘‘How to Break MD5 and Other Hash Functions’’, in

Advances in Cryptology–Eurocrypt 2005,athttp://www.infosec.sdu

.edu.cn/paper/md5-attack.pdf

[1318] R Want, A Hopper, V Falcao, J Gibbons, ‘‘The Active Badge Location

System’’, in ACM Transactions on Information Systems v 10 no 1 (Jan 92)

pp 91–102; at http://www.cl.cam.ac.uk/research/dtg/attarchive/

ab.html

[1319] W Ware, ‘Security Controls for Computer Systems: Report of Defense

Science Board Task Force on Computer Security’, Rand Report R609-1,

The RAND Corporation, Santa Monica, CA (Feb 1970), available from

http://csrc.nist.gov/publications/history/index.html

[1320] M Warner, ‘‘Machine Politics In the Digital Age’’, in The New York

Times November 9, 2003; at http://query.nytimes.com/gst/fullpage

.html?res=9804E3DC1339F93AA35752C1A9659C8B63

992 Bibliography

[1321] SD Warren, LD Brandeis, ‘‘The Right To Privacy’’ Harvard Law Review

series 4 (1890) pp 193–195

[1322] J Warrick, ‘‘Leak Severed a Link to Al-Qaeda’s Secrets’’, in the

Washington Post Oct 9 2007 p A01, ‘‘U.S. Intelligence Ofﬁcials Will

Probe Leak of Bin Laden Video’’, ibid., Oct 10 2007 p A13

[1323] Waste electrical and electronic equipment (WEEE) regulations 2007, at

http://www.netregs.gov.uk/netregs/275207/1631119/?version=1&

lang= e

[1324] M Watson, ‘‘Sat-nav ‘jammer’ threatens to sink road pricing scheme’’, in

Auto Express Aug 8th 2007; at http://www.autoexpress.co.uk/news/

autoexpressnews/209801/sat nav jammer.html

[1325] RNM Watson, ‘‘Exploiting Concurrency Vulnerabilities in Kernel

System Call Wrappers’’, in First USENIX Workshop on Offensive Tech-

nologies (WOOT 07), at http://www.watson.org/~robert/2007woot/

[1326] ‘‘Developer tortured by raiders with crowbars’’, M Weaver, Daily

Telegraph, 31 October 97

[1327] W Webb, ‘‘High-tech Security: The Eyes Have It’’, in EDN (18/12/97)

pp 75–78

[1328] SH Weingart, ‘‘Physical Security for the μABYSS System’’, in

Proceedings of the 1987 IEEE Symposium on Security and Privacy, IEEE

Computer Society Press, pp 52–58

[1329] SH Weingart, ‘‘Mind the Gap: Updating FIPS 140’’, at FIPS Physical

Security Workshop, Hawaii 2005; at http://csrc.nist.gov/groups/

STM/cmvp/documents/fips140-3/physec/papers/physecpaper18.pdf

[1330] SH Weingart, SR White, WC Arnold, GP Double, ‘‘An Evaluation

System for the Physical Security of Computing Systems’’, in Sixth

Annual Computer Security Applications Conference, 3–7/12/90, Tucson,

Arizona; proceedings published by the IEEE (1990) pp 232–243

[1331] L Weinstein, ‘‘IDs in Color Copies–A PRIVACY Forum Special Report’’

in Privacy Forum Digest, v 8 no 18 (6 Dec 1999), at http://www.vortex

.com/privacy/priv.08.18

[1332] L Weinstein, ‘‘The Online Medical Records Trap’’, Oct 4 2007, at

http://lauren.vortex.com/archive/000306.html

[1333] L Weinstein, ‘‘Not on Track with ‘Do Not Track’ ’’, Oct 31 2007, at

http://lauren.vortex.com/archive/000326.html

Bibliography 993

[1334] C Weissman, ‘‘Security Controls in the ADEPT-50 Time Sharing

System’’, in AFIPS Conference Proceedings, Volume 35, 1969 Fall Joint

Computer Conference pp 119–133

[1335] C Weissman, ‘‘BLACKER: Security for the DDN, Examples of

A1 Security Engineering Trades’’, in Proceedings of the 1992 IEEE

Symposium on Security and Privacy pp 286–292

[1336] G Welchman, ‘The Hut Six Story’, McGraw Hill (1982) ISBN

0-07-069180-0

[1337] B Wels, R Gonggrijp, ‘‘Bumping locks’’, 2006, at http://www.toool.nl/

bumping.pdf

[1338] A Westfeld, A Pﬁtzmann, ‘‘Attacks on Steganographic Systems’’, in

Proceedings of the Third International Workshop on Information Hiding

(1999), Springer LNCS v 1768 pp 61–76

[1339] AF Westin, ‘Data Protection in the Global Society’ (1996 conference

report), at http://www.privacyexchange.org/iss/confpro/

aicgsberlin.html

[1340] E Whitaker, ‘‘At SBC, It’s All About ‘Scale and Scope’ ’’, in Business Week

Nov 7 2005, at http://www.businessweek.com/@@n34h*IUQu7KtOwgA/

magazine/content/05 45/b3958092.htm

[1341] O Whitehouse, ‘‘Bluetooth: Red fang, blue fang,’’ in CanSecWest/core04,

linked from ‘‘Bluetooth PIN Cracker: Be Afraid’’ at http://www

.symantec.com/enterprise/security response/weblog/2006/11/

bluetooth pin cracker be afrai.html

[1342] A Whitten, JD Tygar, ‘‘Why Johnny Can’t Encrypt: A Usability

Evaluation of PGP 5.0’’, in Eighth USENIX Security Symposium (1999)

pp 169–183

[1343] J Wildermuth, ‘‘Secretary of state casts doubt on future of electronic

voting’’, San Francisco Chronicle Dec 2 2007, at http://www.sfgate

.com/cgi-bin/article.cgi?f=/c/a/2007/12/02/BASRTMOPE.DTL

[1344] MV Wilkes, RM Needham, ‘The Cambridge CAP computer and its

Operating System’, Elsevier North Holland (1979)

[1345] J Wilkins, ‘Mercury; or the Secret and Swift Messenger: Shewing, How a

Man May with Privacy and Speed Communicate his Thoughts to a Friend

at Any Distance’, London, Rich Baldwin (1694)

[1346] C Williams, ‘‘Surge in encrypted torrents blindsides record biz’’, in The

bittorrent encryption explosion/

994 Bibliography

[1347] CL Wilson, MD Garris and CI Watson, ‘‘Matching Performance for the

US-VISIT IDENT System Using Flat Fingerprints’’, NIST IR 7110 (May

2004), at ftp://sequoyah.nist.gov/pub/nist internal reports/

ir 7110.pdf

[1348] R Wilson, ‘‘Panel unscrambles intellectual property encryption issues’’,

EDN Jan 31 2007, at http://www.edn.com/index.asp?layout=article

&articleid=CA6412249

[1349] T Wilson, ‘‘Visa Gave TJX a Pass on PCI in 2005’’, in Dark Reading Nov 12

2007, at http://www.darkreading.com/document.asp?doc id=138838

[1350] FW Winterbotham, ‘The Ultra Secret’, Harper & Row (1974)

[1351] A Wolfson, ‘A hoax most cruel’, in The Courier-Journal Oct 9, 2005; at

http://www.courier-journal.com/apps/pbcs.dll/article?AID=/

20051009/NEWS01/510090392

[1352] K Wong, ‘‘Mobile Phone Fraud - Are GSM Networks Secure?’’, in

Computer Fraud and Security Bulletin (Nov 96) pp 11–18

[1353] N Wong, ‘‘ Judge tells DoJ ‘No’ on search queries’’, Google blog Mar

17 2006

[1354] CC Wood, ‘‘Identity token usage at American commercial banks’’, in

Computer Fraud and Security Bulletin (Mar 95) pp 14–16

[1355] E Wood, ‘Housing Design, A Social Theory’, Citizens’ Housing and

Planning Council of New York, 1961

[1356] L Wood, ‘‘Security Feed’’, in CSO, Apr 20 2007; at http://www2

.csoonline.com/blog view.html?CID=32865

[1357] JPL Woodward, ‘Security Requirements for System High and Compart-

mented Mode Workstations’ Mitre MTR 9992, Revision 1, 1987 (also

published by the Defense Intelligence Agency as document DDS-

2600-5502-87)

[1358] B Wright, ‘‘The Verdict on Plaintext Signatures: They’re Legal’’, in

Computer Law and Security Report v 14 no 6 (Nov/Dec 94) pp 311–312

[1359] B Wright, ‘The Law of Electronic Commerce: EDI, Fax and Email’, Little,

Brown 1991; fourth edition (with supplement) 1994

[1360] DB Wright, AT McDaid, ‘‘Comparing system and estimator variables

using data from real line-ups’’, in Applied Cognitive Psychology v10

no 1 pp 75–84

[1361] JB Wright, ‘Report of the Weaponization and Weapons Production

and Military Use Working Group–Appendix F to the Report of the

Bibliography 995

Fundamental Classiﬁcation Policy Review Group’, US Department of

Energy Ofﬁce of Scientiﬁc and Technical Information (1997),

http://www.osti.gov/opennet/app-f.html

[1362] MA Wright, ‘‘Security Controls in ATM Systems’’, in Computer Fraud

and Security Bulletin, November 1991, pp 11–14

[1363] P Wright, ‘Spycatcher–The Candid Autobiography of a Senior Intelligence

Ofﬁcer’, William Heinemann Australia, 1987, ISBN 0-85561-098-0

[1364] JX Yan, ‘Security for Online Games’, PhD thesis, University of

Cambridge 2003

[1365] JX Yan, A Blackwell, RJ Anderson, A Grant, ‘‘The Memorability and

Security of Passwords–Some Empirical Results’’, University of

Cambridge Computer Laboratory Technical Report no 500; at http://

www.cl.cam.ac.uk/ftp/users/rja14/tr500.pdf;alsoinIEEE Security

& Privacy, Sep–Oct 2004 pp 25–29

[1366] JX Yan, S Early, RJ Anderson, ‘‘The XenoService–A Distributed

Defeat for Distributed Denial of Service’’, at Information Survivability

Workshop, Oct 2000

[1367] JX Yan, B Randell, ‘Security in Computer Games: from Pong to Online

Poker’, University of Newcastle Tech Report CS-TR-889 (2005)

[1368] JX Yan, B Randell, ‘‘A systematic classiﬁcation of cheating in online

games’’, at Proceedings of 4th ACM SIGCOMM workshop on Network

and system support for games (2005), at http://portal.acm.org/

citation.cfm?id=1103606

[1369] T Yl¨

onen, ‘SSH–Secure Login Connections over the Internet’’, in Usenix

Security 96 pp 37–42

[1370] KS Yoon, YK Ham, RH Park, ‘‘Hybrid Approaches to Fractal Face

Recognition Using the Hidden Markov Model and Neural Network’’,

in Pattern Recognition v 31 no 3 (98) pp 283–293

[1371] G Yuval, ‘‘Reinventing the Travois: Encryption/MAC in 30 ROM

Bytes’’, in Fourth International Workshop on Fast Software Encryption

(1997), Springer LNCS v 1267 pp 205–209

[1372] MC Zari, AF Zwilling, DA Hess, KW Snow, CJ Anderson, D Chiang,

‘‘Personal Identiﬁcation System Utilizing Low probability of Intercept

(LPI) Techniques for Covert Ops’’, in 30th Annual IEEE Carnahan

Conference on Security Technology (1996) pp 1–6

[1373] ZDnet, ‘‘Software blocks images of money’’, Jan 12 2004, at http://news

.zdnet.co.uk/software/0,1000000121,39119018,00.htm

996 Bibliography

[1374] K Zetter, ‘‘Scan This Guy’s E-Passport and Watch Your System Crash’’,

in Wired, Aug 1 2007, at http://www.wired.com/politics/security/

news/2007/08/epassport

[1375] RS Zhang, XY Wang, XH Yan, XX Jiang, ‘‘Billing Attacks on SIP-Based

VOIP Systems’’, in WOOT 2007

[1376] L Zhuang, F Zhou, JD Tygar, ‘‘Keyboard Acoustic Emanations

Revisited’’ in 12th ACM Conference on Computer and Communications

Security (2005)

[1377] P Zimbardo, ‘The Lucifer Effect’, Random House (2007)

[1378] MW Zior, ‘‘A community response to CMM-based security engineering

process improvement’’, in 18th National Information Systems Security

Conference (1995) pp 404–413

[1379] M Zviran, WJ Haga, ‘‘A Comparison of Password Techniques for

Multilevel Authentication Mechanisms’’, in The Computer Journal v36

no 3 (1993) pp 227–237

Index

NUMBERS AND SYMBOLS

*-property

of BLP model, 245

in Chinese Wall model, 282

3gpp (Third Generation Partnership Project),

617–619

4758, IBM

API attack on, 551–552

high-end physically secure processors,

486–487

how to hack, 491–492

5000 series microcontroller, 495–496

9/11 terrorist attacks

security engineering conclusions, 891

security engineering framework, 5

terror, justice and freedom, 769–771

A5 algorithm, 613–617

AACS (Advanced Access Content System),

701–703

Abadi, Mart´

ın, 180

absolute limits, 57–59

abstract networks, 564–565

access control, 93–96

Apple’s OS/X, 101–102

capabilities, 103–104

environmental creep, 125–126

Security Engineering: A Guide To Building Dependable Distributed Systems Engineering

Navigation menu

Versions of this User Manual:

Views

Navigation