Digital Technical Journal, Volume 8, Number 3 Dtj_v08 03_1996 Dtj V08 03 1996

dtj_v08-03_1996 dtj_v08-03_1996

User Manual: dtj_v08-03_1996

Open the PDF directly: View PDF PDF.
Page Count: 83

DownloadDigital Technical Journal, Volume 8, Number 3 Dtj_v08-03_1996 Dtj V08-03 1996
Open PDF In BrowserView PDF
I
INTERNET PROTOCOL V.6

Digital

PRESERVATION OF HI STORICAL
COM PUTER SYSTEMS

Technical

FORTRAN FOR PARALLEL COM PUTING

Journal

SERVER PERFORMANCE EVALUATION
AND OPTIM IZATION
INTERNET COLLABORATION SO FTWARE

Volume 8 Number 3
1996

Editorial

The !)fgiwl 'f(•cbuical.foumal is a refereed

The folloll'ing arc rradem,11·ks of Digital

)an� C. Bl.1kc, ;Vl;ln;1ging Editor

journJl published quarterly by Digital

Equipment Corporation: AlphaServcr,

Helen

Equipmenr Corporarion,

L. Pattnson, Editm

Kathleen M. Stetson, Editor

Circulation
Catherine M. Phillip>, Administrator
Dorothc� R. C1ss;1dv, Sccrct<1ry
Production

T�rri Auticri, Production Editor
Ann� S. K�tzcff, T)'l.lOf'.L1phcr
PetcT R. \.Yoodhury, Illustrator

Advisory Board

Subscriptions un be ordered by sending
:�check in U.S. funds (lll<1cl� pJvable ro
Digiral Equipment Corporation) to the
published-bv address. General subscription
rates ar� $40.00 (non-U.S. $60) for four
issues and $75.00 (non-U.S. $115) tor
eight issues. Univnsitv and ..:ollegc pmti.:s­
sors and Ph.D. sructcnrs in the dectric1l
engineering and computer s..:ience tields
re..:eive COtnf'limentJry subscriptions upon
request. Digit;1l's utstomers m.1y qualit\•

S�rnuel H. hiller, Chairman

for gift subsu·iptions <1nd Me cncour « ged

Rich.1rd W. !kane

ro cont«([ their account represenrr a ..:omplctc listing. R.:cetlt issues are

Robert M. Supnik

also <11.;1ilabk on the fnrcnKt ar

http:/jwww.digital.com/info/dtj.
Digit:1l ctnployees may order subscriptions
through Readers Choice chnico!Jollnwl ;Jt the
publishcd-lw address or the ek([mnic
m;1il ;1ddress, cltj@digir<11.com. Inquiries
can ,1lso be tmcle bv calling thejol l rnal
Comments on the conretlt of anv f'aper
;1rc 11·ckomed and mav be sent to the
managing editor at the publishcd-lw or
electronic mail address.

Copvright © 1996 Digital Equipment

Corpor<1tion. Copying ll'itholl[ fcc is per­

bv t:JCulrv

mitted provided that such copies arc ll1<1de
for usc in cclucarional institutions

menJbcrs and arc nor distributed for com­

Equipment Corporation or lll' the comp:1·

innumcr:1bk peopk in "cl'ixrsp:1ce." Inside
the issue, tii'O p<:rs r collabor:1rion
on rhe lmcrnct.

t.lkenlw Chuck Cilkttc ofsh dil'<:rs 11·ho

timmrion.

set a rcwrd in October 1996 tilr rhc lllllll­
of Digit�I\ Corporate Design (;roup .

The· COI'l'r dc:sign is 111' l.ucincb O'Neill

& Associates,

J\t!EMORY CH ANNEL is <1 tr<1detmrk ot·
Encore Computer C:orpor:Hion.
Microsoh and Visual C++ .He registered
trademarks and Windo11·s and Win doll's NT
.liT tradem,Hks of Microsoft Corporation.

fne.

MIMIC is a trademark of Sierra
Geophvsics,

nics herein rq1resented. Digital Equipment
Corpor.Hion assumes no responsibilitY for
atw errors that lll<1)' :1ppear in thejounwl.
ISSN 089R-90 I X

Orack7 is :1 tr;Jckmark ofOr�de Corporation.
SoLlris ;111d S l't\ RCcelltcr ;H·e rcgistuecl
trademat-ks of Sun Minosvstcms, Inc.
SPFCint is <1 tradcm:Jrk of the St�nd;1rd
Pcrtimn:HKe b·;1luarion CotHKil.
Svbasc is <1 registered tr;1tkmark of
Svbase, Inc.
TPC-C is a tr;�ckmark of the Tt-;1nsacrion

Documetlt,Jtion ::--lumber �:C-;-...17285-18

Processing Pt'rt()rm;lncc Coun..:il.

Book production \\'aS done bv Qun.

be construed .1s .\ cnmmittnenr bv Digit<11

(CllllllH111iGHion. The sr.tphic ()l) our({)\'(]"

r rhc purpose of

INI-'ORNIIX and INI-'ORMIX-OnLinc

nH:Tei
  • r rmance that can he computing histon·. The re-creation, dcri1·cd ti-om a I'Crl' bst intnconnecr both p h ysica l and logical, ofcompur­ p•1pcr demonstrates how easv it is to build J S\'Stem that would ha1·c been ,md sofr11arc srructu1·es ro m.ltch. •1 111ajo1- f)rojccr just fin: v c11·s <�go. ing S\'Stcrns ofthe past on onII- he! p ·rhe pcrr(>rm•wce Je,·cls •Khic,·ed This case of construction is a bcnctit remind us that the artif:Kts 11·c crcuc sh,lttcr 11 orld records on a bench­ ofthe programming techniques and ha,·c a longer lire than we anticipate. mark that Ius l1ad exrensi1·e atten­ infi·,lstructurc im·estments and J spur As our programmers write new code, tion .md ll'ork. to keep doing more ofit. or our hardw:H"C designers produce new architectural approaches, or our The other pr m :�ki n g the u l timate decision . I n J u ly 1 994, the 1 Png d i rectora te selected the the evol ution of the pJc ke t t(mnat hender. I t retai n s som e ti c l d s ( ve r ­ is rcn:� med t h e H o p Li m i t ) , a n d i n trod uces n ew o n es I nternet ( s u c h as Fl o\\' l D ) wi t h as vet u n tapped potcmia l . The Protocol \'CI'Sion 6 ( l l"·6 ) as t h e repbce mc n t n c t\\' or k ne xt hudcr J a ,·e r p rotoco l , Jnd I ETF \\'orki n g groups bega n to ticld ai iO\\'S fo r m od u l a r constr uction o f complex p�1c ke ts : d i ffere n t h eade r t\vcs em he c h �1 i 1 1 cd b u i l d spe c i fi cati ons . "The Recom m e nd ation t( >r the toge th er to pro,· i d e speci a l i z e d tu nctio n a l i rv, i n c l u d ­ I P Next Ge n e rati on Protocol" su mmarizes the c1 11d i ­ i n g sec u r i t\' a n d source routing. h n a.l l v, a l l hc:�dcrs arc d a tcs �m d c xpb i ns the selection of this p rotoco l . ' s t r u c tured to a l l o\\' 64 - b i t alignment, \\'h ich shou l d a l l ow opti m : d p rocessi n g both a t source a nd desti n a ­ tion systems, as \\'ell as i n transit ' Digital UNIX Prototype Th e c u rrc l l t Di gi t:� I U N I X 1 Pv6 p rototype p roject is Th e most stri ki n g d epa rtu re ti-om is the :�d d rcss s i t.c : i t has i nc reased from 32 b i ts to 1 2 8 bits. d eve l op a n d e va l u ate The J P\'6 ad d ressi n g arch itecture is ri c h , with p re fi xes the co mpe ting I P n g p roposa l s . This began w i t h t h e S i m pl e I n ternet Protocol ( S I P ) , t( Jr m u l ti cast a d d resses and p rc d c ti ncd which used e ight octet ad d resses . S I P was later melded with another ea rly proposal a n d bec a me kn o w n as u n icast and m u l ticast a d d resses. One s peci a l type of u n ica s t �1d d ress is tl1e l i n k - loca l add ress , w h i c h pe rm i ts S i mp l e I n ternet Protocol comm u n i cations \\'i t h on l y t hose syste ms d i rectl y con­ P l u s ( S J P P ) , the d i rect a scopes rcJr bo th a nteced e n t of ! Pv6 . ' Th e p r i ma ry goa l of D igi ta l 's nected on the same l i n k . Th is a l l ows cftC>rts has b ee n to ev:� l u ate the tec h n i c a l fe as i b i l i ty o f s trap pi n g mecb:�nism, so that S\'Stc m s can lc�1rn a b o u t the proposed :� rc h i tcct u re and p ro,·id e k e d bac k to the op tio n s h�1\'C been d e fi n ed , i n cl u d i n g hi er:�rchical models based u pon regional 1-cgisrries �1 n d sen·ice tiple i ndependent �md interoperable i mplcmenutions �1 stan (brd boot­ ne i g h bo rs a n d scn·ices before a ro u r :� bl c :� dd rcss i s �1 ssi g n ed to an i n terrace . Va rious ad d res s :.� ssi gn mc n t \\'Or ki n g gro u p s . This is critical to the s t:� n d a rd s dC\ e lopmel l t process in the I ETF, ,,·bich req u i res m u l ­ I ETr spccitic:�tion bdore it ma\' become a n I ntern e t p rm·id c r i d c n t i ti e rs .' n I n each case, care h:�s been ta ken sta n d a rd . An a d d itional goal has been to e,·�1 l u arc svs ­ to e ns ure p ro pe r m u te aggrega tio n , \\ hich \\'i l l h e l p rcm d esi gn :� l te rnativcs to gain the expe rien ce needed vi c ki more e ffi ci e n t b:�ck bone router pcrt(mn�mcc . of M u l tip l e mc:�ns of acq u i ri n g add resses have been to a l low D i gi ta l to i ncorporate t h is nc,,· :� rc h itccru rc i mo e x isti ng p rod u c t s . D i gita l h :� s mad e the protot pe dcti ncd t(Jr J P, 6 add ressin g, wi t h y Jva i b bl c to rcsc ::1 rc he rs with i n the company as �1 source V E RSION I P R I O R ITY I I NEXT H E A D E R SOURCE ADDR ESS DESTINATION A D D R ESS F i g u re 1 1 1\·6 H e ader Digi r:d Tcclmic.ll jou rnal Vol . 8 N o . 3 l 'o/9 6 the go:� l s o f a l l o\\'i ng tl c x i b i l i t\' t h rou gh d i fferent a d m i n istrative po l ic ies FLOW LABEL PAYLOAD LENGTH 6 I l'v4 D igi ta l ' s most recent add ition to an on goi n g cHc>rt to I HOP L I M I T and , p e rhaps more i m port J il t , o F d e m J n d i n g that net­ work add ress r eassi gn m e n t be supported t h roughout the an: h i tectu re . T h e two new :1dd rcssi n g services are EJ Statel ess A d d ress A u roco n tigu ra tion a n d t h e state fu l , transac tion - b:.1scd Dynamic H os t Confi gu rati on P ro ­ tocol version 6 ( D H C P,·6 ) . c s I n t h e stateless mod e l , add ress prefixes a rc learned lw l i s te n i n g fo r rou ter ad\-ertise m c n t packers . A d d resses a n: t( J rmcd bv com­ bi n i n g the p r e fi x ,,· i t h a l i n k-specific token such as the 4 8 - b i t Ethernet h a rdware a d d ress . I n the stateful pro ­ ccd ure, hosts ma\' req uest Jdd resses, con fi gu rati on i n formation, and services tl·om d e d icated confi g u ra ­ USER KERNEL res u l t i n g a d d n:sses have associated l i fe ti mes, and sys­ tems m ust be prepared to both lea rn new add resses a n d release e x pi red add resses. Combi ned with the a b i li ty to register updated add ress i n formation with NETWORK COMMANDS AND U T I L I T I E S TRANSITION MECHANISMS I Ll I 8 El I I so c KET LAY E R TRANSITION MECHANISM DYNAMIC N ETWOR K LAYE R ADDRESS tion sen·ers, with routers pote mi a l l y sen·ing a s rel ay stations d u ri n g t h e i n i ti�1! phase . I n both cases, the I P-BASED A P PLICATIONS __________J s E c u R I TY ROUTING TABLE AND N E IGHBOR CAC H E N E I G HBOR DISCOV E RY L I N K-LAY E R MODULES I PV6/IPV4 T U N N ELS F i g u re 2 Base Platform Cha nges DNS servers, these mecha n isms provide a path toward n e twork re n u m berin g , a goal thJt has proved d i ffic u l t t o ach ieve i n t h e l l'v4 world . Final ly, the I nternet Con tro l MessJgc Protocol ver­ sion 6 ( I C M Pv6 ) >vas d eve lop ed ." This specification aimed to m erge the fu nctions o f two d istinct I Pv4 pro­ tocols tor report i n g errors and status, I C M P to r u n i ­ cast packet transm ission a n d t h e I n ternet Gro u p Message Protocol ( I G M P ) tor m u l ticast traffic . The messages ddi ned i n this protocol arc catego­ rized JS either e r ror or i n t(mnational , with a fa m i ly of messages i n the second gro u p used to prov i d e the Neigh bor DiscO\·e r v P rotocol . ' " Neigh bor d iscov e ry ser v es m u l tipl c p u rposes with t h e O\'crJ ! I t h e m e of p r m· i d i n g a system \\' i t h to pological a n d e n v i r on me n ­ tal h i ms . F o r e xa m p l e , l i n k - l aver J d d rcss reso l u ti o n , rou ter d iscovcrv, d esti n a ti o n a d d ress red i recti on , and :� d d rcss a u roconfigu r:�rion mecha nisms arc :� I I spec i fied usi n g neighbo r d iscO\·erv packe t tvpes. Although the network layer did experience the l argest Jmoum of change, Figure 2 shows that t h e effects of this work tou c h nearly all aspects of the Digital U N I X system . We point o u t examples ofdccisions nude d ue to our fu n d a m e n t: tl d es i g n philosophy, which is based u p o n i mcg ration with the U N IX system trJ m cwo r k, mod u la r Jlld extensible sofuv:1rc, su pport tor m u ltiple ope ratjo nal policies, and J desire to take the Alpha pla t tcmn without a d vantage of com p romisi ng portabi lity. I n t h e t(J!Jowi n g sections, we study these topics i n d e p th , begin n i n g with t h e network layer, then cover­ ing t h e transport layer rnod i ticJtions a n d the n ew neighbor d i s cove ry algori t h m s . Alte r that, we d i scuss Jdd ress a u toconti g u rJtion mecha n i sms and their e ffects u pon the svste m . W e concl u d e wi th services t hJt will be a fft:ctcd by the transition tl· om I Pv4 to Network Layer In t h is section , we rn· i cw the p rocessing req u i re m e n ts of t h e I Pv 6 m od u les , i n c l u d i n g I C : M Pv6, extension header op ti o n s , a n d ti·Jgmcnta tion . A n early d esign decision was made to base the networ k i n g su bsystem o n the B erkeley StJmiJrd D istri b u tion ( B S D ) 4 . 4 m o d e l a n d c o d e base , w h i c h J ! !ows great tlcx i bi l i ry i n d e al i n g with m u l t i p l e n e twork layers . 1 1 ·rh i s ::dso h ::1 s the ad\'an tage of pro vid i n g s u pport tc) r vJri�l b l c - bi t ­ l e n gth net masks ( also known a s Cl D R-stvlc nctm asks , fro m C l assless I n tcr- Dom J i n Ro u ti n g ) , w h i c h �1re appropria te to b ot h I p, 4 Jnd I p,·6 . " We ha,·e :� I so tried to take m Jx i m u m ad,· a n tage of the 64 - bi t A l p h J a rc h i tecture when i mp l e m e n t i n g I p,·6 , w h i l e ma ki n g certain that this i m pl e m enta ti on \\'O u ! d r u n on 3 2 - b i t C P U s as wel l . For c x J m p l c , t h e c h ecks u m rou ti nes operate on 3 2 - b it q u a n tities ( a l lowing t h e GI ITV ro ove rflow i nto the upper 32 b i ts o f J 64 - bi t registe r ) . T h e checks u m rou t i ne i s a lso designed t o a l low i t t o b e issu ed t o m u l t i p l e A l p h a exec u ti o n un its, which remains a topic tor fu rther i n vestiga tion . Adaptations to Existing IP and ICMP Routines The 1Pv6 and I C M Pv6 rou ti n es J rc completely i n d e pe n d e n t of the cor r espond i ng 1 Pv4 and I C M Pv4 r o ut i n es , and the processi n g styles have d istinct d i fk r­ e nc es . In l Pv6, the inco m i n g packet is treated as bei ng r e a d - on ly, w h i l e the B S D 1 Pv4 code m a n i p u ! J tcs fields wi th i n the lPv4 header. We also avoi d u n neccsS:Jrv usc ; of the m_ p u l l u p routine ( w h i c h conso l i Lbtes ch i n cd lPv6 such as the socket J p p l i cnion p rogra m m i n g me mory b u ffers i n to a single l a rge b u fk r ) bccJusc t h i s i ntcrfJce ( A P I ) a nd D N S . c o u l d cause the packet to be necd lcsslv lost. F i n a l l v · i n s tead o f p assi ng n u mero us cl rgu m en t w h e n c a l l i n � � from fu nction to nmction, a COI111110n cbt J structure is Digital Tec h n ical J o u rn a l Voi . R ::--.J o 3 1 996 7 used to store ncccssarv d J t: l a n d p o i n te rs ; r(Jr most rl.1 n ction c a l l s , i t is o n l y necessary to p�1ss a poi n te r first valid ated fo r correct packet r(m11Jt :� n d c h ecks u m . The packet is t h e n rL! rthcr processed b�1scd u pon i ts t o t h i s structure. T h i s red u ces t h e stack overhead a n d I CiV!Pv6 type va l ue . I f i t has an I C : M l\·6 error type ( i .e . , a l so \ ' i c l d s m od u l a r <1 11d c a s i h · extensi b l e s u bro u t i n e s . tvpe v a l u e J ess t h a n 1 2 8 ) , t h e a p p ropriate notifications I P,·6 has a d e d icated i n t e r r u p t processi n g th read , are sent to the a fkcted protoco l . Neigh bor d iscovcn· and rccci ,·cd 1 1".·6 packets J rc p l :�ccd onto t h e i r own packets , which arc al l i n r(mnati o na l , hJ\'C a n u m be r of i nt e r race i n p u t q u e u e ( i fl1 u c u c ) . When an l P\ 6 packet a d d i tion<1 1 is taken off the i f(] u c u c , basic ,·a l i d i tv tests are d o n e ; d ropped if it h i l s the m . Afi:er the I C : M P,·6 packet has is d i rected t o The to rmer yiel ds the d esirable larger O R I G INAL PACKET FRAGMENTABLE PART U N FRAGMENTABLE PART FRAGMENT PACKETS UNFRAGMENTABLE PART U N FRAG M E NTABLE PART U N FRAGMENTABLE PART F i g u re 4 Fragmenrarion Digir�l Technic\! ) o u rn ;\l Vol . 8 No. 3 1 99 6 9 p:1ckers , w h i l e the l :mc r :1 \'o i d s u n de s i ra ble fragm ent loss ( d ue ro t he tr:1 gm en r p a ck e t b e i ng too bi g ) . T h e D ig i ta l U N I X 1 Pv6 p rototv p e su pports either choice on a s �·stem -wide , per - d es r i n :lt i on, or pe r- sock e t basi s . T h i s i s J n cx:1mpk of separJtion of m e c h J n is m trom pol icy, J bJsic gu i d e l i n e bei n g used across this project. Reassembly rcasscm b l v process r ec o n st ru c t s t h e ori g i n a l p a cke t ti·o m tra g m e n r p�1c kets. F ra gm e n ts be longing to the same p<1cket a rc i d c n ritied b\' J combin:1tion of so u rce I P J dd rcss , not h e a d e r tYpe ( fi rs t h e:1dc r of the tr a gm e n t•1 b lc p:1rr) and ri·a g m c nt i d e n t i fier. I n d i ,· i d m l fi· J gmc nts arc queue d wi t h i n the network l aver until the orig inJI pac ker c J n be co m ple tc l v reassembled, at which p oi n t i t i s pa ss e d to the appropriate p ro to co l m od u l e . \Nhen :1 l l fr agme ms h �l\'e arr iv e d , the origi n a l packer can be n.:as se m bl cd . A s i n gk copy of thc u n tra gm e n r ­ Jble part is kept , Jnd the data from ea ch ti·a g m en t packet is ap pen d e d . The p :1 y lo :1d len gth fie l d of the I Pv6 header is u p dated to rdkct the le ng t h of the reasscm­ bkd pa c ke t , and the next hc;�dcr ticld of the last header of the u n ti·agmcntablc par t is restored to rdlect the fi rst header in the ITa g m c n tJbl e p Jr t. As with the ti·:lgmc ntJtion c o d e, care must be ta ke n so thJt ti elds b ei n g u p lhtcd arc not in bu ffers shared with other cop i es of t he p J c kc t . When the tirs r tl-:1 gm c nt of J p Jc kc t arrives, J ti mer is s ta r ted . I f the rimer expires bdo rc th at p ac ke t is co m pl et e , the tr:1g mcnts Jre d isc;�rded . I f the othe r zero tra gmcnr h :1s been rccci,Td, Jn I CM p,·6 error message is sent. The Forwarding and Routing ! f a rcccin.:d pa c ker docs nor m:nch one ofrhc S\'Stc m 's Jnd the svstc m is not J c tin g :\S a rou ter, the packet is s i l c nr lv d ro p p ed . Othcn,·isc, an attempt is made to t()rW:lrd t h e packet. The �i rst step in torward ­ ing i� to do :1 lookup in the rou t i n g t:tblc; t h e tvpe of look u p de pe nds on whether the p a c k er contains a nonzero tlow bbcl. I f i t docs, the lookup is based o n both the source :1d drcss Jnd the tl ow label; othe rwi se the destination add ress is used. If the l ook u p su cceeds and the len gt h of r h e p:1 c k c t tits within t h e MTU of rhc resu l u n t route a n d i n rc r �ace , the packer is tr J n s rn i t re d to r h c next hop :\S i n d i u rcd by the ro u t e . O t h e r wise an appro p ri a te I C M Pv6 e r ror is sent back to the origi­ n a t i ng n od e . add resses Tu nne l i ng is a mechanism that J l l ows pa ck ets of one network type to be e n c 1p su l � nc d :lllli to nva rded wi thin a n e tw or k layer p a c k e r of r h c SJnH:: or a d i ffe re nt rvpe . J P,-6 pa c ke rs CJil be ru n n c l cd over e i t h e r I P,·4 or 1 Pv6 ne twor k s , JS m:w I p,.4 p :1 c kc ts 1"r T h e tu n n el i ng rou­ tine r:1 kcs as i np u t a p :1c kc t, p rep c n d s t h e ap p ropr i a te Tunnels 10 Vo l . 8 No 3 1 996 h e a d e r tor t h e network ove r which the packer wi l l be tu n n el ed , :1 nd trJnsmits r h c re s u l ta n t p ac ke r m·cr rhar n etw ork . Tu n nels JI'C unid irccrionJI; there need not be a co r res pondi ng tu n ne l in the reve rse d i recti o n . Rather thJn h :l\'ing m u l ti pl e tu nnel i nrcrtaccs (one tor e J c h p o ssi b l e combi nation of protocol Y 0\·er protocol X ) , the D i g i t:1 l U N I X i mplem e nt at ion uses a si n gle runnel i nrcrtac c . This method was the sugges­ tion of Ke ith S kl c m e r of the Un i,nsit-v of C:-� l i torn i:-� ar Berkc lc\'. 1 8 Wh e n th e i n te r race is i n i ri �1 l i ze d , on l\· :l u to m a ric t u nn eli n g o f I p,-6 over I 1'\-4 is cn:1blcd . 1 '' To confi gu re :1 static tu n n e l , w h e re ti \cd e n d po i nts are use d , a sutic ro u te is �1d d cd to rhc ro u tin g t:lbles wid1 the proper d e sti n a ti on a n d gJt C \\ 'J\ ' ( ru n n e l e n d poi nt ) ad d re s se s. 'vVhen J pa c ke r i s p re se nte d to rhc ru n n e l i n te r race, i t looks up the route entrv of rhc desti nation address. The route co n te nts te l ls the tu n n e l in g rou t i n e how the packer is to be e ncJ ps u iJ tcd and t()nva rded . T he route's gateway address i n d ic at es w h a t u n d c rlv i n g network to use, and t he rou te's d estination add ress ind icate s w h rt was the co m ­ to preserve the \\'e l l - u n d e rstood r r:� ns po r rs, TCP Jn d U D P , upon w h i c h a \\'CJi t h of app l i cn io n s hJ,·e b e e n b u i l t . The 1Pv6 s pc c i fi c J t i o n c:-� l l s r(>r thr ee pa r r i c u l �1 r re q u i rem e nts of u p pe r- l aver p ro toc ols : One m i r ment 1 . The pse u do h e a d e r c h e c ksu m must accom modate larger Jddresscs. ma \ i m u rn m p u ted . 2. The co p:1c kcr l i tctimc 1s no lon ge r 3. The l a rge r I P\'6 hc:-�dcr( s ) m u s t be Ll ken i n to acco u n t when c o m p u t i n g the 111 3 \ i m u m p::tylo�l<.i size ( e . g . , TC I)'s m J \ i m u rn seg m c m s i z e [ M S S ] ) ' I n J dd i tion ro th e se ma n d ate d mod ifi ca tions, we h ad to nuke a fu ndJment:ll design ch oice . With rwo d i ffe r­ e n t ne tw or k l a y e r protoco l s in th e syste m, each u s i n g a d iftere n r si ze :-�ddrcss, our design c hoice c o u l d h J ve been ro use two i n depe n de nt tra ns port mod u les, one fo r each n e twork l a ye r. fi g ur es 5 a n d 6 s h ow the i nde­ p e n d e nt versus th e i ntcgr� ue d t ra ns p ort de sig n op ti o ns . Although the i nd e pe nd e nt model oftc rs an clement of d es ig n sim p l i c i rv, i t w:l stc s memory by d u p licati ng e a c h transpon l ave r ti.mcti o n . I n t h e Digital U N I X i m p l e m e ntat i o n , these modu les JIT i m p le m e n te d in the k e r n e l , and d u pli c a ti o n \\ O u l d be ex p e n si ve . Also, the d e s i g n :1nd usc ot' a si n g l e p ro g n m m i ng i mcrLKc to acce ss both sets of scn·iccs wou ld be c o m p l i c :l te d . SOCKET LAYER �� - - - - - - - - - - - - - - - 1 KERNEL rPC'Bl--i � I AF_INET V4 TRANSPORT I PV4 I I I I AF_IN ET6 V6 TRA N SPORT I PV6 _ _ �rPC'Bl I � Al though the concept of a sockad d r i s generic i n the BSD arch i tecture, the tl ow l abel a nd i n 6_ad d r mem ­ bers of t h is structure are u n i q u e to I Pv6 a n d wou l d be used o n l y in the AF_INET6 add ress fam i ly. The detai ls of this are specified in Reference 2 1 . The i n_pc b data structure is created for each socket using TC P , U D P , or other c lie nts of the net\vork layer. In :-td d i ti o n to stori n g the sou rce and d estination addresses, various other pieces of i n tonnation req u i red F i g u re 5 tor proper communication are stored here, i n c l u d i n g Indcpcndcnr Tra n spo rt 1 mple m e n ration the p o r t n u m bers, options a n d tl ags , a pointer r o the socket receiving the d ata , a header te m p l ate, and a SOCKET LAYER �� - - - - - - - - - - - - - - - 1 KERNEL AF_INET/AF_INET6 �r- V4 A N D V6 T RA N S P O RT ::::::: �::::::::: -��: : � , Pv4 _ _ ... L _ _ _.1 �-�--'_P_v6 _.1 _ - _ _ pointer ro the routi ng entry tor the given desti natio n . F o r 1 Pv6, this basic model h as bee n retai ned, a n d addi­ tional i n formation is stored . This i nformation incl udes l ocal and remote tl ow l a bels a n d i n d i o tors of which , PC B I � _ address family the application is using a n d which net­ work layer the transport com m u n ication is u si n g . F i n a l ly, a partial checksu m o f t h e transport pseudo­ header i s stored here as well ; its usc i s described in the fo llowi ng secti o n . Fig u re 6 I n addition t o the e x p l i c i t storage of the JKt\vork I n tegrat ed Tran spo r t I m plementation layer and address fam i ly, the fu ndamental tech n ique that fac i l itates the use of a common tra nsport is the The a b i l ity to maintain, let alone exte n d , the code base storage o f I Pv4 add resses i n a n I P\·6 format. This is wou ld a lso su ffer. Fortunately, d u e to the fact that known as a n 1 Pv4 - mapped a d d ress a n d is described I Pv4 add resses are a well -defi ned su bset o f the e ntire i n "IP Ve rsi o n 6 Add ressi n g Arc h i tectur e . " 2 0 This I Pv6 address space , it is rel a tively straightforward to a d d ress fo rmat is expl ic i tl y reserved to store add resses i mp l e m e n t the transports so that a single set of mod ­ of systems that arc capa b l e of u s i n g o n l y the I Pv4 u l es can be used over both network layers.20 To accom ­ protocol , and rhus is an appro p riate form of storage p l is h this, w e i nc reased t h e storage space aLlocated in t h e PCB for comm u n ications that wi ll be sent using tor add resses and separated those fu nctions that arc the I Pv4 protocol, as opposed to fPv4-compati b l e dependent u pon a particular n e twork layer. vVe discuss OOOO : OOOO : OOOO : OOOO : OOOO : F F F F : 204 . 1 2 3 . 2 . 75 Storing Large Addresses Two spec i fi c data structu res m ust be m o d i fi e d to accommod ate add resses l a rger than the 3 2 - bi t 1 Pv4 ty pe . The fi rst of these is the sockaddr struct, which i s used w h e n deal i n g with t h e B S D socket layer and passed alon g to user appl ications. T h e secon d is the I n ternet Protocol Con trol B lock ( PC B ) data struc­ ture , the i n_pcb. I n this secti o n , we review the modifi ­ cations t o each structure. A program that uses a transport does so by means of the BSD sockets i nt e r face and passes address i n g i n for­ m ation in a sockaddr str ucture. For 1Pv6, this is a sockadd r_i n 6 . I n te rn al ly, the structure is d e tl ned so that 64 - bi t ali gnment is preserved; however, it has the fo l l owi ng p u b l i c defi n i tion : st ruct } ,. sockaddr addresses, w h i c h are sent using I P v 6 packets. These mapped add resses are o f the fo l lowi n g form : each of' these issues in this sectio n . i n6 These add resses arc m a n i p u lated with i n the I Pv4 TCP a n d U D P protocols by means of macros that a l l ow the I Pv4 add resses to b e i n serted , extracted , or compared w h i l e in (in6_addr) . As an I Pv6 address structure t!·agmcnt in F i gure 7 shows a n add ress bein g extracted tor use in eva l u ating a conflgurable 1 Pv4 socket opti o n . Special Transport/Network Layer Interactions W i t h i n the integrated transport layers, the transport protocol i s treated independently o f the particula r n e twork l a y e r being u s e d , a n d n e t\vork - l aycr-speci flc functions are used to i n terface to either I Pv4 o r 1Pv6. There are { an example , the code t\vo partic u l a r i nstances i n which the i nteractions with the 1 Pv6 nct\vork transport l ayer has u_ c h a r u_ c h a r s i n 6_ l e n ; layer over a n d above the excha n ge o f· d ata packets tor s i n 6_ f a m i l y ; u s i n 6_p o r t ; i n p u t or output. These are the notification a n d u pdate short u_i n t struct s i n 6_ f l o w l a b e l ; i n6 addr s i n 6_a d d r ; o f path MT U , w h i c h is req u i red in 1 Pv6, and the pote n t i a l to refresh the neighbo r d i scovery cache based o n f(xward progress; i . e . , i f the transport knows that d a ta is reac h i n g i ts destinati o n , it can val i d ate the Digital Technical Journal Vol . 8 No. 3 1 996 II I* * T e s t a d d r e s s f o r ! Pv4 *I i f ( i n p - > i n p_ n e t l a y e r = = AF st ruct tmp; i n_a d d r t m p . s_a d d r i f character i s t i c !NET) { I N 6 _ E X T R A C T _ V 4 A D D R ( i n p - > i n p_ f a d d r l ; = ( ! i n_ L o c a l a d d r ( t m p ) ) } F ig u re 7 Code F ragm enr of a 1 Pv4-mapped Address c u rrent n e nvork l aye r pa th . We i n vestigate each of occurrence on I P v6 hosts, a n d so t h e p rocess o f these issues i n turn . c h oosi n g a sou rce add ress to usc becomes i m porta n t . Path M T U d iscove ry, as previ ou s ly descri bed , i s T h e source address selection i s t\'pically d o n e w h e n l ayer, with learned i n formation stored i n the r o u tin g fu nction is a l so a va i l ab l e ro users of raw sockets a n d to tabl e . In t h e course of p rocessing a PT B mess a g e , t h e other n c t\vork - l ayer users s u c h as ICMPv6 . triggered b y ICl v l P messages processed i n the network transport l ayer i s notifi ed t h rough i ts control i n put , The source address selection fu nction takes as argu­ ( ct l i n p u t ) path This i s req u i red because the reception m e n ts a d estination add ress and a n optional i n terface of s u c h a n I C M P message i n d icates that the packet i n po i n te r. The latter is used wh en k11 own, bu t in the case transit has been d isca rd ed thus the protocol m ay need of i n i tiating a transport connection i t is n u l l . T he to take appropriate a ct i o n . In the case o f TCP, it i s d estination rm�l t i o n to be l e a rn ed �m d the S\'Stem to a u tocon ­ t:� b lc i s u p d a ted :�.s neccssa n·, b \ ' m e J n s of a ra\\' socket encom p�lSSeS the r( > i i O \I 'i n g i te m s : • Ro u te r d i sco\'en· • O n - l i n k prdi x d i sCO\'Cl'\' We dcs i gn cd :t • l n tedace artri b u n: con tl gu ra ri o n • State l ess �1d d ress c o n fi g u ration • the I Pv6 ra \\' socket's i n re r f1cc \\'ith the abi l i t\· to pass on \\' sp e ci fi c I C M P\·6 messages to u se r a n d to ri l te r e x tra n eo u s packets or p rotoco l s . The n d 6 h ostd d a e m o n sets a socket opt i o n to receive o n l \' n c i g h bm d iscm erv R.As. It then e x e c u tes a d is ­ p�ltch ro u ti n e t h a t po l l s the ra\\ ' socket, :1\\':J. i ti n g p:� c kc rs . vVhe n d �1 ta i s �wa i l a b l e on the socket, t h e d :�c­ S ra re h d add ress config u ra tion m o n d e termines the c haracteristics o f t h e messag�.:, The m e c h �w i s m tc l r delive r ing t hi s i n tornu ri o n t h e h osts i s the rou ter a d \' e rti se m e n t ro ( RA ) packer of crc1 tcs a d a ta structure to con ta i n i t, and c:�. l ls rhc ncc­ cssarv fu n c ti o n s to perform a u toconti g u ra t i o n . The the Neigh bor D i sco\'l: ry Protoco l . I n t h e t( J I I o w i n g d i spatch mod u l e , in add i ti o n to po l l i n g socket d escrip­ sections, \\'e d escribe the m e thods w e d e \ ' C ] oped to tors, s u p po r ts necessary t i me r m a n agem e n t fu n c ti o n s process these p ac ke rs a n d u p d a te t h e syste m . s u c h :�.s creation , d e l et i o n , and expirati o n . f i g u r e s h o\\'S t h e :t ppl i ca tion ll daemon design cen ter. Host Autoconfiguration Daemon To process these R.As, \\'C designed :1. hos t daemon Kernel Interface Data Structures c a l led nd6hos td , \\· h i c h rcsidcs i n the appl ic:�.tion space o f th e Di gi t: d U N I X operating S\'Ste m . We d e te rmined that J user- mode th e m o n \\' l l o\\·­ I n nJJnv WJ\'S, the data l i n k i nt e rrace is t h e foc u s of I P\ · 6 :� u roco n riguration support. The k e rn e l d a ta struc ­ tll iTS r( > r ! P\·4 i n te rfaces arc n o t su ffic i cnt m e n ted llC\\. i n tert:Ke data structures t h a t e nopsu l a te d • A user- mode daemon \\'O u ld a\·o id kern e l bloat. • M a i me n �1 nce a n d exte n s i b i l i t\' would be c1sicr. • The nmction is not pe r rorm a nce cti t i ca l . the o i sti ng I f\·4 stru ctures . This a l ltl\\nl u s to :1. \ · oi d a rccom pi l a ti on o f the ex isti n g d a ta l i n k d ri\·crs on the D i g i ta l U N T X operati ng S\'Ste m . In t h e fu t u re , \\'e \\' i l l a ttem p t J design in \\'h i c h the i n terbce s tructu res rclr The a ur o c on tl g u ra ri o n process i n g is i m ple m e n ted as a si n gle executable i mage , as a cohesive set o f tig h tlv cou p l e d mod u l e s . T h e d:�.emon c ur re n t l y i s d esigned as a si n g l e - t h readed appl ication that uses a d i sp:nc h mech�1 n ism to c a l l each spec i a l i zed function mod u l e i n t u rn . We w i l l c x :1 m i n e t h e i d e a of b a \ ·ing t h i s d a e m o n r u n as a m u l tith re a d ed a pp l i ca tion i n the rtl tlii'C . The n d 6hostd daemon commun icates "·i th t h e nct\n>rk su bs\·srcm i n the kernel th rough m u l ti p l c tec h n i q ues. F i g ure 10 s h O\\'S the au rocontigmarion process i n g mod u l es . 'The Ll\\' socket i nt e r race is used to recci \·e 1\As, :md 1/0 control messages ( ioctl s ) :�rc ROUTER D ISCOVERY i ON-LINK PREFIX DISCOVERY i u sed I l'\'4 and I l'v6 arc completely i nt e grated . As shown in F i g u re 1 2 , structure to Ethernet, ! ' P I' , s u pport l o o pb a c k ) J l'\·6 :1 d d rcss co n ti g u re d (e.g., existing in the d :� t:J. str u c tu re mell t l i n k - h vf'r option of the RA is then added Digjul 'Ji:l'hnical Jounl:ll to the ro u t i n g r.1hk, 17 AUTOCO N F I G U RATION I N T E R FACE AT TRIBUTE PROCESSING U S E R SPACE IOCTLs SIOCIPV6ADDRTR SIOCIPV6DELRTR SIOCI PV6 1 F I N IT SIOCI PV6A I FADDR SIOCIPV6SI FATTR - - - - - - - K E R N E L SPACE SIOCIPV6 G I FATTR SIOCI PV6MIFADDR SIOCI PV6GIFADDR SIOCI PV6D I FADDR - - - - - - - - - - - - I PV6 I NT E R FACE CONTROL M O D U LE A D D R ESSES AND STATE ON-LINK P R E F I X E S R O U T E ATT R I BUTES DATA LINK ATTRI BUTES F i g u re 1 3 l nrcrt:Kc Artrib urc Upd � rcs e d e leted ,,· h u 1 .1 I l l'\\ 1\. \ prc�c n h t l 1 c J'i'C tix \\ i t h a li ktimc of zero . In that c.t s c , the o n - l i n k prdix mod ule will stop the rimer rou t i n e and de lete the 1 l 1 e 1 > I L' I I .\ I I ' 1 1 1 1 I I J c l l l l l l l l l t!. i .t i J I L- . \ l l l . ti . '" ad m i n i s trators wi t h c o n t rol O\'er the usc of ne t\\'ork add resses without m a n u a l i n tcr\'c n tion of each host o n The daemon :-�dds the prdix to the routing ta ble. I l ' !l' i .d I c< i l l l l l . d ' " " " " d pcrmi ttcd ( as comm u11 ic:� tions r(lr u s e rs need tig h t control over add ress co n fi g­ rile network. The state l ess model i s i n tended The l e n g t h of t i m e the prdi x i s \'d . h !j l l l 'l' 1 -! o,IH I\1'� t i ll' l c l c i i C " . \VIwn rlw c� w m n n ,.,,,-,,;,., . , a u tocontigu ration d i a gr:-�m . \U , 1 he L Ll c n H l l l ,·:1 1 1 �n ' R A _ � n ci r h c A - h i r is me t h e p rc l i x D prodded ro p e r f(Jrm stateless a d d ress conti guratio n . T h e d ae mon us es on-link prefix( es) provided in tl1c IV\ to con tlg u re .hldr�.·\,1.'' h > 1' .1 1 1 i i l k l' f .l c'I.O . A l ci i'Cl.'c' .1 1' ' c. I'C.l l c ..l , the ADDRESS AUTOCONFIGU RATION DAEMON STATELESS AUTOCO NFIGU RATION: USE ON-LINK PREFIXES AND I NTERFACE TOK E N STAT EFUL AUTOCONFIGU RATION: PRO CESS CONF IGURATION I N FO RMATION. START DHCPV6 CLIENT l LOCATE DHCPV6 S E RVER DHCPV6 USER SPACE - - - - "'""'ere - KERN E L SPACE -l ":e 1 OCTL ADMIN ISTER ADDRESS CONFIGU RATION STATE FOR THE RECEIVING I NTERFACE Figure 1 4 Address A u roconrigur,nion d e l e te d , or u pd a ted 011 the i n tcrbcc based o n t h e pre ­ ri :xes ,1 n d l i k ti m cs rccci ,ni i n the RA packe t . I Pv4, w h i c h represe n ts a \ C rv l a rge i il\'esr m c n t a n d a pote ntial pool o H Pv6 appl ications. Seco n d , t h i s A P I is perhaps in the most wid espread usc i n the i n d ustrv ,md Interactions with Stateful Address Configuration \Vh e n the d J e mon rccc i \TS '1 n RA, a n d the m an aged bit rbg is set, the host c.m u se st,nc fu l ,1d d ress c o n ­ ti gur:tri o n , u s i n g D I--I c : I'\'C) . D H C P,-6 i s i m p l e m e nted as ,l se p,lLlte Lhenwn process in our protot\·pc . is avai l a b le on a ,,·i d e \',l r i c t\' o f p l a t r( m n s : t h e bendits of sta n d ardization :ti-c com pe l l i n g . DNS AAAA Support D :"-J S prm·i dcs suppor t fo r nnp p i ng n ,1 m e s to IP D I--I C : l'\6 d di n cs 'l c o m p l e te Ill'\\' m o d e l ti·om t h e a d d resses a n d m a p p i n g I P �1 d d rcsscs h,Kk to t h e i r cor­ n i s t i n g D H CI),.4 i l l l f1 l e m c n t,n ions i n the i n d ustn· to respo n d i n g names .''' ' l' h e t\'f1e A resou rce recmd is d\'llJill i c a l h- c o n ri g u re ,ld d resses. The usc of l i n k - local add resses, m u l tiClst, ,1d d ress con h gu rari o n , and i n hcr­ Cllt su pport r(>r dV!l,l l l l i C r e n u m b e r i n g o f h osts i n I p,-6 ca u sed '1 lle\\' a rc h i tec t u re J n d d e s i g n i n t h e used to h o l d an I P\'4 .1d d ress. S i n c e i ts size is ti xed a t 4 b\·tes, a nt:\\ resou rce record n-p e , AAAA, \\ as d di n e d to h o l d I Pv6 ,1d d resses.'- The D i g i o l LJ :--\ 1 \ J P,-6 protot\•pe i n c l udes �� \\' i d e l \ - used l l l l p l c m e il t'Hion D H C : l'\ 6 spec i ri cn i o n . A c o m p a rison o f t h e a r c h i te c ­ o f the tural ciLmges be tween l! I--I C I'v4 a n d D I--I C : P\·6 c a n b e D o m a i n ( B I .'! D ) , ,,·h i c h 1 1 ,1s been m od i ried D�S k l l 0\\'11 '1s Bnke l e v r( m nd i n the D H C : l'v6 spec i ricati o n . ' AAAA record s . Application Services Address Manipulation Routines I nt e rn e t ro N .1 m c s u p port A t\'pical I P i m p l e m e ll t.ltion pro,·i d cs se\·e ra l l i hr:-� n· M ost TC: l' ,m d U D l' 'l p p l ications can be used wi t h rou t i n e s r()r m a n i pu b r i n g I P ,1 d d resse s . Th ese i n c l u d e I Pv6 with rc Ln i vel y m i nor mod i ricati o n s . The pri mary rou ti nes r()r conve rti n g ,1d d resses bcrween hi narv ,m d issue is the h rgcr add ress s i z e , both r( > r i n t e rn J I storage tex t u a l representations ,l l l d ro u ti n es r( J r tLl llSi:tti n g n eeds in rhe appl i cnio n ,m d r(>r add ress tra nsre r across n ames to a d d resses and .1ddresses to n a m e s . Ne\\' r o u ­ svstcm i n terbces. I n t h i s secti o n , we review t h ese t i n es h a d to be provi d e d t o pe rt(>rm th ese fu nctions issues a nd others . fo r J I'\·6 a d d resse s . 'fi le D i g i t :� ! UN I \ 1 1'' 6 prorotvpe provi d es the API routi nes d escribed I n terface E x te nsions r( > r in " g ,1sic Socket I 1\·6." " ' Am· A P I c u rren th- i n use r( > r I p,-4 could be m od i ri e d for l l'v6, b u r o n l v the BS D sockets A P I is bei n g i m·es­ inetd Daemon tig:tted wi t h i n the I ETE r( > r t\\'O reaso n s . "-'' F i rst, l a rge The i n etd d a e m o n c reates sockets on h e h :-� l f o f a p f1 l icl­ n u m be rs of a p p l i c :n ions use t h e sockets i n te r face fo r tions, i m-ol<. i n g the a p p l i urions o n h· ,,· h e n needed a n d Dig:itJI Tcc h n i u l ) o u m .il ] \) pass i n g the open sockets ro them . W i th t h e a d Y e n t o f t h e A �_ l N 1.:,T6 soc ket tvp e , i n c rd w a s mod i f-ied to accept a new a p p l i ca ti o n u m ri gu r a ti o n option i n i ts co n f-i gu ration t-i k . T h e kC\wmd i n c t6 i s u sed to i n d i cate a n �1 p p l i cnion that ,,.�1 1 1 ts to usc A r_ I N ET6 State fu l :td d ress c o n ti g u ra r i o n ll'i l l be i m p l e m e n ted as spec i fied in D H CPv6 �1 11d ll'i l l c o n ta i n :1 c l i e n t, a senu, a n d .1 rc Ll \ ' - :lge n r . This ,,·ork \\' i l l be tigh tlv cou­ p l e d \\'i t h d\·namic u p d �1 tes to D N S to pnl\'idc r n C (\\·orks that USC StJtC­ word ) i n d icates usc o f A r _ 1 1\ LT sockets . l css :�d d rcss a u tocon fi gurati o n , D HC I \·6 ,,·i l l b e a\·a i l ­ a b l e to configure other pmution JSSr I P v 6 . D i gita l Ius been acti\'C m od i fied ro s u pport l P\'6 , i n c l u d i n g Te l n e t and FTP . \\'i t h i n the I ETf \\'Orki n g grou ps t h ;n �lrL' d c ti n i n g the These programs were mod i f-i ed t o u s c I Pv6 sockets, ATM solutions. a d d ress structu res, and l i b r�1 n· routi nes. N o te that the I P\'6 now su pports rl ow i n f o r m �Hion in the I J\·6 ! Pv6 sockets a l so s u pport collllll u n i c a tions <)\'er I Pv4 , header a nd in the I l)v6 BSD socket so that a p p l i u tions n eed not lll a i nr ::t i n separate sockets i n h e re nt q u a l i t\'-ofscrvicc ( QO S ) m e c h a n i s m i n ! Pv6 r()r T Pv4 a nd ! Pv6, :ll l d a s i n g k c x cc u t:� ble i nugc c a n t\ P l s n· u c r u r c . T h i s meshes we l l wi th c ff-( l rts to su p p o rt reserve resources i n tcropcratc with b o t h tvpcs o f rc lllotc systclll . on a n c t\vork as speci fi e d in the Reso u rce Rese rvation Future Work n o n bmadCJst d a c1 l i n ks w i l l c ncom p<1SS �� d e s i g n cen ­ Furu rc i m p l e me n tation c fr(H·ts \\'i l l i n c l u d e securi t\', parameters ro u r i n g , mod e l fo r \'i d e o - :m d :w d i o - rc b tc d �l pplications :�cross Protoco l ( RSVP ) . ''' Usi n g RSVP over bro a d c:�st :� nd ter that s u p ports a wide range of resou rce rcscrv�Hion st�ncfu l add rcss con ri g u ra t i o n , dvna m i c upd ates t o D N S , I P,·6 m·cr P P P a n d ATivl , t·csourcc rcscn·�n i o n , �l lld scn·icc l ocation . I n add i ti o n , \\'C \\·i l l m:� i nt a i n a consiste n t p c r t( mnancc a n C t\\'Ork pr 1 P\'4, c r:� b i l i rv C\'CIHS, ,,·b i ch is a pr:tctic:tl and concentrated nU\' ben e fi t ri·om the grcltcr IC\'CI o f su pport r(> r h�lsic d't-<>rr to d e b u g the specif-i cati o n s I P \ - 6 scc u r i t\' s u p ports both the tec h n o l ogies, such <1S scc u ri t\· a n d m u l tiust capa b i l ities. and the cod e b:�se . f-( .)1' these ri. mctions \\' i l l sen·ice l o u t i o n h o l d s prom ise r(Jr I P\·6 and a u th e n ticati o n a n d the cncnvtion o f H\·6 pac kets e n d - to-end .�' The mod ­ ule reside i n the knnel :� n d most Sum mary l i ke i l' w i l l be ca l l ed a t the p o i n t \\'here the I P,·6 nct\n>rk D igital has designed a protot\'fX of l p, 6 o n the D igir:d l :wcr p�Kket is processed . A kc\' n1<1n.1gc m c m fra m e ­ U N I X operati n g S\'Stc m . Tec h n i q u es �md tec h n o logies ro work is be i n g d c,·c lopcd t o su pport b o r h a u th e n tica­ ha\·c been d c,·c l o pcd tion :� n d e n crvption . To access the kc\' m :tn:tge m c n t I P\ 6 arc h i te c ru rc ; in p r u n iust ro u t i n g . I n ad d i ti o n , c n h <1 11CCL11Ct1ts have been m a d e to l P\'4 s u pport Open S h ortest ( OSP h·6 ) and to p rm·ide m u lti ­ References l. P. G ross ;md P. A. l m q u isr, " ! ESC Dc l i bn;Jtions on Ro u r ­ ing :m d Addr�ssing," Rl C : 1 3 8 0 ( "Jo,·c m bc r 1 99 2 ) 2. S. B ra d n e r Jnd A. lvb n k i n , "The Recommend�nion fo r the I I' Next G e n e ration ( J uly 1 9 8 2 ) . R . H inden, " S i m p l e I n ternet Pmroco\ P l u s 'Wh i te P a p e r , " tu:c I 7 1 0 ( October 1 994 ) . 4 . S . D ee r i n g a n d R . H i nd e n , "Inrcmet Pro toc ol , Vecsion 6 ( I.l'\ 6 ) Sp c c i ti c atio n , " RfC 1 8 8 3 ( J�nuacy 1 996 ) . 2 5 . W. Stevens and M . T ho m as, "Advanced Sackers A P I for !Pv6,'' vVo rk i n progress ( October 1 996 ) . 26. cast Add ress A l location," R.FC 1 8 8 7 ( j a n u a r y 1 99 6 ) . R . Hinden a n d J . Pos t e l , " I Pv6 Tes t i ng Address Al loc at i o n , " IU'C 1 897 ( Ja n u ary 1 996 ) . 7. S . T h omso n cmd T. N c1 r re n , "1Pv6 Stateless Add ress Au roconfigura tion ," RFC 1 9 7 1 ( A u g u st 1 99 6 ) . 8 . J . Bound and C . Perki n s , " Dy n a m i c Host Configura­ 9 . A . C o n ra a n d S . Dee ri n g , "l nrcrner Conrrol M e s s age S u pport I P Ve rsion 6 , " RF C l 8 8 6 28. Discoverv [()!' l P Version 6 ( l Pv6 )," RFC 1 9 70 ( August 1 99 6 ) 1 1 . M . McKusick e r a \ . , The Design and implementation o/ the 4 . 4 BSD Operulinp, 5 ) '5/elll, ( Rea d i n g , Mass . : Addison -Wesley, I SB N : 0 - 2 0 1 - 54979 - 4 , 1 99 6 ) . cr a \ . , " C l ass l e s s I n t e r - D o mai n 1995). R . Atk i nso n , "Securitv Archirccrure tor the Internet Pro r o col , " ( Work in progress, J u ne 1 996 ) . Fu ncti o na l Specification," ( Wo rk u1 progress, August 1 99 6 ) . 30. J . Veizades e t a \ , Se rvi ce . " Loc a ti o n P r o tocol , " ( Work in p r ogre ss , J u n e 1 996 ) . General References 1 0 . T. Narten, E . N o rd ma r k , a n d W . S i m p s o n , "Neigh bor 1 2 . V. Fuller ( December 2 9 . " Resource ReSerVation Protocol ( RSVP) -Version Protocol ( l Clvl l'\·6) for r h e lnrerner P ro toc o l Ve csion 6 ( ! Pv 6 ) ," JU'C 1 88 5 (January 1 9 96 ) . " D o ma i n N a m e s - C on ce pt s v6 Sp e c i ti cn i o n , " Work i n progress ( October 1 996 ) . 18. K . S kl ow e r , p ri va t e com m u n i cation to lvb tt Thomas, September 1 99 5 . 1 9. R . G i l l i g:lll a n d E . Nordnurk, "Transition Mecha­ n isms to1· ! Pv6 H osts and Ro u te rs , " lU'C 1 9 3 3 (April 1 99 6 ) . 2 0 . S . D e e r i n g <111 d R . H i n d e n , " I P Vnsion 6 Add re ss i n g A rc h i tecture," RFC 1 8 8 4 ( J a n u a r v 1 99 6 ) . 21. R. G i l l i gan et a\ . , " Basic Socket I nrertace Extensions tor I Pv6," ( Wock i n progress, A p r i l 1 996 ). 2 2 . M . C ra w ford , "A Method tor the Transmission of ! Pv 6 Packets over Ethernet Networks," R.FC I 97 2 ( A u gust 1 99 6 ) . D a n i e l T. Harrington As a principal soft w a re engi neer i n Digita l 's l Pv6 Program Office, Dan H a rr i ng to n p a rti c i p a te d in the Digital UNIX I Pv6 prototype effort. P ri or to th i s the DECnct/OSI wo rk , he helped d eve l op prod u c ts on the ULTRIX and t h e Digital U N I X platforms. After j o i n i ng Digi ta l i n 1 98 2 , Dan worked in p e rto rma n ce analysis, field support, c1nd sofTware devel­ o p m e nt He reec i , ·c d a B.S. i n mathematics ri·om Rensselaer Po l y te c hn i c l nstitur.e. Dan is currcnrlv with Lucent . Technologies. Digit� I Technical Journal Vol. 8 No. 3 1.996 21 Matt Thomas James P. Bound J i m R o u n d is :1 c o n s u l ri ng; soft\\ :u·c ,· ng;i n c c r :1 1 1 d rhe re c h ­ n i o l d irector tor l r\ 6 \\ i t h i n t h e 1 1'1 (> l'rog;r:Hn O ftice . ) i n 1 is rL·spo m i h l e f( >r t h e Ol n.l l l ;�,h .mccd dc l e l oj> l l l c n t ;�rc h i ­ in C1 l i ti m 1 i :1 A l t hou gh h e i s :1 J>ri n c i pc1 1 soft\1 :11-c e n � i n el'l" tec t u rc :1 n d re kr�nec A l p h :1 Dig;it:1l L1 :-\ l \ code boi s e , " h i c h i n t h e OJ'e n \ ' ,\ ! S <; ,·ste ms S o ftll':1rc C r o u p , /vb tt Ius spcm tile: l :1st eigln 1 c:11·s :ls :1 d e 1 · e l opcr ol · I J C t\\ orking p rod u c ts i s :1 l so D i g i ta l 's I f: I T l l'1 6 re c l 1 1 1 i ct l lc:l d n :md o n e o t" t l1e h i s o n go i l l ):'; i m o i H· m c n r "·i th D i g; i t .d l' :-\ I X ! 1'1 6 c t'ti ll-rs, ,-cri fi e s t h ;�t t h e 1 1',·6 S f1C C i tic:Hiom :11·c i m p l c nl c l l tol h l c . H e l l'1 6 ad1·cmced d c 1 e l o p m c n r e n f'i l l C C I'S on A l j > h :l ) ) i g; i toll l ' :-\ 1 \ . I n 1 99 3 , J i m lx·�:ln h i s p.l rt i c i l' " t i r m i 11 t h e I E I F t o " ork on t h e I P n f' m : d t h e :1d1·:u 1 ccd d e 1 e lop 1 1 1 e 1 1 t l l' n g; p r o r o n v e . . \s :1 !ll c m hcr o i ' t h c I LT F \ l l' n � D i rccror.ltc, J i m h e l jlL'd determine the rc q u i rcme 1 t rs :md core :t rc h i t c·c r u n: ti>r I Png's I me rntr p ro toc ol :111d re b red fu ncrion:1lm to s u pp or t l l' n g . T h e rc , u l t \\";l s r i t e s e l ec t i o n o f :1 proJ>os.l l , 110\\ K n ll\\ 1 1 cci li ca­ r i o n s :md :1 C O i l r l· i ll l l t i ll):', cn1 r h o 1· tt> t h e hook ! ! 'I Ii'. /uicrucl Protocol . \ ex! Cir·uc;-r;!/!J / 1 . H e i s a 1 1 1 c m bcr of the I F. F. I : a n d the I n t e rn e t SociL·n·. John J. McCan n ):KK McC1nn is :l J >rinci J >al sotr\\·:1re c· ngineer i 1 1 the LIN I \ F n gi n ec ri n g G roup a n d cl rn c m b e r of t h e I J',·6 projccr rcu n . H e comri bu rcd t o t h e d es i gn :md i m p l c m e l l t:ltion of t h e D i f'i Lll U N I \ l l'\'6 protot\'Jll: , i n c l u d i n g; ro u te r d i scmc 1·1·, a u toco n ti g u ra t i o n , ti·agm e n t:t r i o n , re.\Ss c m bh-, p:nh MTL · d i scr >�-c rv, l ( >rw:lrd i ng . :md t h e I J'1·u .\ 1'1 . H e J>:1 rt i c i p:ltcs i n sc1·cr:1l J ET F wmki n� gro u f1S .1 11d is :1 co:� u thor o f I n te r n e t RFC l 9 1::i l , " P:nh ,vlT L; D is c muv �()!' I l' 1'C1's i o 1 1 6 . " j :tck j o i 11 c d D i gi t:1l in 1 98 1\ to become :1 1 1 1 e m h n of the D i srri h ­ Llted S1 stc rm Tec h n i c J I F1·:d u .nion C m u p . H e ,1 lso \1 or ked in the n �: C n e r/OS I li > r OJ>c n V M S r: n g i n e cr i n g C ro u p bctorc t:l king h i s c u rrc n r posi r i o n . H e recei1·ed :1 B .S . i n eOlllJ1Li l e r science ( lll;lgJu c u m laude ) ti·on1 the t : n i1·crsitv of Lo\\T I I in 1 98 8 and a n ,vt . S . i n compu ter scie nce ti·o 1n Bosto n U n i 1·ersin· in 1 99 5 . 22 . r\.h t r , 'h o ll l articipant i n 1·ariow, l FTl. " · orki r 1g groups :md is :1 co:Hi thor ot'sc1 n.1l l me m c: t Dr:1tls. I MaA"well M. Burnet Robert M. Sup n ik Preserving Computing's Past: Restoration and Simulation Restoration and si mulation are two tech n i q ues The Computing Past for preserving com puting systems of h i stori cal i nterest. In com puter restoration, h i storical sys­ tems are returned to work i n g cond ition thro u g h repair o f broken electrical a n d mechan ical sub­ systems, if necessary s u bstituting current parts for the origi nal ones. In com puter s i m u lation, h i storical system s are re-created as software prog rams on current com puter systems. In each case, the operating enviro n ment of the origi nal system i s presented to a modern user for i nspec­ tion or analysis. T h i s d i ffers with com puter con­ servation, w h i c h preserves h i storical systems i n their cu rrent state, usually one of d i s repair. The authors argue that an und ersta n d i n g of computi n g 's past is vital to u n derstan d i n g its futu re, and thus that restoration, rather than just conservation, of h istoric system s is an The conri n u o u s i m p rm·cmc nts i n c o m p u t i n g te c h n o l ­ o�· Cl usc the r;lpid ohsolcsccnu.: of' com p uter S\ ' S tr m s , a rc h i tectu res, m e d i a , and dn·iccs. Since old c o m p u t­ i ng S\'Stcms ;Jrc Ll rch- fK rc c i , -c d ro h ,l,·c am· , - a l uc , rhc danger o f los i n g p o r ti o n s of the compu ting record is si g n i tic m r . 'vVh e n a com p u t i n g a rc h i tecture bec o m es e x ti n ct, i ts so ft\l',llT, d a ta , and ll't' i rtcn ;m d o ra l r e cord s oti:cn d i sa ppear 'll ' i th i t . O l d e r co m p u te r svste ms e m bodv major i m·estm c n rs i n soti:ware, the \'J l u c o t \\'h i c h ma\' pe rs i s t long a her t h e S\'Stems J J a,·e lost their rcclm il' process o f restori n g a particu lar P D P - 1 1 mi n icom ­ p u ter. The se co n d section d i s c u sses the s i m u l a t i o n of old com p u ters on modern S\'Stems. 1 t d e s cri bes a s i m u l a ti o n framework c a l l ed S I M , ll ' h ic h h:�s been used to i m p l e m e n t s i m u la tors ri:>r the l' D P - 8 , P D P - 1 1 , P D P - 4/7/9/ 1 5 , :� nd No1·:1 m i n i co m p ute rs . Restoring Old Computers Since the com p u te r b c u m e �• ma ss - p rod u c e d i te m in the late 1 960s, i ts t\'\)ic a l l ik c1 cle has consisted of i ni t i a l i nst�1J iatio11, re ll t a l or d c p r c c i �H i o n re te nti o n fC>r a b o u t tJ I'C \'Cars, and usc r(Jr a k11· more 1'c1 rs ( j u s t in c1sc ) , and th e n retircmcllt :md J t ri p to r bc refuse d u m p . There is o n l v J brief window of opportu n i ty to col lect o l d com­ p u ters at the end ohhcir working l i k . O nce that wi n ­ dow i s dosed, t h e c o m p u ters arc gone r( J re v cr. The Australian Museum Collection I n Syd ney, A u sr ra l i�1 , t h i s window of opportu n i ty ti rst became �• pparellt in 1 9 7 1 , when the e a r l y P D P s y s te ms IT :Jched t h e e n d s o f their l i re c y c l e s . D i g i ta l ' s Austr:� l i a n s u bs i d i�1 r v bcg:1n c o l l e c t i n g svstcms b y a c rea tive p rogr: un o r· tr:Jd e - i n s t(Jr new eq u i p m e n t . ' I t w a s c sp c c i a l l v urge n t to o b tJ i n e x �1 m plcs of the 1 2 - bi t , 1 8 - bi t , ;md 3 6 - b i t l' D l' s e ri e s , �•s rhev ll'cre rc l a t i l - c l y bv in n u mb..:r. Table I l i s ts rh..: percen t�r d i ;�gnostics. Thne is a nnwork of e n r h u si�1sts who c:�n h e l p at this stJge . :� ro u n d restore �1 1 9 60s m ac h i ne a rc as t( J I I ml -s : • I nspect the h ard11·:�rc rcll· or t h e 11 o r l d 26 • sak t�·, r �1 r ri c u l ;�rJ,· \'ol . li :\n 3 Po\\ ' c r u p t h e com p u ter, checki n g s u pplv 1·oJ tagcs, ;� n d front consok t(Jr s i g n s ofl i k . U s e si m p le ro u ti n es at t h e s11· i t c h register to c h e ck 1 996 Fit a s e ri a l l i n e u n i t so t h a t �1 VT or �1 'Te l c tvpe co n ­ sole c a n b e used . • I 9 7 0 s l' i nta ge the heal'\' d rJII'e rs J n d s l i d e m ec h a n i sms. nu1· open Assemble :1 min i ma l s 1 ·s re m o f C P U , me mon·, :m d console swi t c h r e g i s te r �(J r l l l i tial te st s . t:lns, • p h 1 ·si u l peop l e rl:x elcme nt�1 r\' o p e ra ti o n . O nce the " i n gred i e n t s " h a v e been co l l ec te d , t h e steps n e ed e d to • • Co l l ec t h:1rdw�1re m:m uals ;�nd s c h e m a t i c s . p r i n c i p a l criterion i n restora ­ the back door. :1 S \ ' S t cm C o l lec t d i agn o sti c �1 11d opc r �H i ng software o n origi­ ;1 tion, since so m ed a v n o n te c h n ical Col l e c t t h e h a r d 11·�11-c, i f p o ss i b l e two or i d c a lh· to work on a n d a s pa re , as w el l JS the a bi l i n · to m a ke • Careful II' i nspect t h e po11 c r SI'Ste m , h i g h - 1 oi L1 g e sources can ki I I . AI though most o r· the pm1 n 11 i r i n g is n o t essemi a l . ) • Ph, · s i ca l k assemble t h e h a rd ll'�1re, c h e c k i n g mod u l e allocations, u b l i n g , e tc . goocl' m es s . I t s h o u l d be re m m ed as s oon �1s possible ; G e t t h e k e 1 · hoa rd e c h o i n g to t h e screen 11 i t h s i m p l e ro u ti n e s . m pri mer I f t h e y arc ;1 1·a i L l b l c , r u n t h e i me m �1 l tests of t h e read - o n h- m e m o n · ( ROJ\11 ). 1 I } } } RX01 DUAL 8-INCH FLOPPY D I S K ETTES TD8E TU56 ACCU M U LATOR TRANSFER DUAL DECTAPE SYSTEM PCBE 300 CPS READER. 50 CPS PUNCH PA PER TA P E PDP-8/E C P U WITH EXTENDED ARITHMETIC ELEMENT. 1 6K WORDS MEMORY. KLBE 2400-BAUD CONSOLE. KLBE 2400-BAUD COMMUNICATION PORT. DECTAPE BOOTSTRAP. RK05 DISK BOOTSTRAP. R EAL-TIME CLOCK RK05 REMOVABLE 2.4-MB CART R I DG E DISK }} } STORAGE RACK FOR 1 0 DECTAPE SYSTEMS H861 POWER DISTRIBUTION F i g u re 1 P D P - .S/1'. C : o111 p u rc r SI'StL' l l l Com e n tion:�l "·isd om \\ O u l d no\\' �Hh ise t h a t a l l t he bi l i t i s q u es t i o n a b l e , hm,·c ,·cr, ;l nd the proced u re i s d i a g nos t ic rou ti nes be ru n . H o"·e,-cr, d i a g n ost i cs m:re ted io u s . M a m· d i ;1 g n ostics \\'ere 0 1 1 p ;1 pe r t;l p e , b u t ( p h i losop h i ca l ! \· ) �1 ��\ �1\'S used ous ! \ · g oo d m a c h i n e ; they :tre to ti n d b u gs i n a pr ev i ­ too co m p l ex when h u ge c h u n ks o f t h e rm c h i nc m i g h t sti l l be m issi n g . The most f1 L1C ti c : d next stq1 i s to ge t m :tss stor:�ge o n - l i n e . D e p e n d i n g on t h e m :� n u t�1 c r u r e r, t h e target device Y u s u a l J v the q u i c k e s t t e s t i s t o l o a d J co m p l e t e p;1[)Cr S\'Stem ( such as fOCA l . t()r D i g i u l 's S\'S te m s ) I h h c diagnostics run, the svstc m is prolx1 bh· fu nctio n a l . Once the CPU, c o n sol e , Jnd m c morv ;lrc \Triti c d , ad d i t i on a l p e ri p h e r a ls c a n b e ;J dd ed , one ;H a rime . I t 111:1\' b e a t-loppy d i s k d r i 1·e , a e :� rr r i d ge h a rd d is k d ri v e , pays to tJke t h e ti me a n d c ft( > rr to rcsc1rch bus rape . With a work i n g mass a d d ressts, i n ter r u p t \'CCtors, po\\'c r s u p p l y load i ng, or some t(m11 of m ;1 g n e r i c storage device a n d �1 boo t s t r a p rou tine, it beco mes poss i b l e or to hoot a si m p l e operating system ( l i ke OS/8 RT- 1 1 t( JI' D i gi t a l ' s svsre m s ) . This q u i dd v sh ows whet h e r t h e m ac h i n e is \\' or k i n g or not. I f �1 mass storage de,·ice is nor :w:�i b blc , t h e next best t h i n g i s papn t:�pc. T h i s C1I l be e i t h e r the S\'Stem's LlC k - m o u ntcd IT;1 d e r ;md p u n c h o r the paper t a pe rc1dcr on :111 AS R 3 3 or ASR3 5 co ns o l e . The re l i a - and mod u le pbce m e n t, and to ke e p :1 l o g book \\'ith con ti g ur at i o n d i a gr a m s :�nd resu l t s . ! 1 1 gene ra l , if t h e conti g u ration rules a re t(> l l owed , t h e i te m s \\' i l l w or k . T here arc Jl:w e l e c t ron i c tJ i l u res, e\·cn i n 2 0 - or 30- ycar- o l d mod u l es . ·when �1 J1ro b l c m ;1 rises, i t is u s u al h­ add ress \'ector stra ppi n g, p lws i u l d a m ;1gc, or m i ss i n g cables. Corrosion of bo;lrd contacts c a n be :1 prob l e m ; thev s h ou l d be d cm c d \\' i t h Digiral Tcc h n ic J I j o u m ;ll :1 c l c :� n c l ot h or G l rd board \'o l . � '-:o 3 l '>96 27 Table 3 Ta ble 4 Goa l s of t h e Austra l i a n D i g i t a l M us e u m D i g it a l Data M e d i a from 1 9 60 to 1 99 6 To preserve o n e of each m o d e l of D i g i t a l 's com puters P a p e r t a pe To keep each major D i g it a l o p e ra t i n g system wo r k i n g 80-co l u m n p u n c h ed a nd m a r k sense c a rd s To h ave a work i n g u n it o f e a c h D i g i t a l term i n a l , con­ 7-track, h a lf-inch m a g n e t i c t a p e s o l e, and PC 9-track, h a l f- i n c h m a g n e t i c t a p e To prov i d e convers i o n a n d a r c h i v a l f a c i l i t i es for o l d media To p reserve s i g n i f i c a n t D i g i t a l l itera t u re a n d m a n u a l s To prese rve a VAX - 1 1 /780 comp uter a s t h e o r i g i n a l u n it o f 1 V U P To d i sse m i nate i nstruct i ve a n d e d u cat i o n a l m ate r i a l D E Ct a p e a n d LI N Ct a p e syste m s A u d i ocassette D E Cta p e II c a rt r i d g e (TU 58) Com pacTa p e (T K 5 0, etc . ) Q u a rter- i n c h ca rt r i d g e t a p e D i g it a l a u d i o t a p e To e d ucate a n d a m use o u r sta ff, o u r custo m e rs, a n d the public 8- i n c h f l oppy d i sk To s u p p o rt t h e D E C U S N O P (nost a l g i c obsol ete prod­ uct) S p e c i a l i n te rest G ro u p 3 . 5 - i nc h floppy d is k To p reserve spa res, too l s, test g e a r, a n d docu m e n t a ­ R K 0 5 r e mova b l e d i sk t i o n to k e e p t h e c o l l ect i o n work i n g 5 . 2 5- i n c h f l o p py d i s k R K06, RK07 remova b l e d i s k To prese rve a nd protect these treasures f o r fut u re g e n erat i o n s R L0 1 , R L02 rem ova b l e d i s k R P 0 1 . . . R P06 remova b l e d i s k R M 03, R M 0 5 remova b l e d i sk RC25 remova b l e d i s k ( for example, a busi ness urd ) , not \\'ith a pencil cr�1scr, \\'hich l ewes resid ues. S i l icon components appca1· to be \'Cl'\' stable and a tri b u te to the conscn·ati\'C d esign pri nciples of carlv computer engineers. The main components that scCill to age �li'C power supplv capacitors, bns, and l igh ts . The fi l ter capaci ­ tors across the high-voltage sou rces can short, and reference electrolytic capaci tors in power supplv regu­ lators can dry o u t . Although the large c1pacitors in power supply RC fi l te rs have prc)\'cn to be reliable, some restorers replace them as a m atter of course r(>r sarctv reasons. Sm Expniencc p roves that tl1ey arc . A classic PDP-8 system restored in 1 9 8 8 still turns on happily ( u n touched ) eight years l a ter. A fu l ly c on fi g ure d P D P- 8 / E system is sti ll work­ i ng tour years after restoration . Restoring a Minicomputer: A Case Study An o n go ing project is the restoration of a large, U N I B US - based P D P- 1 1 system with many U N I B U S perip herals attached to i t . The project was started us ing the origi n a l P D P- 1 1 /20 CPU. S i nce many P D P- 1 1 peripherals were d esigned long �1h-cr the P D P - 1 1 /2 0 C P U , it could not cope with single- board d i rect memory access ( DMA) d evices, mct:d-oxidc 28 Digit:�l Technical j o u rn a l Vol . 8 :--: o . 3 ! 996 semicond uctor ( MO S ) m c morv, and other l ater in\·e n ­ tion s . T h e project rd(>cuscd on t h e 1J1id- r�ngc I' D P- 1 1 /3 4 , \\ hich in retrospect h �1s prm cd \\ isc. The P D l' - 1 1 / 3 4 su pports MOS mcmorv, h as an LED and p u s h - b u tton consol e , and represe n ts a mature i m p l e ­ mentation of the P D P - 1 1 i nstruction s e t . I t h as a n optional cac he, battCI'\' bac kup, rl o�l ting-point op era ­ tion, �111d the extended i nstruction set ( E IS ) . The cmrcllt conriguration occ upies three large c:Jb­ i ncts in \\·h:n used ro be the d in i ng room or· Max B ur n et's house. The \'irtucs of the U N I B U S �11-c m:my; i n parricu lar, i t J l lows mod u b r connection of l/0 dC\·iccs and other componcms. H m\'e\'Cr, 1/0 d evices of the e ra ofte n we igh 1 0 0 pounds and :�re mounted in 1 0 - i nc h d rawers; their s heer p hysical size and weight arc disi ncent ives to reconrigu r:n io n . The project c u rrenrly u ses rhc RT- 1 1 operating system bcu usc of i ts simpl icity and extensive device d rivers . Evc n t ual lv, it may be poss i b le to r u n the RS X - 1 1 M �111d the RSTS/E syste m s , but there is l i trl c to gain h·om a media conversion poin t of view, beca use l\T- 1 1 includes u ti l i tie s tor d ea li n g with r(>rcign fi l e t(mnats . The nuin d i Hi c u l tics cncou ntcn:d h:wc been associ­ ated with the power supply: the DC l ow sigml threads i ts way thro u gh even' periphera l . The �1 bscnce of U N I B U S gra n t comi n u i ty cJrd s can c reate h a\·oc. Since this P D P - 1 1 system is very L l rge , it is stra i n i n g the design ru les conccming r1 oating \'Cctors , cu rre n t load ing, and bus l oads . The C : l ' U � n d mc: mon· . to check . caS\' . arc re l a t i 1 c l1· ou t . D ue to the ,·c rsa t i l i ry of t h e U N IB US, holl'ever, checking out the 1/0 S\'Stcm i s ,·en· labor i o u s . Starti ng ,,· i t h programmed 1/0 te sts works best, tol ­ lo11ni lw i nt e r r u pt tests, a n d ri na l l v DMA or non ­ processor rch:rcnce ( N PR) tests. Ex p eri e n ce s hows that rests need to be: re r u n whe never l loll'ing: • proposed d es i gn . For e x a m p l e , prior to ta bricari o n , a l l modern m icroprocessors a re extensively s i m u ­ la te d , fi rst a s a bstract p c rtcmn a nce mod e l s a n d then CR1 1 p u n c h ed card read e r a t i n c reasing levels of deta i l . • r u n i n h i g h l y constra i n e d envi ronments s u c h as e m b edded systems. S i m u l r program d e b u ggi ng, i t becomes a usefu l rooJ for d e b u gging programs t h a t rhcv all'air i nsta l l ation r ime a n d cft(>rt: goa l ' D e buggi n g r()r embedded syste ms. tor contains Although the to l loll'i n g peri p h e ra l s arc al'<1i 1 J b l c , eve n t u a l Design of n ell' S\'Stem s . The sim u l ator m i m ics t h e beha1·ior of a fu t u re chip or com p u te r S\'Stem a n d i s PC 1 1 p :� pe r t:�pc read er a n d pu nch The provide more e l a borate f:1 c i l i ties to su pport pertormance i nstru mentation or debugging. Seri al line u n i ts tor conso l e a n d seri a l p ri n te r • • l/0 completion, t(>r wa1· to sched u l e events, s u c h as later c\·a l uari o n . It c:�n :�!so i m p l e m e n t other time­ used to understa n d :� n d d e b u g t h e beha1·ior o f the • • tations of ad1·a ncing ri m e, s u c h a s t h e n u m ber o f well as basic control comma nds s u c h as start and stop. is r s tra i g h tto r wa rd repl ication or l as t grea t a l most e ver y p l ayback of com pl icated seq u e nces, r e m ovi ng the Ti me w i l l te l l . ra ctor t h a t o ften plagues the d e b u g ­ randomn ess gi ng of asy n c h ronous software on real syste m s . Simulating O l d Com puters • Preserl'ation of past sof t w a re . S i m u lators can pro­ A si m u l a tor is a com p u t er progr:1 m o pc r:ni ng o n o n e c o m p u te r s1·srem ( k11011 n as the h os t S\'Ste m ) ll' h i c h vid e m i gration assistance in rbe transition fi·om older m i m i cs t h e be h a ,·ior of a n o t h e r c o m p u ter S\'Stem systems hal'c prm·ided simu lators tor older arch i ­ ( k no11 1 1 ;ls the t:1rgct svsre m ) . The s i m u l a tor's d at :l is th e state of rbe target c o m p u te r syste m - registers, to ne"'er architectures. Manv tra n sitional computer tectures, t y pi c 1 1 l v at the m i c rocode lel·e l , r o assist c u sto mers r d ecod i ng i n structions i n si m u l ated m c m on· and p er r( m n ing the sp cc i ri ed :� I t e rations of sim u l :1ted m ac h i n e sta te . Th e ex ec u t ion e n gi n e keeps track of s i m u l ated time in a rbi ­ tr:ll'\' u n its, ll'h ich n1av be precise representations of the fe r ln e l .len: I s o f si m u lation are register tr;lns­ ( RTL) , i nstructi o n , and software speciti c . An RTL si m u b tor atte m p ts t o m i m i c t h e major h a rdware blocks of the target S\'Stem and to i m pl e ­ ment i ts actual l og ic e q u ations. The goal i s absol u te Di[; ir�l Tec h nic� I )ot tl'll,li Vol . 8 No. 3 ! 9 96 2\J tid c l i ty, the test o f ll' h i c h is t h a t 110 11 iccc o f softll'�l rL· General Design Considerations The d esign of Jn r u n n i ng on the s i m u l a to r s h o u l d bch l l o11·s processor- mcmon· s11 i t c h ( PMS ) Ne11·cll a n d implc­ t h e a c tu a l accc l cr;H i o n c u n·c o f <1 ] ) 1-: C :ra 11e srm:�gc s r r u c t l l rc proposed lw B e l l a n d s1·stc m ) and access to impkmcn ta tion d o c u 1 n c m�1 tion m e med i n M I M I C :t n d cou n tless other s i m u b rors :1 t h Cus on the i n te n t i o n s o f rhc t i o n . The ,11 <1 i b b l c co m m a n d s i n arc h i tecture spcc i tic:�tio n . As <1 res u l t , i n struction s i m ­ Ta b l e S . F i n a l il·, a sofu,·arc-spccitic needed lw most s i m u i ;Hors, s u c l 1 as e1·cm q u e ue m a i n ­ s i m u btion ti 1 r thcr a bstracts rht: ti.1 ncrions o f the target s1·srcm to on I I · those needed L11' J par ticular piece of t:�rgct s1·srcm soful' 3 l l! % Table 5 C o m m a n d s Ava i l a b l e i n S I M Com mand Def i n ition attach < u n it> Assoc i a t e f i l e with u n it's m e d i a . detach < u n it> I A L L D i sassociate u n it's (a l l u n its) m e d i a from a ny f i l e . reset < d e v i ce> I A L L Reset d ev i c e (a l l d ev i ces). l o a d Load b i n a ry p ro g ra m from f i l e . boot < u n it> Reset a l l d ev i ces a n d bootstra p from u n i t . r u n { < n ew PC>} Reset a l l d ev i ces and res u m e exec u t i o n at the cu rrent PC {or n ew PC}. g o {} R e s u m e exec u t i o n at the cu rrent PC { o r n ew PC}. cont R e s u m e exec u t i o n at the c u r r e n t PC. step {} Execute one i n struct i o n {or n u m b e r i n struct i ons}. exa m i n e < l ist> D i s p l ay co ntents of l i st of m e m o ry l oc a t i o n s or reg i sters. iexa m i n e < l ist> D i s p l a y co nte nts of l i st of m e m o ry l ocat i o ns o r reg i sters a n d a l low i ntera ctive m o d if i c at i o n . d e posit < l ist> Store va l u e i n l i st of m e m o ry l ocat i o ns or reg i sters. i d e posit < l i st> I nteractively m o d ify l i st of m e m o ry locat i o n s o r reg i sters. save Save s i m u l a tor state in fi l e . restore Restore s i m u l at o r state from f i l e . s h ow q u e u e D i s p l ay t h e s i m u l ator's event q u e u e . s h ow co n f i g u ra t i o n D i s p l ay t h e s i m u l ator's co n f i g u r a t i o n . s h ow t i m e D i s p l ay t h e s i m u l ated t i m e c o u n t e r. s h ow S h ow d e v i ce's conf i g u ra t i o n o p t i o n s . s e t S e t a d ev i c e co n f i g u ra t i o n o pt i o n . help D i s p l ay a terse h e l p message. exit I q u it I bye Leave the s i m u lator. e a r l i e r. Llsrl1·, t h e marcri.1l is l i k c l \' to be contrJd i c tory, • Design doc u m e n ts . For S\'Stems that do not hJI'C I'Cl'\' b rge - sc a l c i mcgration ( V LS I ) , rile o n lv c x ra n r c m h o d 1 · i n g d i frc r i n g I"C\ is ions or 1nsions o f the a rc h i ­ r c c tu i"C , <1S 11 c l l as e rrors r h a r ha1-c n c p t i n d u ri n g t he de s i gn d ocume nts arc the l ogi c pri nts and t h e bi nr a pJ rtic u l a r su bs\'stem, such as m a i mc n :mcc - l cvcl katmes a n d other ti ne poi nts. Folklore . D u r i n g t h e usefu l l i fe t i m e o f <1 S\'Ste m , i ts target system becomes obsolete . • Prior i mp l e mentati ons . Prior s i m u l ator i m p l e m e n ta ­ tions c a n p n)\'id c usefu l i n formatio n , b u r i t must be used c a utious l y. U n less the prior i m p l c m c n t l klorc, an d most working svsterns a r c in t he The I mc rn c t pbys <1 hands of i n d i vid u a l col lectors . vital role in l oc a t i n g m a t e r i a l h e l d bv e n t h u si a s ts , t hro u g h nc\\·s conferences such as alt. t r h i sto ric wstc ms resides on pa p e r t a p e s , DECta p c storage systems, 2 00/5 56/800 b i ts - per­ i n ch m ag n e t i c Llpcs, d i sk ca r t ri d ges, 8 -i nc h tloppy d i sks , and so on. Few i f any modern systems h a ve t h e se p e r i p h e ra l s; :md tew if any h i stor i c systems have m o d ­ e rn n e twork i nt e rc on n e cts . T h u s , m e d i a transl ation usually enta i l s l i n ki n g a wo r k i n g version of the ta rge t system to a modern system b y m e a n s o f a s e r i a l l i n e . KE RNI I T o r some o th e r s i m p le pr o toco l al lows f(J r a by t e - by - by te network copy trom the or ig i n a l med ia to a fi l e on a mod e rn s yste m . Once the soti:warc has been located and moved t h e n e x t i ss u e is s o u rce s . vV i t h o u t sou rces , J n d other test progrJ ms arc useless; d e t ec t e d errors c a n n o t b e traced back to causes wi t h ­ o u t m a n u a l d e c od e of the bi n a ry progra m . T h e abse n ce of sources \\'as a pri n c i p a l reason for i n c l u d i n g to a ti l e , d i ag n ost ics sym bol i c d is:�sse m blv a n d i n p u t i n S I M . The fi n a l i ssu e in so ftw<1 rc is licensing. Even though the target systems a rc obsolete and often no longer m a n u b.ctu red , t h e opera ti n g S\'Stcm soti:wJIT m a y be prot e c t ed bv c o pyri ghts a n d l i ce nses. Most P O P- 8 soft w a re i s i n t h e pu b l i c d om:�in; however, t h e P D P - 1 1 a n d NO\·a ope ra t i n g systems J re sti l l l i c e n se d , as arc a l l v e r s i on s of N I X . C or p o ra te l i c en si n g p o l i ci e s rarelv accom mod:1te h o b h,·ists; t h i s l i m i ts o p c L H i n g s y s t e m d i stri b ut i o n to le g i ri m Jtc ( t hJt is, b u s i n ess ) u s ers . Table 7 l is ts the soti:warc tcJLmd tc1r eac h si m u l a ­ tor i n S I M . Debug T h e d e bu g p a t h tc) r a s i m u l ator d e p end s on the ;wa i l a b l c softwa re . I d e a l ly, the s i m u b ro r \\'O u l d with t h e s a m e soti:ware tests u sed ro debug t h e target h : m.l\\'arc, but this so ft w a r e is rare l y archive d . D i agnostics c a n p ro v i d e l ow- leve l b e d e b u gg e d ch ecki n g , b u t diagnostics typica l l y c h e c k t(J r broken p a rts i m p l e m e n t a t i o n , rather than a n i m p l e m e n t a t i o n . Even w h e n d i a gnostics d o c h e c k a rc h i t e c t u re rather t h a n i m p l e m e n tation ( as i n t h e b as i c i ns tru c t i o n d i ag n os t ic s on rhc PD P - 1 1 syste m ) , the a bsence o f sources l i m i ts t h e i r u r i l i t \'. C o nse q u e n t l y, the s i m u l a tors were d e b u g ge d m o s t l y with s i m p l e hand tests a n d t h e n w i t h r h c op c r H : i ng syst e ms . Opera t i n g systems a re both e x ac ti n g <1 1ld i m pr e c is e tests of i mple m e n ta ti o n c o r r e c tn es s . U nless l klorc, debugged some of the subtlest problems, c rc ­ worl d w i d e com m u n i n· o f historic m a c h i ne h o b b ,. · i s ts . a n d c mhusiJsts. ;1tcd t h e Ma kctilc, a n d provided the fi rst and most Ben Thomas th e character- by-character I/0 ro u ti nes fi-c q u c n t h· prm idcd used distribmion site . fix VMS . C h ri s Su ddick helped debug the PDP- I I floa t i n g - p o i n t code. Warren Toomey and the e n t h u s i ­ asts Jt P U I'S ( the P D P U N l X Prese rvation Society ) i n A ustr:l i i a a l l owed me a cc e ss to t h ei r archive o f e ar l y UNIX rek:ascs . the P D P- I Lcc n d c r t U N I X V6, and Van l s i m u l ator with Doorn References and Notes I. As Jll�\11�1ging d i rector of D i gi ta l 's A u srr�1 l i a n S I J hs J d i�1n· 2. M . B m nct, "An U pdate on debu gged initiJ! i m pcrus to the p ro j ec t and K e n H a rrc n s r c i n im p o r ra m contri burion to p rese n a t i o n bv i m p l e m e nti n g �1 D ECsystc m - 1 0 s i m u l ator. Last, 3. J\!! J x d o c u m c n t:� tion and 34 B u rn e t software generous�\' pro,·idcd ti-om the Digital Vol . 8 ?'\o 3 ! 99 6 t h e M u se u m Treasu res , " s u rc:s," !)f:Cl s A ustralia Symposi u m Proceedings. M . Bu rn e t , "The '94 U pd a te on the M u se u m Trc;1 Au gusr 1 994. · not least, but ro 1 99 3 . , mJdc an 1 9 8 2 , M a x Burner created a n d operated the J' f)l' trad e - i n progra m . f )fX.'/ (',' A ustralia Svmposium Pruceedinp,s. August Franc Grootj c n w i t h 2 . 1 1 BSD . Larry Stewar t provided t h e fi·om I 975 4. "The A uslmlio S) 'lnjJosiunl Proceedin,r.;s. :\ ugust 1 9 9 5 . M. B ur n e t , Last Great P D I'- 1 1 ," /)/:U .', Table 9 S i m u l ator Perfo r m a nce Ratio S i m u l ator S i m u lated I nstructions per Second Real Instructions per Second P D P-8 1 , 800,000 400,000 440,000 500,000 .88: 1 7 50,000 2.26:1 PDP- 1 1 1 . 700,000 Nova ucoder> 4. 5 : 1 pdp8 PDP-8 s i mu l a t o r V2 . 2 b s i m> a t t r k O o s 8 . d s k s i m> b o o t r k O . DA 08-APR-96 . D I R 0 8 - A p r -96 COPY I T . SV DIRECT . SV CCLX pI p . sv . sv . sv FOTP ABSLDR . SV BAS I C BATCH BCOMP . sv . sv . sv BITMAP . SV BLOAD BOOT BRTS . sv . sv . sv CHEKMO . SV COMPA F . CREF . EDIT . EDITS . EPIC F4 FRTS FUTI L HELP L I BRA . . . . . . SV sv sv sv sv sv sv sv sv sv 2 7 09-Ma r-93 1 1 -0 c t -92 24 1 1 2S-Feb-93 1 1 -0 c t -92 8 1 1 -0 c t -92 5 1 1 1 1 -0 c t -92 1 0 26 s t-92 t-92 t-92 t-92 1 0 SABR . sv SCROLL . SV 24 24 1 7 SET 20 . sv SRCCOM . SV . sv 5 1 1 -0 c t - 9 2 . LD . sv 3 2 0 6- J u l - 9 3 1 0- J u l -9 3 . FT 1 2 1 1 - J u l -9 3 04-Aug-93 2 09-Aug-93 1 2 1 1 0 1 0 09-Aug-93 9 1 4 1 1 -Aug-93 EMTST S INST1 . SV 1 0 1 4 1 1 -Aug-93 1 1 -Aug-93 1 1 -0 c t -92 F O RT 3 CLOSE FORT4 1 1 -0 c t-92 FORT4 1 1 -0 c t -92 1 1 -0 c t -92 FORT6 FORTS 1 1 1 1 FORTS FORT6 1 1 -0c t-92 1 1 1 1 -0 c -0 c -0c -0 c t -92 t -92 t-92 t-92 . LD . LD . FT . LD . FT METSC . METSC2 . EMAT . EMDCT . sv SV sv sv . sv 09-Aug-93 09-Aug-93 1 1 -Aug-93 1 1 -Aug-93 1 1 -0 c t -9 2 VERSN3 . SV 32 1 0 24 1 5 1 1 -0 c t -92 1 1 -0 c t -92 BUILD BAS I C . OV 33 1 6 1 1 -0 c t-92 1 1 -0 c t-92 1 1 -0 c t-92 5 1 1 -0 c t -92 1 1 -0 c t -92 1 1 -0c t-92 BUI LD6 . SV . sv 33 1 1 -0 c t -92 ADDER . sv 1 3 1 1 -Aug-93 BU I LT 1 2-0 c t -92 1 8- 0 c t -92 FORT? CLEAR . FT . LS 1 2 3 0-Aug-93 1 3-Ja n-94 1 8- 0 c t -92 1 8-0c t-92 C LEAR . sv . C F 2 1 3-Ja n-94 1 3-Jan-94 . PA 2 1 . BN 2 28 1 3-J a n-94 2 1 -Ma r-95 s 1 3 1 0 6 14 20 26 26 5 1 1 LOADER . MATST . MDTST . OCOMP . OPT F4 . SV 1 2 SV sv sv sv . sv 9 1 4 8 1 3 1 9 F i les I n S i mu l a t i on s i m> -0c -0c -0c -0 c RESORC . SV RUNO F F . SV 20 5 8 1 9 TECO 5 1 6 95 1 1 1 1 . sv PASS20 . SV PASS3 . sv . SV RALF 1 1 -0 c t-92 1 0 LIBSET . SV LOAD . sv PAL8 1 1 1 1 PASS2 . sv 1 1 -0 c t-92 HELP . H E 33 1 1 1 -0c t-92 1 1 -0ct-92 1 1 -0 c t - 9 2 HELP HELP . HL 72 FORT? . LD 4 2 07-Sep-93 1 1 -0 c t-92 1 1 -0 c t-92 J MPTST . SV JMPJ MS . SV 3 3 1 8-0c t -92 1 8- 0 c t -9 2 1 1 1 1 RK8ENS . BN I NST1 . SV I NST2 . sv FORT . FT 1 1 4 1 1 1 1 1 1 -0 -0 -0 -0 c c c c t t t t -9 2 -9 2 -92 -92 FORT 1 1 -0 c t -92 1 1 -Aug-93 1 1 1 1 1 1 1 1 FORT2 FORT2 -Aug-93 -0 c t -92 -0c t-92 -0 c t-92 980 s topped, DOS SHELL FORT3 Blocks PC : . oc - 221 2 01 207 . LD . LD . FT . sv . sv . FT Free 1 2 2 1 2 2 1 C LEAR CLEAR CLEAR D EMO DOS 1 1 -Aug-93 1 3-J a n -94 01 -Dec-92 0 1 - D e c -92 1 7- J u n-93 DOS DOS . PA . BN 4 1 . LS 1 0 2 5- J an-94 2 5 - J a n-94 25-Ja n-94 SHELL . PA 1 25-Ja n-94 09-J u l -93 SHELL SHELL . BN . LS 1 2 2S-Ja n-94 BAS I C FOO FOO . WS . PA . BN 1 1 1 1 0-Ma r-94 3 1 -Ma r-94 30-0c t-92 09-J 22-J 2S-J 25-J 26-J u l -9 3 un-93 an-94 a n-94 un-93 2S-Ja n-94 3 1 -Ma r-94 Blocks C KS F ) Figure 2 P D P -8 S i m u l ator Ru n n i n g; OS/8 D i girc1l Tech n i c a l j o u rn c1 l Vol. 8 N o . 3 1 996 35 ucoder> nova NOVA s i m> s i mu l a t o r V 2 . 2 b a t t dpO rdos .dsk s i m> set s i m> boot tti dasher dpO Fi Lename? NOVA Date RDOS Rev (m/d/y) T i me C h : m : s ) � ? 7 . 50 4 8 1 6 R L i stie sys- . ­ SYS5 . LB 96 26 0 SYS . SV 1 72 1 6 56320 D SD 05/24 /77 1 2 / 1 4/95 SYS . LB SYS . OL 20240 D 30720 c SYSGEN . SV 23040 SD 1 3: 18 1 6 : 21 05/31 /85 1 2 / 1 4/95 [001 01 7] [005057] 0 0 04/30/85 1 4 : 49 05/31 /85 1 2 / 1 4 /95 1 6 : 21 1 2/ 1 4/95 [000746] [005272] 0 0 05 /02/85 22 : 20 05/31 /85 [ 00 1 401 ] 0 R di s k LEFT : 21 58 USED : CONTIGUOUS : MAX . 2 706 2054 R S i mu l a t i on s i m> s t opped, PC : 41 740 C LDA 1 ,4,3) Figure 3 No1·;1 S i n w h tor Ru n n i n g IU)OS :1 . A. A h i , C. R u r r o u g h s , A . Core, S. LuVL1 r, C : . - Y . Li n , rl l d }01 / rJ /{{/. 1'01. 4 3 , 110. 4 ( 1 Sl 9 2 ) . 6 . \V A n d erso n , " Logical Vc r i tic�tion o h h c NVAX Cl'U C h i p De s i g n ," Dip,ita/ 'l(,chu ico/ 110. 7. jo u m o l. mi . 4 , 3 ( 1 Sl9 2 ) : 38-46. I C C � b g n i and W S hetwoocl , " V A X (,000 M o d e l 400 C : I' LJ Set Chip F u ncriotd D c si gt t f ),�� i/{{/ 'i ('cl11 1 iwl Jou rnol. 1 · o l Vcri ti cni on ," 2 , n o . 2 ( 1 99 0 ) : M -7 2 . S . A. H u tc h i n g� , "The E1 o l ur i o n o r· r h c C : w, ro m C A D S u i re Used on r h e ;'d i c roVAX 1 1 S1 s t c m , " /Ji:� itul 7 ('Cb l l iWl/OIII'I/{{/. 1 0 1 . I , n o . 2 ( 1 9 8 6 ) : 48-::i ::i . 9. M . 1\.;l nrroll'irz .l: Priu c iplcs m1d h\WIIjJies. "The IB M Svstc m/360, Svstcm/370 , 3 0 3 0 , Ill/- c u be/col l e e r . h tlll . 1 6 . For i n foml;l t i o n on ;lnd p i c t ur e s of nu t l l ' etml[' U rc rs, sec 1'. h i s tmic l'i c rc e ' s II'Cb page Jt h rtj) :// 11 11 11 . re l c p o rt . c o t n /- p t · p /c o l l c c t/i n d n . h rml . 1 7 . !-'or d oc u m c m�rion a n d r c l c 1 a n t l i n ks , sec D . j o n e s \ 1 1 c b pre i n l( >rmation on ) . W i l s o n ' s s i m u l ator ( n c · c u ra b l e o n k ), sec his 11·cb p ag e at hp // trp . u pd rmation sec t he l ' l ' I'S (>11 the 1' 1) 1'- 1 1 home page L' :-\ l X m i n nic .cs.;ld I J .o;: . � u / l' L1 1'S/i n d n . h t tn l . at arch in· , h t t p : /I ucoder> pdp1 1 PDP-1 1 s i mu l ator V2 . 2b s i m> a t t r k O r t r k . d s k s i m> boot r kO RT-1 1 SJ . da (s) V05 . 04 8-apr-96 . d i r 08-Apr-96 NL . SY S RT1 1 SJ . SY S PTESTX . MAC B I N C OM . S AV . SAV DIR 2 80 1 8-Sep-89 1 8-Sep-89 23 27-J an-94 27-Sep-88 24 RT1 1 FB . SYS S POOL GVI DUP . REL . S AV . SAV 5 1 8-Ap r-90 2 7- S e p-88 27-S ep-88 . SAV RESORC . SAV ODT . SAV 61 25 27-S ep-88 SSP 61 27-S ep-88 IND 27-S ep-88 MAC RO F O R MAT . S AV 49 24 2 7-S ep-88 27-S ep-88 PBCOPY . SAV 2 1 6- F eb-89 . OBJ . SAV 8 21 05-0 c t -89 S Y S L I B . OBJ SYSMAC . SML 2 7- S ep-88 DATE l OP TT . SAV 1 1 DM DX . sys . sys . sys 2 5 24-Apr-89 1 8-Sep-89 1 8-Sep-89 SWAP . SYS . SY S 4 5 1 8-Sep-89 05-0 c t -89 2 1 8-Sep-89 RK MT SP 30 8 2 27-Sep-88 26-De c -90 0 1 - J a n-80 . SAV LlNK . SAV ODT S I PP L S LP . SAV PIP LD LC . sys 8 DL DP 27 4 3 2 7-Sep-88 1 8-Sep-89 . SYS . SYS 3 9 . SYS 6 1 8-S ep-89 AV AV 7 61 27-Sep-88 AV OM 1 3 1 HANDLE . S MAC . S UCL . S STARTS . C 07-0 c t -90 27-F eb-87 MTROL . SAV 1 7 MLIB XPC . SYS . SAV 300 1 6 20-De c-90 25-J un-91 HELP DESS . SAV . SAV 1 32 1 8 PTESTX . OBJ 8 . sho 1 432 1 8-Sep-89 1 8-Sep-89 1 8-Sep-89 1 6- F e b-89 2 2 - D e c-89 1 9-J a n -94 2 7 - F eb-87 20-De c -90 09-Ma r -88 B l ocks b l o c ks dev Devi ce Status -----NL ------ CSR Vecto r ( s ) --------- DOD DOD Insta l led 000000 TT I n st a l l ed 000000 DL DM D P I n s t a l l ed 1 74400 Not i nsta l led 1 77440 Not i nsta l l ed 1 7671 0 254 I n s t a l l ed Resi dent -Not i ns t a l l ed 1 77 1 70 264 220 DX RK LS 1 6-Ma r -89 . sys . sys . sys 4 28 F i l es, F ree 05-0 c t -89 05-0 c t -89 02- F e b-89 . SYS . CC L . SAV 3330 27-S ep-88 3 . SAV UCL MTPIP 49 1 8- S e p- 8 9 1 4 - A p r -8 7 49 58 . SAV 1 9 24 LlBR 94 1 1 1 77400 1 76500 MT LP I n s t a l l ed I n s t a l l ed 1 72520 1 7751 4 SP LD I n s t a l l ed I n s t a l l ed LC I n s t a l l ed 000000 000000 1 7751 4 S i mu l a t i on s i m> s t opped, PC : 1 46506 1 60 2 1 0 470 224 474 300 304 200 1 1 0 000 200 (ASR R5) Fig u re 4 P I W- 1 1 Sinw bror Ru n n i n g IZ"T- 1 1 Digital k c i l n ic.il ) o l l rn ,l l Vol . t; No . .i l 'J'J6 .i 7 B iograph ies Maxwell M . Burnet 1vl a x B u rn e t has been with D igital i n Austra l i a �or 29 yc;Jrs. D u r i n g that time, he h:1s sol d , serviced, or marketed ;J I I t h e m a c h i n es i n the co l l ection . H e m a n aged t h e D i g i t;l l i\ ustra l i a s u bsid iJrv �ix se1·e n l'c:lrs. He 11 as a s a l esin:lJJ ;m JBM 1 6 2 0 a t Tufts Unin:rsitY 11·ith c l PDP- I 0 . H r is c u r rc mll· in Boston durin g 1 9 7 1 :1 n d m a naged to rcpbce the o l dest sun·i1·ing " n.:chie" i n the S vd n e y oftin: and m;lkcs num• corpor:1re prescncnions in Austr:Jii;L H e m a nJgcs the A u s tra l ia n D EC US Society, t h e S u bs i d i a r y 's l cK a l content a n d e xporr o b l igations with the Austr:J\i;ln Go1·emmenr, and the local Prod uct Assurance G r o u p . He h : 1 s co.l l ected a m u seum o f e a r l v Digi tal machi nes a n d is kno11 11 a m u n d S v d n c v as " M u s e u m 1\ll a x . " He recci1·cd :1 R .Sc. ( h onours ) from M e l bo u r n e U n i 1·ersin·. Robert M. Supnik Bob S u pn i k has bem with D i g i ta l i n the U ni ted States �(>r 1 9 years. H e j o i ned the 1\lbss Sro1·age Group and then m oved i nt o Semicond uctor Engineering, where h e succes­ sively Imnaged the !Jst PDI'- l l i m plementation ( th e ) - l l ) , Adv;lnccd De1-clopment, the tirst singl e - c h i p VAX i mple­ m c l lt ­ processor G rou p . H e ;1 \ so IITote or contri bmed to the m i crocode of e1·er1' s i n gl e - c h i p VAX m icroprocessor. I n 1 9 8 8 , h e started t h e Alpha progra m , w h i c h h c ma n;l!J,Cd t h ro u g h b u n c h of the �i rst products in 1 992 . He t h L· n b e c a m e tech n ic 1 l d i rector, tlrst of Engineeri n g � n d t h e n of the C o m p u te r Systems D i v i s i o n . I n 1 99 6 , h e bec1mc vice presi d e n t of Rcsc1rch and Advanced D e v e l opme n t . He h:1s B . A . degrees i n math e matics a n d i n h i srot·y ti-om M IT, ;llld � n M .A . in historv ti·om B randeis U n i 1·ersi ty. 3R Dig:ital Tcch nic.ll j o u m;ll Vol. 8 No. 3 1 996 I Willi am N. Celmaster Modern For tran Revived as the language of Scientific Parallel Computing New features of Fortran are changing the way A Brief History of Fortran i n wh ich scientists are writing and maintain i n g large analytic codes. Further, a number o f these new features make it easier for com p i lers to generate hig h ly optim ized architecture-specific codes. Among the most exciting kinds of architecture-specific optimizations are those having to do with parallelism. This paper describes Fortran 90 and the standard ized language extensions for both shared-memory and distributed-memory parallelism. I n par­ ticular, three case studies are examined, show­ ing how the distributed-memory extension s (Hig h Performance Fortran) are used both for data parallel algorithms and for single-prog ram­ multiple-data algorith m s . The Fortran ( fO Rm u l a TRANslating) compute r b n ­ begun by John 1 9 54. The g o a l o f t h i s project \\'JS to guage \\'aS the resu l t o f J project Backus at 113 M i n provide J "'ay ti.>r programmers to express mJthemJti ­ cal torm u l as through a t(xnd ism that computers could tra nsl a te i n to m a c h i n e i n structions. I n i tially there w:ts a great d e a l of skepticism Jbout the efficKy of s u c h a schem e . " H ow," the scientists as ked , "wo u l d :tnyone be a b l e to tolerate the i n dti cicncies that wo u l d res u l t from compiled cod e > " B u t , as i t t u rned o u t , the ti rst compil e rs we re s u rprisi n gly good , and progra m m e rs were a b l e , fo r t h e f-i rst rime, to express mJthematics i n a higb- le\'el compu ter langu age. Fortran has e\·ol ved cominuall\' over t h e years in response to the needs of users , parricularh· i n rbe are:ts o f mathematical o p ressi\'i ty, program mainta i n :t bi l i t\', h ard\\'are comrol ( s u c h as I/0 ) , a n d , of c o u rse, code opti m i zations. I n the m e a n ti m e , other l a nguages s u c h as C a n d C + + h a v e been d esigned to better m e e t the n o n ma thematical aspects o f soft\va re d esign , such JS graph ical in terraces a n d complex logical l ayou ts . These l a n guages have caught o n a nd have gra d u a l l y begun to erode tbe scien tific/e n gi n eering Fortran code base. By the 1 980s, pronounceme nts of the "death of Fortra n " prompted la ngu age d esigners to propose exte nsions ro Fortr:tn t h a t \\ O u l d i n corporate the best fea tu res of other h i g h - b·d Llnguages a n d , in :tdd ition, p ro\·ide ne\\' b·c l s o h n a t h e matical exprcssi\·it\' pop u ­ l a r o n supncom p u ters s u c h a s t h e C YB E R 2 0 5 and the CR.AY S\'Stems. This l angu age became sta n d a rd i zed as Fortran 90 ( I SO/I EC 1 5 39: 1 99 1 ; ANSI X 3 . 1 98 l 992 ) . A t t h e prese n t t i m e , Fortran 9 5 , w h i c h i n c l u d e s m a n y of t h e para l lclization features o f H i g h Performance Fortran d iscussed later i n this paper, is i n t h e tl nal stages o f standardi zation . It i s n o r yet clear w hether the mod e rn i zation of Fortran c a n , of i tsc l t� stem the C tid e . However, I \vi i i demonstrate in this paper that modem FortrJn is a via ble ma instre a m l a n ­ g u age tl lowing code ti·a g m c n t i l l u strates the s i m p l i c ­ i ty of d y n a m i c m e mory ;t l locttion w i t h Fortran 9 0 . Ir a l s o i n c l udes some o ft h c new syntax tor d e cla r i n g l'ari­ a b lcs, some examples of ;trrav m a n i p u l :ttions , :tnd :t n c x �t m pk o f h oll' to usc t h e nc11· i n t r i n s i c m :trrix m u l ri ­ p l i et t i o n fu nctio n . J n a d d i t i o n , t h e exc lamation m:t rk, 1\·hicb is used to begi n commcm Str manipul:t.ting �m·a ys. ' I n acidirion, FortLtn 9 0 i ncorporates m o d e rn c on t ro l constructs and u p - ro -datc fcJturcs fi:Jr d ata a bstr.tction �tn d d�tt:t hidi ng. Some o f t h e s e constr u c ts , ti: >r o:�t m ple, n o W H I LE , �tl though not part of F O RT R A N 7 7 , arc a l rcach· p a r t of t h e d e F:Kro forrr�m sra mbrd as pro­ ,·idcd , ri: >r e x a m p l e , 11 ith D E C fortLlll . Among the kcv nell' features o f Fortran 9 0 �tre the ti. > l loll i ng: • • • Perm issi hi l i tv of frec-ti:)rm source cod e , so pm­ grammers em usc l o n g ( i . e . , mc.m ingfu l ) v:t r i J h l e n�t mcs �t nd a r c n o t restricted to begin statements in col u m n 7 Mod e m control str u c tu res l i k e CASE progra m m i n g c::u t constructs Extended control of n umeric rccru rc take �tnd D O ad1·�tt1tagc of i n d c pc n dence processi ng extensions, ti:.>r more c.tsi l y express­ ing a rray opera tions and al so fo r e x p re ss i n g i n d e ­ pendence ot' c l c m c n t operations Poi mer�, t( J r more flexible control oflLtta p LK e m e n r • D�t t�t structures, ti:Jr d a ta absrr:tcrion • Usn-ddi ncd :tbsrr�tction tvpes and oper�ttors, Proced u res a n d m o d u l e s , to rc> r d at�t h elp progr.t m mns ll'rire reusable code • Stream • Nc"' character-oriented i n p u t/ouqJ u t te�ttures i mrin�ic ti.t ncrions 'vVi th these new k arurcs, a mod e rn bortLl l l p ro ­ gr�t m me r c a n nor o n l y su ccessfu l l y c o m p il e �tn d c x c ­ c u re prel' i o u s st:tndard s - c o m p l i a nt Forrr�t n codes b u t also design better codes with • Dram�ttical lv simpli fied 11'<1\'S of doing d 1 ' n�1 m i c mem orl' m:t n age m e n t • • 40 j ) , · n r sintpliticd progra m m i ng b u t :t lso l u r d 11 �t rc-spccitic o p t i m i z a tio n s . i n Fortr�tll 9 0 , o n e c a n ll'rite t h e �t rr:tl' bette r exa m ple , asstgn men t = B + C ll'hich in AtTJI' • • R EA L*8 A precision, for arc h i ­ D I ME N S I O N ( : , : , : ) , ALLOCATABLE of a l l of f O RT RA N 7 7 , so users can co m p i l e their F O RTRA. N 77 codes without nH > d i tiu rion srrucrmcd • REAL, & I ncl usion W H l L E , so progr.mt mers • a n C :\ tc n s i o n to l- O in 1\.'\;-.,J 7 7 . rO R.TRJ\ N 77 II O U i d D0 1 00 DO 200 1 00 END END J = 1 ,N 200 I = be ll'rittcn as 1 ,M DO DO The Fortran 9 0 �t rLtl' :tssig;tJll1Cllt 110t o n lv i s more elegant b u t �tlso permits the co m p i l e r to easih· recog­ n ize that the i ndi,·idu�t l dement �1ssignmcnrs arc I I H . k ­ p cn d e n t of one �mother. I f rhc compi ler 11·ere targctin� a \'ector or p �t r�t l l e l computer, i t c o u l d generate code that e x p l oi ts the .t rcltitcctmc [)I' ta king ad 1·anragc of this inckpcndencc bcrii'CC!t i ter;ttions. Of course , the p:trti c u l a r DO l oop s h oll'n :t bo1·e is s i m p l e e n o u g h tlt�tt m;tnv compi lers would recogn ize rhc i n d ependence of i terations and could the rdi: > rc pert(mn the �� rch i recru rc-spcc i flc opti m i zations with­ o u t the aid or· Fortran 90's n ell' atT r m a nce Fortr r the progr:t m m e r to optimize d a tJ loca l i t\·. I n l c i l s m , :md n 1 :1 n \· d a t:� 11 laecment d i rccti,·es arc pro­ add ition , H l'F i nc l u des \\':1\'S to spec i �· srv lc o f s i n g l e - progr:1 m -m u l t i p l e - d a ta a more ge n er a l (SPM D ) exec u ­ tion i n 11· h i c h scp:1 rarc processors can i n dependen tly work on d i Hi.: rc m [1<\ rtS of the cod e . This SPMD spcci­ ti u rion is t( m n a l i zed in such a way a s to make the re s u l ring code br Cm H P F C.F t h e Sr systems of a code c a l led , t()r C\ <1 l l1 p l c , dd . f, u s i n g rhe command l i n e kf90 -f kapa rgs= ' - co nc ' cfd . f -o cfd. exe l l lo\\' i n g s h O\\'S hO\\' a a n d the H P F spc c i tication h a s l o ts of ,,·a rn i n gs about seg m e n t bcl o\\' i s so s i m ple t h a t the compi l e r cJn J u to­ constr u c t . S o r par:t l lcl execution of \'J rious thre ;t d s of con trol . The C<1 11 on ic:t l e \ a m plc is a be i n c l uded i n the cod e . ( A ctu:l l h-, the code par;1 1 l c l izc a l oo p . The tr:l d i tion
  • 41 Cluster Fortran Parallelism 1 HPF$ D I STR IBUTE ! HPF$ ALIGN R EAL*8 X ( B LO C K l W I TH Y X X ( 1 000 l , Y ( 1 00 0 ) H igh Performance Fortran V l . l is cu rrc n tl \ ' the o n lv language st�1nd ard for d isrri butcd - mc mon· para l l d com p u ti n g . T h e most s igniticm r \\'JY i n ll' h i c h H P F extends Fortran 90 is t h ro ugh a rich b m i l\' of d a ta place m e n t d i re ctives. There arc ;llso l i b ra r y rou ti n es and some extensions for con trol p:lr:JIIclism . H P F is the s i m p lest way ofpara l lelizing d ata-para l l el ;lppli ­ cuions on c l usters ( also known as " b nns " ) of work­ s t : Hi o n s and se r ve rs. Other m ethods of c l uster pa ra l l el ism , such as message p a s s ing, req u i re more bookkeeping and a rc there fore less ClS\' to e xpress and less cls\· to m a i n t:�i n . In :.�ddition, dming the past \'Car, H Pl-' h;� s become ll'idely ava i l a ble :t nd is s u p ported on the pLnfcmns of all m aj o r v e ndors . H P F i s often considered t o b e ;1 dolo JHtm!!c! l :m ­ gu;lge . That i s , i t fac i l i tates p a r a l l e l iz :Hion of a rray­ bas:d a l gori th ms i n which the instruction stre a m can be d escribed as a seq u e n ce o f array m a n i p u l a ti o n s , c:�ch ohv h i c h is i n h e rentl y pa ra l l e l . What is l ess we l l known is t h a t H P F a l s o p rovi d es a powe r fu l wJ y of expressi ng the more ge neral S PM D p a ral l e l i s m m e n ­ tioned c:1rlier. This ki n d ofpat·allc l i s m , often expressed with mc s s:� gc - p a ssing li braries s u c h Js t\tl l' l ,' is one i n whi c h i nd i v i d u a l processors c a n oper;ue s i m u l ra n c ­ o u s l v on i nd e p e n d e n t i nstru ction strea m s :1nd gcner­ :tl l v � x c h a n ge data either b\' e x p l i c i t!\' s h a ri n g m c morv Y ( Z : 99 9 l X ( 1 : 998) + X ( 3 : 1 00 0 ) - 2 * X ( 2 : 99 9 ) a n s wer> END The H Pr compiler is responsible f()r generating a l l of the bound:trv-elemem comtmmication code . The com­ : piler is also , csponsiblc f(>r d e termini n g tl1C most C\'en d isrribmion of ;HT�lvs. ( I t� f(lr e x a m ple, there were 1 3 processors, some chu n ks \\'Ou l d be bigger than others . ) This s i m p l e example is useful nor o n l y as ;l n i l lustra­ tion of the po\\'er of H PF hut a lso as a way of pointing to one of the h ur­ proccssor system , e:t c h processor wou ld operate on 250 cle ments with 750 floati ng-point opera t i o n s. ln a d d i t i o n , each processor wou l d be req u i red to com­ m u n icate one word of dJta f(x each o f the two c h u n k bouncbric s . The ti m e th;lt eac h of these c o m m u n iu­ rions takes is k n m\·n as the com m u nications l a tcn cv. T\v i c a l tra n s m ission control protocol/i n ternet proto­ col ( TCP/IP ) n c t\\·ork l atencies a rc t\\·enn· thousJnd ti mes (or more ) l o n ge r th:t n the time i t t'l'picalh' t:1 kes o r lw cxclu n g i n g messages. Three c a se studies f( > l l m\ · a h igh- pedclJ'Iumce S\'Ste m to perform a floating­ of progra m m i n g . rions arc negl i g i b l e com pared with t h e t i m e t;1ken to A One-dimensional Finite-difference A lgorithm C o n side r a s i m p l e o n e - d imensional grid pro b l e m ­ klism wou l d be a n c r loss, si nce t h e total execution the m o s t m i n d - bogglingJ y s i m p l e i l l ustr:J tion o f J-! P l-' latencv. ll'h i � h i l l ustrJte the d ata pa rJ I Ic l and the S l' tVI D srvks i n actio n - i n w h i c h e a c h g rid val ue is u pd ;Hcd as poi n t opera tio n . T h u s C\'en 7 5 0 floating- point oper;l ­ comm u n i cate . I n the a bove e x a m p l e , nct\vork par T h e s i m p lest \\'J\' t o i m �1ginc p FORALL ( ! =2 : N X - 1 , J = 2 : N Y - 1 : Z , K = Z : N Z - 1 : 2 ) U ( ! , J , K ) = FACTOR* ( HSQ* F ( I , J , K ) U ( I -1 , J ,K ) + processors. T h e beaut\' of H PF is t h :l t a l l of t h i s c a n be done i n the con te x t of m a t h e m atics rather than i n the U ( l+1 , J , K ) + + code ti·a g m e n t i n f i g u re I i l l u str;l tes how t h i s is done. & & On nvo processors, the two col u m ns of the � assi g nme n t causes one o f those co l u m n s to be move i The d istri b u tion d i re ct ive lays o u t the a r r ;1 y so that to the othe r processor. This kind oL1n o pe ra tio n begins the ti rst d i m e nsion is completely contained w i t h i n to p rovide progr;1 m m ers with e xp l ic i t wavs to con trol a processor, w i t h t h e o t h e r two d i m e nsions block­ d a ta co m m u nicJtion ;md thc rdi:>re to more e x p l i ci tl v d i stribu ted across processors i n recra n g u l : u· ch u n ks . m a n age the associJtion o f d ata and operations to T h e red-b l a c k c h e c k e rboa rd i n g i s pedi:mn e d a lo n g the second and t h i rd d i me n s i o n s . FortLln processors. Notice that the progra m m e r need not be N ote :�lso t h e expl i c i t about t h e pJra l l c l is m . I n fact, scien tists ,m d 90 ri-cc - to rm synt:�x e m p l oved h e r e , i n w h i c h engineers rJrclv ''';m t to e x press pJra l l c lis m . In tvpie1l the a m pe rsand i s u s e d as a n e nd - of- l i n e conti n uation messagc- pJssi n g progr;1 ms, the messages often express state m e n t . com m u n icJtion oh·ector and atTa\' i n t()rn1 ation. I n t h i s e x ;lmpk, the para l l e l i s m is s i m i l a r t o t h a t of the o n e - d i rn e n siotl J I ti n i t e - d i fkrence However, despite the rc n·e n r hopes of progra m mers, e x :t m p l e . t h e re a rc ti mes when a p::1ra l l d a l gorith m c1n be However, communication now occurs along t h e two­ d i mc n sion J I bou nd a ries between blocks. The H PF expressed most s i m pl y :1s :1 col l ection o f i n divi d u ;l l i n s tr u cti o n streams opcoting o n local datJ. T h i s SPM I ) commu nications. comp i l e r is respons i b l e tor t h ese U U( : ,Z) 1 MOVE A VECTOR BETWEEN PROCESSORS Fig u re 1 Code Ex,l lll p k Showin g Con tro l of Data Co1nnJ u u i carion \\·i rhout Ex p ression of Para l l e lism Digital Techn ic-;tl )ourn.tl Vol . 8 No. 3 I <)')(> 43 CALL C F D(V) comp l ex C F D ( VLO C A L ) : : VLOCAL : : VLOCAL work w i t h v l ocal> END Figure 2 Cmk E x ;l m p l c of l\Jra l k l A l g o rith m Ex f>rc·w,;d a s Col lection o f T nstruction S trc;uns as "Amlb h l 's LJ\\·" ) . I n the cJsc D t u·i n g these l�1st tC\\' vears of the second m i l l e n n i u m , rems, the \\T:Jk l i n k \\'o u l d be the i nrcr-SM P com m u ­ sist of c l usters of sh arcd - m cmorv S M I' c o m p uter s . nication ;lnd T h i s e x c i t i n g dc\·e l opmcnr is the logiul res u l t o r· t h e o f l o u l comn1 u n ie;1rions op r i l ! l i zarion s . E x pe r i m e n t a­ opon c n ti;l l i ncrease in p c r rcm n:mce o f m i d - priced tion \\·i l l be ne CCSSr e i ght p rocessors ( m ore t h i n g is certa i n : The fu tmc w i l l nor be bori n g . Summary fortran \\':JS dC\-c lopcd :md h ;l S c o n t i n ued to c\·okc :-� s a co m p u ter l ;1 n g u :1ge t h a t i s p J rt i c u l arh· s u ited to ge nera l ! \ ·, r(>r e ight peers ) . I f t h e progrr i n stance, block - d i s tri b u ti o n d i rccti\'CS, t h e :1th.:cted extensions t o fortr:l l l a rc :l lT,l\ 'S \\'Ou ld be sp l i t up i n to eight chu n ks ofc omigu­ ous aJTJ\' sections. The second \\ '�lY o f wr i ti n g p:1r:1 l l c l fortran pro­ grams ti.)l' c l u stered S M P svste ms is to usc H P F to m ac h i n cs. for cxa mple , 011c m i ght writc are able s h a red - m e m or y o r c n ­ s i o n s ) t o ac hicve para lle l i s m loc:Jlly on c:1ch o f thc SM P ! H Pf $ DISTR IBUTE (* ,BLOCK ) : : SUBROUT INE l f the target S\'Stem consisted of t\\'0 S J\.l l' s�·s re m s , compiled t-i.)l' t\\'O peers, then the V :-tiT�l\' \H H d d be d is ­ C:lCh \\'ith rcmr processors, and t h e :l b O\'C J1n>gr:lm \\':JS per SJ'vl P svste m . Then the ro u r i n c , C � D , wo u ld bc tri b u ted i mo rwo c h u n ks o f co l u m n s - 0 1 1 c c h u n k exec u ted once per SMP syste m ; and r h c l'Cr d i rectives wou l d , on e:-�ch system, c a u se pr ­ nunce c h a i n ( a n obsen·at i o n t h r c l us ter ll011 Li n i t-i. m n - !11e l11or\' - ;Kccss R E G I ON e x p eri e n ce has s h 0 \\ '11 ( PC � ) for e x p l i c i t Fortr C*KAP*PARALLEL e x te n s i o n s t e r prog1·a m n 1 i n g . TmLl \', the most e x c i t i n g fro n ti e r E X T R I N S I C ( H P f_LOC A L ) ,1rc s h a red - me m or\' p:lr:l l l c l i t.;l t i o n : 1 n d also data -p:1r:1 ll c l m u c h of the red i u m th:J t o t h er w ise m J c h i nes a nd t h e n to usc 44 of clusrercd SM t> s\ s­ Clusters of SMP Systems ;\1 . , '' C :o m 1>i l i u g H i gh tor Disrrib u t c d - mcmorl· S\·srem,," cr Pcrform;Hl cc �onran Digito/ Tecb uicol ,·ol . 7, no. ,) ( 1 9 9S ) :=i-2 3 . 4. R . Ktt h n , B . t(Jr D E C �ortL\11 ;\ 1\d Lcls u t·c , ;J nd S . S h �1 h , "The KA P 1\J r�l l l c l i z n D I-: C : C Pmg 1·,J ms," lJI:l!, ilol Tech n icol (o u mol. m i . 6 , no. 3 ( 1 9 94 ) : :=i 7-70 . ;:1 . I-'m c .\;l l n p l c s e c !'mceediu,r.;s rJ Supenomp u t inp, 'l)j ( I f. L E , \l o1 - c m bcr J 993 ) : 8 7 8 -8 8 3 , :md W. G ropp, E . l .usk, � 1 1 d A . Skjcllum, C'>ing I WI ( Bosron: M IT . Prc.\s, 1 9':>4 ) . 6 . 1� . l k n so n ct ;1 1 . , " Dc s i g: n o f D i gi t J I 's PcHJ I I c l Soft11·arc E 1 1 1 i ro n m c n r , " nt:r.; itrtl 'f i-chu ical.fo / 1 1 '1 101. \ OJ 7, no. 3 ( 1 9% ) : 7. 1-' o r 24-38. c x c1 1 n p l c s e c A . (;c i s r c r a l . , P I � I f: !'o m/lei I ir/t {{{/ . \ fochiue. ;I I \e1\· Gu ide oud Tu tol'ial j(;r Se!tt w l.: fJrtmlll'l Contj){{/ inJ,; ( Bostr p c1 r c1 J J c J SOrtW;l i'C c\ci11011StratiOilS and pcr­ t(mnancc c h cll·c\Ctni �:.uion o f D igital's h igh-pc r·t(xmancc s1·sn: r m . He has p u b l i s h e d n u n�er·ous papc1·s on paral l e l c o l l 1 J) U t i n g m c t h ocb, .1s 11 c l l c1s on to pi c s i n the ti c l d of j)ll\'sics. B i l l rcce i 1 nl .1 bach elor of science d e gree i n m a t h e ­ matics a n d j) l l \ ' s i c s f m m t h e U n i 1-c rs i r v o f B r i t i s h Co l u m b i c1 cllld <1 P h . D . in p h Ysics ti·om J-1 an·c1rd U n i , ·ersm·. Dig.it.ll Tcc l m i o l )ourn;ll Vol. 8 �o. ·' 1 9W> I Judith A. Piantedosi Archana S. Sathaye D. John Shakshober Performance M easurement of TruCi uster Systems under the TPC-C Benchmark Digital E q u i pment Corporation and Oracle C u r re n t i n d u strv tre nds ILl\'C mm'Cd from centra l i ze d Corporation have annou nced a new TPC-C computing ofkred lw u n ip roc esso rs a n d svmmetric m u lti processing ( S M P ) S\' Stem s to m u l tin o d e , h i g h h· performance record in the competitive mar- a\·a i l a b le a n d sel l a b l e s\·stems, c a l led cl u s t er s . T h e ket for database applications and UNIX servers on the A l phaServer 8400 5/350 fou r-node TruCiuster system . A performance evaluation Tr u C i u ster m u l ti c o m p u t e r svstem tor the D i gital U N 1 X e n \ ' i ron m e n t i s t h e l a t e s t clu ster p r o d u c t from D igital Eq u i p m e n t Co rpo r a tion . 1 In t h is pa per, \\'e d isc u ss our test a n d resu l ts on a tou r- n o d e Al phaServcr 8400 5/3 5 0 TruCiuster configuration s u pp o rt i n g the strategy enabled Digital to achieve record­ setting performance for this TruCiuster con­ figuration supporting the Oracle Parallel Server database appl ication under the TPC-C workload. The system performance in this environment is a result of tuning the system u n d e r test and tak­ i n g advantage of TruCiuster features such as the M E MORY CHAN N EL intercon nect and Digital's d i stributed lock manager and d istributed raw d isk service. Oracle Para l le l Server data base appl ication. We e va l u ­ ate t h i s system u n d e r t h e Tra nsaction P r oc e ss i n g Pert(m11 a n ce Counc i l 's T PC- C be n c h m a rk to prm·id e p c r t(J rma n ce res u l ts in t h e c o m pe t i t i\ e market f(>r d a ta ba se Jpp lications. The TPC:-C b e n c h m �1 rk i s a med i u m - c o m p l c x i t\', on - l i n e trans:tction Jl ro cess i n g ( O LT P ) ,,·orklo:� d . ' ' I t i s based o n :111 o rd er-enrrv workload, \\'ith d i ff-ere n t transaction t y pes ra n gi n g hum si m ple transactions to med i u m -complcx itv t ra nsac t ions t h a t have 2 to 50 t i m es t h e n u m be r of ca l ls of a s i m pl e transaction! To r u n t he TPC-C b e nc h m ark on a c l ustered system, the o pe rati n g system and the d ata base e ngine m ust prese n t a single chtabJse to the b e n chma rk c l i e n t . T h u s t h e sys te m r u n n i n g the Oracle PJra l lc l Server T r u C l u s tc : r d i fte rs grea th· �i·om a network-based c l uster S\'Stem l)\' two signiliGlnt katures . f i rs t , rhe Di gi t:t l U N I X d istri b­ the d istrib u ted uted ra1,. d i s k ( D R D ) sen·ice e n �1 b l e s O ra cl e Para l l e l Se n n to access a l l Ll\1' d i s k \'O i u mcs re gard less of t h e i r p lwsicll l ocation in the c l uster. Secon d , the Oracle Par,1 l l e l Sen er uses D ig i t aJ ' s d istri b­ uted lock maiLlger ( D LM ) to S\'ll c h ronize a l l access to s h a re d resources ( s u c h as i tJ m e m ory cache blocks o r disk blocks) ac ros s a T r u Cl u ste r sysrem . the system u n d e r tes t , \\'e used t h e D RD sn v ice s to balance t h e d a ta base across the T r u C i u st e r m u l t ic o m p u t er syste m . The c on fi g ­ uration i n c l u d e s a s p ec i i l o ws . In the The rest of the paper next sec t i o n , we p rovi de a svnopsis of the T r u C i ustcr map tra n s fe rs ti·o m t he M E M O RY m c m o r v :1cccss i nstructions. Th e u se of m e m or y store i nstructions ra ther t h a n spe c i a l 1 /0 i nstructions p rc)l 'i d e s low I J tc ncv ( tii'O m i crosecon d s ) - t e c h nol o gy and i ntroduce the O ra c l e Para l l e l Scn·er, a n d Jow OIT rhcad f()r a tra nsfl:r o f a m · J e n gth . Tr u C i u s tcr tcc h nolog1· with the Oracle relational up to ei g h t ( t(w r r(>r co n 1 m c rc i a l D LM / D RD - based d atabase management S\'Stc m . F o l l owi ng th;lt, we gin: app l ications ) c l u ster me m be rs c o n n ecte d to a c o m ­ b c n c h m ;Jrk . Next, 11-e d esc r i be the system u n d e r test and o u r ped(mnance supp o rted wi t h i n a c l u s te r arc Al p h aSe r ve r systems of ev;ll u a t i on m e thodology. T he n we d i sc u ss o m pc r f<>r­ varyi ng processor s peed Jnd n u m b e r of processors. a n opti on a l Ora c l e prod u ct tlut enables t h e u ser to use an m- c n·i c ll ' of t h e T P C - C mance meas u re m e n t res u l ts and co mp a re t h e m with T b e Tr u C i ustcr r(>r D igi t;l l U N I X prod u c t su pports m o n cl uster imcrcon ncn. The compute r The m e m be r systems r u n :1pplicarions ( tor svstems exam p le, com peti ti ve vendor results. Fi n a l l y, 1vc present our user appl i cations ) , as we l l JS m o n i t o r the sta t e of e a c h conc l u d i ng remJ rks and d isc u ss our tl l tlll'C work . m em be r syste m , e a c h s h o red d is k , t h e M E M O RY C HAi'l N E L i nterconnect, a n d the netlvo rk. These TruCiuster Clustering Technology c l u ste r m e m bers com m u n i cate o1·er the M E l'd O RY D i gi ta l ' s Tr u C i us t e r c o n fi gu rati on consists of inter­ c o n ti gu ra t i o n consists of a M ENI O RY C H A N N E L C H AN N EL i n tercon nect.'' � A M E M O RY C H A N N E L con n ec te d c o m pu ters ( u n i pr o c e ssors or S M Ps) and e x te rn a l d isks con nected to o n e or mor e s h a red , s m a l l co m p u t e r systems i n te r C1 ce ( S CS I ) b u ses provid i ng ad ap t er i nsta l l e d i n J l'C I slot a n d l i n k c a b l es t o c on ­ nect the ad a pte rs . I n J co n n gu ra ti o n \\'ith m or e t h a n two m e m be rs , the M EJVI O RY C H A N N E L ada pters s c n· i ccs to c l i e n ts . '' It presents a s i n gl e rJII' vo l u m e are co n n ect ed to a 1YI EMO RY C H AI': N E L bu b . A i ty t h a n a si n gle system and be tt e r sca l a b i l i ty t h a n a n CHAN N E L h u b is shown in figure l . n a i1K spac c t o a c l i e n t with better a ppl i cation a v ;l i la b i l ­ ty pi c al Tru C i ustcr c on fi gu ra t i o n with a M E M O RY SM ! ' . A Tr u C i u ster c o n fi gu ra t ion s u p ports h i gh l y par­ a l l c l izcd d a ta base m a nagers, such as t h e Oracle PJ ra lkl A p pl ica t i on s ca n atta i n h i gh ava i l a bility by con n ect ­ i ng two or m o re member svstcms to one or more ,- - - SERVER 1 SERVER 2 M EMORY CHANNEL ADAPTER MEMORY CHAN NEL ADAPTER SHARED STORAGE LOCAL STORAGE F i g u re 1 A Tr u c : l ustc r Contigmation ll' i th M E M O RY C H AN N E L H u b DigitJ! Tcc h n ic;t\ ]ollm;t\ Vol . 8 No. 3 1 9% 47 s h a re d S C S I b u ses , t h u s con str u cti n g an A 1 a i l a b l e S e rver Em·i ronmcnt ( A S E ) . A s h a re d SCSI b u s i s re q u i re d onJv not fix two- m e m ber con tigura tions th �l t d o have a M E M O RY C H A N N E L h u b . A l th o u gh M EMORY CHAN N EL is t he o n lv s u ppo r t ed c l u ste r i n tc n: o n ncc r, Ethernet :md i m c r raec ( FD D I fi ber d i s t ri b u ted c l uster mem hns . The c l u ster comnH1 1 1 i cuion sen i c e is used to a l l oc :1 te the lVI E M O RY C H .\ N N L I . a d d ress space �l lld Jll: pmting o f c l us ­ :1 ppl ications to r u n ll' i t h i n t h e c l us te r, r ega r d l es s o f ll 'herc in the c l uster the �J C t u :� l physic:�! s to ra ge resides. Therdorc a D RD s e rv i c e a l loll's the Oracle Par�1 l l c l S e r v e r pa ra l le l access to stoLJgc media ti·om m u l t i pl e T h e fi1'C tl' pcs of ( of 1 0 or d e r l i n e s ) TPC-C transactions arc l i ste d i n h·om a ware h o u se r h m u g h the c o rrespon d i n g stock level f( Jr c�1 e h i t c ll l . N i n e ty ­ of rhc rime the s u pp l v i n g warehouse is n i n e pe rc e nt WAREHOUSE W WAREHOU S E 1 1K . . . . 3K · · · D I STR ICT 1 0 3K 1K · 3K 1K Source: Transaction Processing Periormance Council, TPC Benchmark C Standard Specification, Revision 3.0. February 1 995 . Fig u re 2 H i erarch ical Relationship l)i�iLli Tcdm ical f o u n d 111 the TPC - C : B usi ness t-: m·ironment Vol . X �o. 3 1 '>')6 I CUSTO M E R CUSTOMER CU STOM E R 1K · I I I CUSTOM E R DISTRICT 1 - · · D I STRICT 1 0 D I ST R ICT 1 �1 s i n g l e th tab:Jsc tra nsacti on ; it i nse rts t h e order ,m d u p d a t e s COMPANY 4X h i e ra rclw The em·i ro n m c m is s h o11·n i n ···· 3K WA REHOUSE DISTRICT 10 W ' 1 0,95,0.00095'W W,89,0 000089'W � 1 00K STOCK W ' 1 00K.306.30 6'W jw H ISTORY W,89.0 000089'W - 3+ N EW-ORDER W'9K.8.0 0 1 6'W+ ITEM 1 00K,82.8 2 1-l_ ORDER-LINE W'300K+.54 , 1 6.2'W+ 3K CUSTOMER W'30K.655. 1 9 65'W � � 5 - 1 5 ,-- - � 1+ ORDERS W'30K+,24 .072'W+ ----- KEY. TABLE NAME CARDINALITY. APPROXIMATE ROW. TABLE SIZE LENGTH (BYTES) Note: + CARDINALITY OF RELATIONSHIP implies variations over measurement interval as rows are deleted or added. F i g u re 3 T P C : - C : D �m b�1sc T�1 b l cs Re lationship Ta ble 1 TPC-C Re q u i re m ents for Percentage i n M i x, K ey i n g Time, Response Time, a n d T h i n k T i m e ' M in i m u m Percentage in Mix Minimum Keyi ng Time (Seconds) N ew order N/A" 18 Payment O r d e r status D e l i v e ry Stock l evel 43 4 Transaction Type Notes ·' Table 1 4 4 3 2 2 2 90th Percentile Response Time Constrai n t (Seconds) Minimum Mean T h i n k Time D i stribution (Seco nds) 5 5 5 12 5 5 5 5 12 10 is p u b l i shed i n the Tra nsact i o n Processing Performance Council's TPC Benchmark C Standard Specification, Revision 3.0, February 1 995. ' Not applicable (N/A) because the measured rate is the reported through put, though it is desira ble to set it as h i g h as poss i b l e (4 5 % ) . t h e local w a rehouse, a n d on l y one perc e n t o f rhc rime a c t i o n pro cesses ord e rs correspon d i ng t o 1 0 p e n d i n g p a y m e nt for a customer, u pdates t h e cus­ corresponding e n tr y i n t h e new-order t a b le i s a lso is it a re m ote warehouse. The payment transaction processes to m e r's a b :t l a nce , a n d rdl e c ts the pav m c n t i n t h e d i s ­ trier a n d ''':tr e h o us e sales statistics. T h e c u stomn resi d e nt II ' J re h ou s e is the l oca l ll'a rc h o u se 85 pe rce n t orders, l tcx each d istrict with l 0 i te ms per ord e r. T h e deleted . The d c l iv e rv transJction is i n tended to be e x e ­ a queu i n g meclu­ nism, rather thJn bei n g e x e c u ted i n teractive l v ; t he re is c u te d i n d e te r red m o d e t h ro u g h of the ti m e :t n d is the remote wa re house 1 5 percent of no terminal response i n d i c a ti n g the trJnsaction com­ t h e t i m e . T h e order-status tra nsaction re t u rn s the st:J.­ pletion . T h e srod:. - ICI'el tus of a c u stomer order. The customer ord e r i s s e l e cte d q u a n t i tl · of stock 60 perc e n t o f t he time b1· last n a m e md 40 percent of last 20 orders i n a d i s tr i c t :m d d eter m i n es t h e i te m s t h e ti m e by idcnti tlcati o n n u m ber. T h e d e l ivery tra n s - that h a v e a stock i c l · c l belo11 a s peci fi ed threshold . tr:t n s action ex�'lmi ncs t h e r(x t h e i te m s ordered b � · e a c h o f t h e Digiral Tcc iJnical ) < >urnal Vol . 8 No . .i 1 996 49 The TPC- C : pe rt(mlJ:tnce metric measures t h e tot<1 1 n u m b e r of new orders com p l eted p e r m i n u te , 11·i rh •1 \\'Ol'kl oad . T h en we d iscuss t h e tes t i n g st rategv used to e n h :tnce t h e per rc>n11:1 nce oft he S L'T 9 0 t h perce mi l e response - t i m e constra i m o f S sec o n d s . We show the confi guration o f t h e c l ie n t-server SUT T h i s me tric me:tsu t-cs the b us i n ess t h ro u g h p u t cn h e r i n h g u re 5 . The server SUT consists o f a TruCl uster t h a n the transaction e x e c u t i o n rate . ' I t i s e x p ressed i n tra nsactions- per- m i n u te C ( tpmC ) . T h e metric i m p l i c ­ c o n ti g u rJtion \\'i t h fo u r n o d e s ; cK h node is an A l p h <1Senu i\ 400 5/3 5 0 svste m with e i g h t 3 S O ­ i t � \' t a k e s i n to acco u n t <1 1 1 the transaction t\'JXS :ts t h e i r mega heru (!'v! H z ) C: P U s a n d i\ giga !)l'tes ( G B ) o f i n d i ,·i d u a l t h rough p u ts are con trolled b\' t h e m i x p e r ­ m e m mY T h ese nodes are con nected toge t h e r b\' l . The t p m C is a l so d 1·i, c n b\' the acti,·it\' o f e m u ! J ted users and the ri·e q u e n C\' of lv! E M O RY C H A I': l\: E L l i nk c a b l e h·om t h e M E I\ 1 0 RY ccn t:tge gi\ e n i n Ta b l e c hec kpoi n ti n g . "' The cvcle t(Jr generJting <1 T P C : - C transaction b y a n e m u l a ted u s e r is s h own i n h g u r e 4 . T h e transactions a r e generated u n i t( m n l y a n d a t ra ndom w h i l e m a i nt a i n i n g :t m i n i m u m perce mage i n a CHA:\ � E L <1cb pter on t h e node to a single M Elvl ORY C H AN N E L h u b . The locJ I storage c o n ri g u ration klr eJch node consists of 6 HSZ4 0 red u n d a n t a r rav of i nexpe n s i ve d i sks ( RA I D ) contro l le rs , 3 1 RZ28 a n d 1 4 1 RZ29 d is k d ri,·es, conn ected t o t h e n o d e bv SCSI buses to 6 K7ol'Sr\ d i sk adapte rs . f m t he r, e a c h n o d e i s T J b l e l gi,·es t h e m i n i ­ each transac t i o n t\'fX , t h e c o n nected t o 1-'DDI lw a DEFPA F D D I :td ap te r. The m i n i m u m ke,· i n g r i m e , t i J e m a x i m u m 9 0 t h percentile nodes co m m u n i u te \\ i t l1 the c l i e n ts 0\'C r t h i s F D D I . m i x for e a c h tranS<1ction t\' p e . m u m m i x p e rce m:tge t( Jr response - t i me consn·:� i nr , and the m i n i m u m t h i n k T h e c l i e m S UT consists of 1 6 A l p h :tSenn 1000 T h e d e l i\·ery transacti o n , u n l i ke the other tr:t n s ­ 4/2 66 S\'Stems, each ,,·ith 5 1 2 MB o f memo n·, one RZ28 d i s k d ri,·e , and one D E F PA F D D I adapte r 1 2 The t i m e d ctin e d lw t h e T P C : - C : specificati o n . m c l i e n ts associated with eac h node Jt'C i n te r val a n d the l e ngth of the c heckpoi n t i n g i ntcr­ connected to \'a l . The TPC-C: speciticnion, however, d c tines the D EC : h u b 900 p md ucts contains t\\'O concentra tors, a DECh u b 900 swi tch . Each of the t() u r u p pe r bou n d on t h e c h ec k p o i n t i n g i n tuv:t l to be one 30 m i n u tes " ( UT P ) coiKentrn 1 1ance i nst�J I !e d o n each c l u s ter m e m ber. The software config­ e\·a l u ation of t h e Tn1 C i usre r S\'Stem u n d e r t h e T l'C : - C : urati on i n s LJ i l e d on e :t c h c l i e nt S\'Stem is t h e Digital S E LECT A TRANSACTION TYPE FROM MENU MEAS U R E MENU RESPONSE TIME KEY T I M E : TO ENTER R EQU I R E D INPUT FIELD DISPLAY SCREEN M EASURE TRANSACT! ON R ESPONSE T I M E THINK TIME: WHILE SCREEN REMAINS D I S P LAY E D Fig u re 4 c,·c l c ri:>r Ccnerating 50 a Digi u l Tcch n i c1 l J o u rn,11 TPC-C Tra nsaction l)\' an Enl l l L1 teci User 1 --- DISPLAY DATA 6 HSZ40 RAID CONTROLLERS 6 HSZ40 RAID CONTROLLERS M EMORY CHANNEL H U B 3 1 RZ28 AND 1 4 1 RZ29 DISK DRIVES 8-CPU. 8-GB ALPHAS ERVER 8400 5/350 SYSTEM 6 HSZ40 RAID CONTROLLERS FDDI 3 1 RZ28 AND 1 4 1 RZ29 DISK DRI VES 8-CPU, 8-GB ALPHASERVER 8400 5/350 SYSTEM 6 HSZ40 RAID CONTROLLERS FDDI 3 1 RZ28 AND 1 4 1 RZ29 D I S K D R IVES 3 1 RZ28 AND 1 4 1 RZ29 DISK D R I VES 4 VAXSTATION 3 1 00 WORKSTATIONS 4 VAXSTATION 3 1 00 WO RKSTATIONS 4 VAXSTATION 3 1 00 WORKSTATIONS 4 VAXSTATION 3 1 00 WORKSTATIONS 16 ALPHASERVER 1 000 4/266 SYSTEMS Figure 5 Cl ie n t-Server System u n der Test U N I X ve rsi o n 3 . 2 D o pera t i n g system and rbe BEA Tu x e d o Systcm/T ve rsi o n 4 . 2 rr.ms:K tion p r o cessi ng m on i tor. t:u r t h c r , each RT E ru ns the O p cnV MS oper­ a t i n g svstcm and a p r op ri c ta rv e m u l a tion p:1e bgc, VA X I\T E . In the re m a i n d e r of this se cti o n , 11·c d i sc uss the r e s t i n g srrareg1· used to gc n c r :nc the rr:msac tions on the h·onr end . Then \\'e d isc u ss the t u n in g done on descri bed i n Ta b l e I ) with a u n i q u e seed . This ensures the mix is we l l d c ti ncd a nd a 1·aricrv o f transaction t ypes arc r u n n i n g cotK l l tTcntlv ( to better s i m u L n c a real -ll'orld uwiro n m c n t ) . We had a local aro tr:: tn s ­ porr ( LAT ) conn e c ti o n onT e m u l ated user and E th e rne t betll'ec n each a c or res po n d i n g TPC c l ie nt t( >rm process on rhe c l i e n t t(lr ta s te r com m u nicati o n . \Ve the bac k. e n d ro achin·e the m :n:i m u m possi ble tp m C s h o ll' the com m u n icnion b<:t11·een an RT E, a c l ie n t , mc1s urc mcnrs ti·01n t h e SUT. a n d a scn·cr i n figure 7 . u s e d a series of RT E s ro d ri 1· c the S U T . The o n e - to ­ sp o nd ing to a tra nsaction rvpe, w h i c h a l lowed us ln con r(mnance \\'it h the TPC-C spcci ti cni o n , \\'C on e corres p o n d e n ce between c m u Ln c d users on t h e vVc b u i l t ti 1'C ord e r q u e ues on each c l i e n t cor r e ­ ro con trol the tr: msaction pe rce n tage m i x . A T P C c l i e nt 1\T F. :m d the TPC c l i e n t fo rms o n the c l i e n t req u i red to rm process q u e u es tr:lllsactions gen e ra ted bv r h c us ro d e ter m i n e the max i m u m n u m ber of u se rs ro be e m u l a te d users t o the :1 p p ropriate o r d e r q u e u e u s i n g ge n era t ed lw the RTE . T h e m a i n bctor we used ro d e tcnn i n c the n u m be r of users \\'as the cl i e nt ' s m e m ­ orv s i z e . We assumed t h a t o n a c l i e m, 3 2 M B o f m e m ­ Tuxedo l i brarv calls. These transaction requ ests i n e a c h q u e u e a rc processe d i n a tirst i n , fi rst o u t ( r l r O ) order b v t h e Tuxedo sen·cr processes r u n n i n g o n the tor client. We h a d 44 Tu x e d o scn·e r p roce ss es th:n 11·crc constra i nts, c1ch RT E generates 1 ,6 2 0 e m u l ated users . \\'ere d istri b u ted s o r h a r r h e n u m be r of Tuxed o sen·cr T h e e m u btcd u sers then generate tra n sactio n s ran ­ processes d e d i c a te d to a q u e u e w a s d i recrh' corre L 1 rcd or l ' is u s e d tor t h e opera t i n g s1·ste m a n d 0 . 2 5 t\!! B eJch TPC : c l i c llt to r m process . The rdi:ltT, \\ ' i t h these domlv b:1scd on t h e p re d c ti n e d tra nsaction m i x ( a s nor e1·e n h · d istri b u te d :1 mong t h e 5 order q u e u es b u t to the pc r ce n t:1gc of the m lrkl o a d h a nd led lw t h e Digitl'e the dat:� b ase hit rati o . 13eGl llSC <\I I vendors sub­ mitting T P C - C re su lts u s e t h i s stvlc of tuning, the per­ t( mnance l i m i tat i o n tor TPC:-C is usu:� l l v t h t: back-end server's CPU powe r. In fac t , rests have s hown that i f this method of t u n ing i s n o t tol lowcd o n rhc back-end server, rhe user will no t obtain the opr i m J I TPC-C pe r­ t(mnance res u l ts . I ns tea d , the tests reveal J back-end s c n n c on figu rati o n that has 1 10t fu l l v util i zed the scrH:r's pote nrial Lw h aYi ng u n baLmced C P U Jnd ljO ca p ab i l i tie s . T h is tvpc o f con ti gu rJ t i on nor onlv red uces the se n·cr's through put c a pa c i t\' b u r :1 lso adl nsc h­ aftccts the price/performance of the S U T. On t h e back end , 1\'e used Tru C l uster t cch nologv te�ltures to a c h ieve t he max i m u m possi ble tr:�nsr t h i s conti.guration, a system of this size s u ppo r t i ng the Oracle Para l l el Se r v e r req u i res 1 million locks ( o c c u p y i ng l C B of m e m o r y ) t()r the DLM when u s i ng S . O GB o f m e morv to r the SGA. Agai n b e u ristiCJ l lv, we o bsen·cd t h :.J t if we used less m e m o rv for the D L!Vl, rhc rot:� I n u m b e r of locks per page ll'aS red u ced . The decrease in locks per pJg.c i n c reases conte n tion �K ross nodes a n d hence red u ces the tpm a s rhe n u m ber oh10des increases. With t h e h e l p of e n gi neers ti·o m D i gi tJ l ' s M E M O RY C H A N N EL Group, 11 e \\Tre able to usc J h a rdware data :� n a ly z cr to m e �1 s ur e t he pe rc e n t a ge of the M EM O R Y C : H t\ N N EL i merconnect's band11 i d th used when r u n n i n g the T I�C-C: be nch m :1rk. By usi n g the d J t a ana lyzer, we determined t h at we do not a p p ro a c h saru ration of t h e PCI - based M E M O RY C HAt"JNEL h a rdware d u r i n g a TPC-C rest, even though i t is c a p:1 bl c o f s u sta i n i ng a peak t h r o u g h p u t rare of 1 0 0 J'vi B / s . In bet, ll't o bsen · e d that rhe M E M O RY C H A N N E L bandwi d th 11·as nor saturJtcd; a TPC-C tes t req u i red a pc1 k throughput rare of o n l y 1 5 M B /s to 1 7 M l3 /s ti-orn th e l'vl E M ORY C H A N N E L . As stJted previo us h·, the b c n c h m r J m i n i m u m of 1 60 m i n ­ u tes . '" T h e measur e t n e n t o n t h e S LI T began approxi­ mately 3 m i n u tes a fte r the s i m u l a ted users had begu n e xec u t i n g tr:\llsacti u n s . The m easure ment period of Beu use t h e M EM O RY CHAi'JNEL i n terconnect not 1 2 0 minu tes, hm1·e1·er, started arter the SUT attained ;1 o n k h ;ls s u fticient ban d"id t ll tc>r T P C : C b u t a lso pro­ stead1· state i n a p proxi m a te[\- 3 0 ,·ides c x ce.l l c n t l a te n c1· ( less than 5 microseco n d s ) , ll'e ment 11 ith t h e T l' C - C : specification , we p e dc>rmed ;l tT 3 I ')'J 6 more to e m u rc the rcprod u c i b i l i n· of the m:n:i m u m mc1surcd rpm C. Dig i t:� ! Equipmellt Corpor a t i o n and Oracle Corpo cH i on :� !so present a pricc/ped(mnance ratio o f $ 3 0 S p er rpm C. I n T:�ble 3, II'C prese n t the rota! occu rrences of each tra nsaction tq1C �1 11d the percen tage n-�m sacrion m i x used ro generate t h e tra nsactions i n each rest r u n . vVe com pa r e tilL' t)erc c n ragc rr:msacrion m i x i n Ta b l e l . Acknowledg ments M �my peopl e \\'i t h i n se\'eral gro u ps �1 11d d iscipl i nes 6. t h e t(> l l owing ind ivid uals trom D i g i t al : Lee A l l i son , B h agyam M oses, Ruth 7. W. CSD Pc rtormance Group; Brian S tc\ ' e n s a n d j i m Wood \\'a rd , D i g i t a l u�IX Pc r t( m n �l n c c Tc:ll11 m e m ber; Scan Re i l l v, D o u g 'vVi l l i a tns, a n d Z:1rb c ;, c t:t n m i c o f t he AVS Pertormancc (; m u p ; �md Don 8. Vi p i n C o k h a l e of the O rac l e Perr(mmncc G rou p . References and Notes I . Throughour th is pa pe r, ll'e u s c r h e rcrm c l u s r c r i n tcr­ W K o h l e r , A. S ha h , ;1 nd f . Ra�1 b , "On: r , ic\1 o f TPC lk n c h l n ;uk C: The O rd n E n tn lk n c h 1 n ark" ( Trclns­ ;lc t i o n Pmu:ssi ng: Performance C :o u n c i I, Tec h n i cal 1\cpon, l) cce m b n 1 99 1 ) 3. TrJ nsacrion Proc.:ssing Pe d( m n a n cc Cou n c i l , ll'C /!euc/JIIIrtd' (.' Sian do rd ,\jJeciji'utl lou. N£ 'n'sion , ) . 0. h: b rtL1 r\ ' 1 9 9 5 . 56 Softll':lr<.: Ca rd o z cl , F. C l o\'cr, a n d W . S n a m a n J r. , " Design o f 5-1 7. 7i·uCius!er .')ujitmre. Hordu w·e COJ!/i'g u ra ! ion ( JVLl \' lu r d , M .1. ss . · D i g, i t .1 l E q u i pme n t C:orporcuicl l l , 9. J i'II CI!Is/cr. Sujitt·ru·e lus!alla!iull and C(,uf%� u m ­ tion ( M a\' ILlrd , M .1ss . : Digital Eq u i p m e n t Corpor:l­ t i o n , O rd e r :\' o . AA - (2 1 .8 ,\ IA-TE, September 1 99 5 ) 10. C hec kpoi m i n g is a p mce ss ro m a ke the cop\' of t h e d a tabase records/pages on t h e d ur a b l e m ed i a c u r r e n t ; S\'Ste m s do n o r 1\' ri r L· r h c mod i ticd records/fJages of t h e daL1bc1sc ,H rh e t i m e o f the mod iticarion b u r at c l l �ln ge a blv ll'ith TruCiuster. 2. Corporation, Order N o . AA - Q L 8 l .A - I t-: , Dece m b n 1 99 ;) ) . H �1rherr and Pa u l ine N is t , t h e te st sponsors. LJstk, \\'e �1 nd Eq u i p m c nr U N I X t-: m· i r o n l l l C I H , " nt:l; i!ol Tech nical jou nwl. · \\'O u l d l i ke ro t ha n k Jef Kennedv, Pcrcr LJ i , K�1 rl H �1as, D i gi t a l ' ol . 8, no . I ( 1 996 ) : S�mj:1v NarJhari, Dave S ta n l c,·, :111d G reg T�1 rs�1 o f the PCJ ," ( h b r u cH' \' 1 996 ) : 1 2 - 1 9 . rhe Tr u Ci usrcr 1Vl u l ri c o m p u t e r Syste m fo r t h e Digi r J I iV!orgc n s te i n , T i m B u rke of t h e D i gi ta l l! N I X Eugi nc<.:ring G roup; I 1 6, no. Prod u c t Descri ption 6 3 . 9 2 , October 1 99 5 ) . Ro ger Desc he nes, Tared' K:�w:�f, M :t r i a Lopez, Joe ivlcl-'add c n , 1 ol . TmCIIIsler ji11· 1 Ji,� i!o/ I .\IX I (T-;fon I 0 ( M a l ' l l a r d , M as s . : i n both D i g i u l and Orac l e co n t r i b u ted to t h e s u ccess o f th i s pertl lows : l. Forty-three pe rce n t -A new-order tra nsaction p l aces an o rd e r ( a n average of 10 Jines) ri·om a war e ­ h ouse t h r o u gh a si ngle database transaction and upd a t es the corr e sp o n d in g stock le\·el tor each item. I n 99 percent of the ne w - ord er transactions, the supplyi ng warehouse is the local wa rehouse and o n ly 1 p erc e n t of th e accesses are to a remote w are h o use. 2 . Forty - th re e pe rc e m -A payment transaction processes a payment t()r a customer, u p d ates the cus­ tomer's balance, and reflects the payment in th e d istrict and wa re h o us e sales statistics. The c ustomer rcsidem warehouse is t h e home warehouse 85 per­ cent of the ti m e and is th e remote warehouse 1 5 pc rce ll t of the ti m e . 3. Four percent -A n order-status transaction returns the status of a customer order. The customer order is se le ct e d 60 percent of the ti me by the last name and 40 percent of the time by an identification n u m ber. 4 . Fo u r perce n t -A del ivery tLmsacti o n p ro ce sses o rd ers c o r res p o nd i n g to 10 pe n d i n g orders tor each d i st ric t wi th lO i te ms per order. The co rre s p o n d ing entry i n the n e w- o r der table is a lso d e l et ed . Th e d e l i very t ransa c ti o n i s i mend ed to b e e x ecu te d i n deferred mode t h ro u gh a queu i ng mechanism . There is no term i n a l r esp onse for co m plet ion . 5 . Four pe rcen t -A s to ck -l e ve l t ra n s ac ti on CXr D ig i ta l's Al ph a environ ment. As a group chartered to measure database p e rtorma n ce on Alpha syste ms, D i gi tal ' s Co m p u ter S ys t e ms D i v isi on ( CS O ) Perfrmancc gai ns . Figure I shows code seq u e n ces that Digital's C S D Performance Group has given to d atabase ve ndors to i m p ro,·e locking i n trinsics i n the Alpha em ironment. These code seque nces can be used to i m plement spi n ­ locks i n the D E C : C compil e r on the Digita l NIX operati n g svst e m . Using O M Feedback As previously m e n tioned, O M is a post- l i n k rime opti ­ m izer available on the Digita l U N I X operating system . I t pertorms opt i m i za tions such a s compression o f add ressing i nstructions m d dead c o d e el i m i n a tion t h rough the usc of fee d back. The pertonnancc i m p rovem e n t provided Lw OM o n Alp h:t 2 1 1 64 systems is d ramatic f(x rhc tc> l l oll'ing two reason s . ' • The 2 1 1 64 m i croprocessor has a n 8 - k. i l o lwtc ( KB ) d i rec r · m dppcd i nstruction c a c h e , \l·hich m a kes code p lacement c xrrc mch' i mporta n t . In a d i rect­ mapped cac h e , t h e cod e Lwo u r and l inking order maps one rr>r o n e to i ts place m e n t i n cac h e . Thus a poorlv chosen instructi o n stream l a y o u r o r s i m ­ p l y u n l u c ky c o d e p l a c e m e nt vvi t h i n l i bra ries c 1 n a l rer ped(mnancc b y 1 0 t o 2 0 perce n t . Rou ti n e s a re freq u e n t l y page a l i gned , which c a n incrc:Jsc the like l i h ood o f c1ehc c o l l isions . • The h i gh c l o c k rate of the Al p h a 2 1 1 64 m icro­ processor ( 3 00 to 500 megahertz [ iYl H z ] ) req u i res a cac h e h i c rarclw to attempt to ketp the C P U p i pe l i n es fi l l e d . The p e n a l tY o f a first- l eve l cache m i ss is 5 to 9 c\-c lcs, ll'hich means t h a t an Digit�! Tec h n i c a l J o u rn a l Vol . 8 No. 3 1 996 6l / / T E S T _A N D _ S E T i m p l e m e n t s t h e A l p h a v e r s i o n o f a t e s t / / t h e Load- Loc ked . . store-cond i t i ona l i ns t r uc t i ons . / / func t i on / /va lue i s i s 0, t h e store-cond i t i ona l f a i l ed, t h e va l ue of the s p i n l o c k a f a i l u re status (0,2, or 3 ) is retu rned in RO. II I I I I I I I I I I status retu rned in RO i s one of the f o l low i n g : 0 - f a i Lure ( sp i n l o c k w a s c l ear; 1 2 - success fa i Lure ( sp i n lock C sp i n lock was was c l ea r ; now set; s t i L L set ) set, - f a i Lure ( spi n lock was set; set) 3 #def i ne TEST set opera t i on us i ng purpose of th i s t o c h e c k t h e v a l u e p o i n t e d t o b y s p i n l o c k _a d d r e s s a n d , i f t h e set it to 1 and ret u rn success ( 1 ) in RD . If e i ther the spi n lock / /va lue i s a l ready 1 or / / rema i n s u n c h a nged a n d / /The and The AND SET s t i l l s t i L L ( s p i n l o c k _a d d r e s s ) c l ear, asm( s to r e-cond i t i ona l s t ore-cond i t i ona l fai led) $0, ( $1 6 ) ; " Ld l "or "st l "s l l f a i led) $0, 1 , $ 1 ; c $1 , ( $ 1 6 ) ; $0, 1 ,$0; $0,$1 ,$0 "or \ \ \ \ \ C s p i n l o c k_ a d d r e s s ) ) ; I I I I I I I I I I B A S I C _ S P I N L O C K_ A C Q U I R E i m p l e m e n t s t h e s i m p l e c a s e o f a c q u i r i n g a s p i n l o c k . I f the spi n lo c k i s a l ready owned o r t h e store-cond i t i ona l f a i l s , th i s func t i on spins unt i l spi nlock i s #def i ne { the sp i n lock acqu i red . i s acqu i red . Th i s funct i on doesn ' t return B A S I C _ S P I N L O C K _A C Q U I R E ( s p i n l o c k _ a d d r e s s ) Long status wh i le { 0; = ( 1 ) i f { C * C s p i n l o c k_a d d r e s s ) { = = s ta t us = TEST i f ( s ta tus == 0 ) AND 1 ) SET ( s p i n l o c k_ a d d r e s s ) ; MB; b r ea k ; } unt i l the \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ F i g u re 1 C(ldc Seq u e nces tor I ncki n g; l nr ri n sics i n srruction - c a c h c m iss r:nc o f 1 0 ro c tfecti ,·cl,· s ta l l the C P U 7 0 to ri m e . 1 2 p c rce nr can • C o n v e rse l 1·, d e c re a s i n g t h e m i ss rate b1· • 2 pnc e n t can i n c rease t h m u g h p u t bl· 10 percent. O lv\ pertorms pro fi le- based opti m i za t i o n . A p ro - r eg i ons con tai n i n g o n h- one en trance :1nd one e x i t ) , to cou n t t h e n u m ­ ber of ri mes each block. is exec u ted . The i ns t r u m e n ted version of t h e program is run r h ::t r conr a i ns �l u s e s rlle p roti l c ro cre�lt<:: �l ked back ti l e of h�1sic b l o c k cou n ts . OM r h e n ked back t o rcuTa n ge the b l ocks i n a n opti m a l \\'<)\' r() r t h e ti rsr - ln · e l cac hes O i l t h e A l p h a c h i p . T h e d e tai I s of t h e proced ure t(Jr using OM m a1· be ro u nd i n rhe m a n page for cc on the Dig i ta l L! l\: I X opcr:1 t i n g SI'Stem b u t c a n be s u m m a ri zed as t(>l]oii'S : Usc p i x i e to prod u c e prog . p i x i e ( t h e i n s t r u mented e xecu u b l e ) and prog . Ad d rs ( :t d d n:s�es ) . • gr�l l l l is ti rs t partitioned i n to b a s i c b l oc ks ( th a t is, �1nd i nstr u m e ntat i o n code i s �1d d e d B u i ld executa b l e ll'i th - n on_s h a rc d ·om options, prod u c i n g prog. 80 perce n t o f the Ru n p ro g . p i x i e to pmd u ce 1)ro g . C :o u n rs , ll'hi c h records the b a s i c b l o c k c o u ms . • Noll' b u i l d prog ag:� i n ll' i t h - n o n_sh a red - om - WL, < > m_i rcorg_reedback. V LM Results ht'- u re 2 s h m\·s t h e i n cruse in th ro u g h p u t real i zed ,,· h e n u s i n g V L M . �ore t h a t t h roughput nearll· d o u ­ m·cr:t ll S\ 'S t e m rhe data base 62 I )i,:itJI Tcc h n icJI JournJI Vol . R No. 3 I 96 M a ny peopl e h·o m :t \ �wi e t\' o f groups th ro u gh o u t D i g i ta l h el p e d r u n e :t nd d e hc r t h e T P C - C res u l ts . I n pa rt i c u l a r, w e w o u l d l i k e to t l 1a n k Lee Al l is o n , Roger De s c h e nes , J o e M c f a d d e n , B h agya m Moses, a nd Cher\'1 O ' N e i l l ( C :S D Pcr tiJrm Ur­ node AlphaSenn 8400 5 / 3 5 0 c l u s ter svsre m running a state-of the - art d atabase a pp l i c a t i on . Tarcef has recei1 cJ rwo exce l le nce a w a rd s rl·om D i g it a l tor his "·ork in T P C - C p erfo r manc e mcJsurcmcm o n the A l p haServer 8000 series. Fol ey, W. Gist, S. Va n Do t· e n , and D . Wissel ! , "The Al p haSc r v er 8000 Series: H i gh -end Server Pl:ttrC:mn .fuu mal. vol . Deve lopme n t," Dif:!Jial Techn ical 7, no. 1 ( 1 99 5 ) : 43-6 S . 2 . At the r i m e r h i s puision 3 . 0, l'<.:bru;ny 1 99 5 . 5 . W. Koh l e r, A. Shah, and R . Raab, Ouerview of TPC l.Jenchmark. The Order Enfly Benchmad<. Te c h n i ca l Report ( Transaction P ro c e ssin g Performance Counci l , l)c.:c c m bc r 1 99 1 ) . 6. MOI'c i n formation a bou t the TPC:-C bench m a rk nuy 1K o b t a i n e d From tl1e TPC: World W i d e Web sire, I mp : //1\ww. tpc .org. 7. j. S h a kshober r better \\'a\'S to col la borate Oil the I nternet. t h e I n te rn e t user pop ul ation has gro\\' ll , so has the Some e x a m p les I n c l u d e tile a b i l i t\' to s h are and d i scuss issues of c o m m o n i n terest, coa u t h o r d o c u m e n ts, :\ lld track proj e c t sr.nus. Al t h o u g h tolL w's W W W is ideal for p u b l i s h i n g i n r(mn ati o n , i t req u i res considerable c us to m i zed progra m m i n g to s u pport colla boration . T h e AltaVista For u m version 2 . 0 prod u c r is both a set of coiLl borati,·e ap pl ic:n ions and <1 too l k i t ( p latform ) that bci l i tates C:1S\', e fti c i e n t , J n d rapid de1·e l o p m e n t of U N I X and Windoii'S NT s\·sre m s . col la bor,Hii'C app l i cations fo r t h e Imemct fo r both I n t h i s paper, \\'e d escribe o u r e x p e ri e n c es i n b u il d ­ ing co l l a borJtion sofu,·arc f() r t h e I nt e rn e t . We begin with ,1 brief d i scussion of W W W tec h n o l og\' and p h i l osophy a n d t h e h·amework of the softw:m: and d is­ grou pware applications. Then we pre sent om design cuss the appl iutions s u p p l i e d b1· Al taVistJ Foru m . Fol l o\\' i n g t h :n , \\'e d i scuss the ,·arious C \ pc r i e nces gai ncd in dn-c l o p i n g sofu\'Jrc to r the nc" I n te rn e t parad i g m . vVe con c l ud e the paper lw d iscu ssi ng o u r p l a ns tor fu tu rc de1·e l opmenr eft(>ns. World Wide Web Tech nology Tod a\''s Inrcrnct \\'JS origi n a l l y c o m p u ter ne t"·or·k that a go,·c rn m e n t - funded fa c i l i tated c o l l a boration among :1c1de m i c researchers. l n r(mllation exc h a n ge \\'J.S c o n d u cted and til e lw mcms of ele ctro n i c m a i l ( c - ma i l ) o,n t i m e , b u l l e ti n - b oard st\k tr�1 11Skr. d i scuss i o n s "·ere s u pported b\' the N c t\\'Ork Ne\\·s ( N NTP ) , ll'b i c h propagJted rc �tual discussion t h reads ro a large n u m be r of NNTP senns Tra n s fe r Protocol for viewi ng. \V i t h the d evelopm e n t of the W W W tec h ­ n o l ogv, c o l b hor�l t i n g over the I n te rn e t h ;�s become e\'e!l eJ.SJcr. The vV WW tec h n olo gy consists o f the h > l l o" i ng c l e m e n ts : • U n i i'Cl'S<11 rcsouree locator ( U RL), a com e n t i o n ror i n t(x mJri o n n ;l l n i n g and l i n k ing • Hype rtext markup l a n guage ( HTlvl L ) , a text-based JangtL 1ge f(lr i n ftlrnUtion renderi n g 66 Di1;iral Te chnical ) o u rn.ll • c l i e n t-sen-cr protocol r o tnn sport i n form a t i o n associ:tted ,,·ith a U RL • vVe b browser, a program th:tt renders H T M L d o c u ­ m e n ts , provi des U R.L cac h i n g , a n d s u pports a d i rector\' fi.lr U R.Ls • Todav, severa l c o m p u te r :t ppl ications faci l i tate s u c h Hvperte x r Tra n s fe r Protocol ( H TT P ) , a s i m p l e vVc b server, J col l a boration . • i n fi.>rmJtion fi·om the We b browsers • W W W te c h n o l ogy h as transformed t h e way users i n fi.>rmation through co m p u te r applications are M a n a gem e n t of a set of users a n d g ro u p s S to ra ge of s h a red i n ti.mnation i n a d ata b a s e ( some ­ rimes with rcplicnion Clpabi l i ty ) Information Access access t h ese support t h e fo l l ow i n g capa b i l i ties: • s e r ver that responds to req u ests for C o l l ectil-ci l', known a s grou p'''<1 re . Lotus Notes is a popu l a r group­ ware appl ication . '1\-pi c a l h-, grou pware appl ications networks. Viewi n g i n tcmnarion sto r<.:d i n t h e d a ta bases by m e a n s of a gra p h i c a l i n te r t:Kc • Protec tion of the c o l l a boration e n v i ro n m e n t when Access to i n ti.m11ation on the I nternet was p r i m aril\' nec<.:ssary t h rou gh a u t he n tication ;md access con rroJ W vV W , users :tre �1 ble to :tccess tornnr. The com b i n a ti on neous c l i e n t c n,·i ronmcnrs, s u c h as the M i c rosoft tex t- based ; with the i n ti.m11:t ti o n i n m u l t i med i a of fu n c rion a l in· ( i n formation l i n ki n g, gra p h ic l i i nt e r ­ f..1cc, �md u c h i n g ) , extensi bil i n• ( for d e a l i n g with n e w protocols and n e w i n fo rmation t\'pc s ) , case - of us e , and low cost a p p e a l e d to a wide range o f u se rs i n homes, offices, a n d corporations. I n a d d i tion , t h e M os�� i c - s ty l c of " po i n t - a n d - d i c k " gra p h ical I n te r n e t browser has become t h e most w i d e l y accepted user i n tcrbcc fi.>r network computing. The most popu b r use of the W W W tochy is f(lr p u b ­ l i s h i ng i n f(lrmation, a n d t h e proc<.:ss i s compara b l e to G rou p,,·arc SI'Stcms :trc b u i l t to run in homoge ­ W i n d o 11 s · em·i ronmcnt. The1· reh· on specific c l i e n t ­ s e r v e r tec h n ol ogv, w h i c h is often proprietary, to s u p ­ p o r t remote opcrJtion s . T h e popu l a ri n· and rapid growth of the I n rerner a n d the WWW h ave created a n o p e n , u n i1-ersal , and easv - ro­ program i n fi·astr u c t u re t h a t can readily serve several groupware fu nctions. E n g i n e ers ar Digita l 's I n temer Sofuvare B u s i n <.:ss G ro u p recogn i zed the porenti:1l of u s i n g rhe W W W as the u nd c rlvi n g i n frastructure f(Jr grou pware solu tions :m d at the same time saw th�1t the t h e wa1· a n <.:wspa p e r pu b l i s h <.:s or a tclc,·ision station . bro:tdc asts i n ti.m11 a ti o n . The roks of t h e i n fcm11 ation groupware appl ications ;l \'a i l a b l e todav ha1·e fc,uu res prm· i d c r a mi the i n formation consumer arc c l c a rh­ features to the n i zes the pertin en t inform ation, conve rts i t to the a n d the ddi ncd . The i n tcm1 1 a t i o n p rovi d e r ga t h e rs a n d orga­ HTM L scripting fonnat, a n d m a k<.:s i t :w a i la b l e o n a Web scn'Cr. The i n formation consu m <.:r, a fru o b ta i n ­ i n g i n i ti a l access to the vVe b s<.: rv<.:r ( as o n e m i g h t t u n e i n ;o the correct: television station ) , can then browse and search ri.> r various types of i n tcmlution av::J i ! a b l e on r h :1 t server. T h e l i n ki n g c rmation o n \'arious sen·ers to b e casilv p u b l i s hed a l o n g ll' i t h th<.: ori g i n a l i n formation . l n comrast, m u l t i p l e i n formation pr01 ·i d c rs '' ork in col bborJtion ro ge n e rate the c o n t e n t of s h ared i n f( mn a r i o n . For the p u rposes o f this paper, \\'e ,,·i l l assu m e t h a t there is only one n·rx o f uscr- i n f(mlla­ t io n c o l l :� borators. The W W W is usc fi.tl to r m a n y t\'fXS of c o l l a boration . for o :.u n p l c , a project team may ne<.:d to keep track of project status a nd i n d i v i d u a l progress; pcopk with a common i n terest ( e . g . , fi l m e n t h u siasts ) may wa n t to sh are �1 nd d i scuss their views on t h a t to pic; �1 custo m e r s u pport gro u p m a y n eed a S\'Stem t o prov i d e o n - l i n e a n sll'crs t o rea l -world c u stomer problems; or se,·e r a l �1 W W W lacks. Our goal \\'as to add grou p11 arc W W W to fac i l i tate collaborati o n . We starr<.:d e x p lo ri n g t h e i d e a of u s i n g t h e I nrcrn c t W W W t(Jr grou pware appl ications i n th<.: sum­ mer of 1 994 . 131' the e n d of that 1·car, \\T h a d b u i l t a prototype r h ::� t s u p p o r te d t h e s i m p l i fied d iscussion ( b u l l eti n - bcxmi ) katu rcs o Lm i n te rn a l prod u c t known as DEC Nor<.:s . 1 T h i s prototype generated consid e ra ble i n terest a m o n g activ<.: DEC N o tes users who were seeking a s i m i l a r sol u tion b u i l t a r o u n d an l mc rne t i n fl·astructurc. B a s e d on t h e i r kcd back, the prorotvpc \\'as redesign<.:d :� nd became :1 prod u c t . ' B1· S e p te m be r 1 99 5 , \\'e h : t d b u i lt several col bbora ­ W W W. I n a \\'Orkshop ti,·e app l ications to r u n 0 1 -c r the o rga n i z<.:d by the World W i d e Web Consorti u m a n d t h e .Massac h u se tts I ns t i tu te o f Tec h nol ogv, \\'e p a r ­ ticipated in d i sc u ss i o n s on h o\\' r o e x t e n d the W W W tec h n o l ogv t o s u pport c o l l ::� bora ti o n . AJ I t h e \\'mk­ s h op participall ts prese n ted t h e i r ideas to the Collaboration and Groupware :� urhors ma1· ,,·ish t o w o r k o n that the d oc u me n t rog<.:th<.:r. \V\V W Consortium t(>r review.-' Design I n t h i s s<.:cti o n , we s u m marize o u r d esign p h i l osoplw and d iscuss the fra m ework and a p p l ications dn·cl opcd for t h<.: A l taVist:<1 foru m prod u c t . For our d esign , \\'C adopted an object-ori<.:med :t p proa c h , ,,·h ich m e a n t that we wou ld haYc t o m od u l a ri ze t h e Yarious compo­ nents tor re use and m od i fic a tio n Digital Tcch niccl l ) o 1 1 rn :1l . Vol . 8 No. 3 1 996 67 Design Philosophy t i m e ; and a nc\\'spapcr r o u r c o l l a boration soft\\'are . A fte r years of experi ­ :1 n u m be r of d i\'crsc applicnions . Figure 1 shows the m e n ting A l ta Vista For u m tool kit :m d �1ppl ication layers. and process, t h e col l a borati ng to d evelop an open Internet developers reali zed that the I n te rn e t had reached :1 state ofcriticJI mass. In the case o f n e tworks and con nectivitv, reac h i n g critical m ass i s :t tre m e n d o u s i m petlls r(1r agt·cc i n g o n a c o m m o n The tool k i t is • h i c r:t rclw relationsh i p bci l i t\ ' a\'a i l a b l e to those a l so i ncreases . I n ad d i tion , the \'e n· n ature of the r bu i l t - i n d a tabase support.' Another objecti,·e WJS to m�1 kc c o m p l e m e nts <1 1ld look t h e m up using a search tool . • the d a tJbase l i bran· i n the Berkelev U N l X d i s tri b u t i o n . fu nction provi d i n g a h i g h - speed q u e r y fac i l i ty. for less­ l a nguage tor dn-e l o p i n g m ost of o u r a r p l ication a n d i n g to deal 'll ' ith HTM L d i rccrlv and cope \\'itl1 i ts S U IT our software changes over time . Abstract definitions or· user i n ter­ wou l d be easy to porr to a l l the rel e va n t operating race objects a lso tend to si m p l i �1 and c l a ri�' the code system platforms. This p r i n c i p l e g u i ded our selection o f compo n e n ts and h e l ped us iso l a te a smal l set o f p lat­ and create a more u n it(mn appc1rance on the screen. • fo rm -depe n d e n t f'u nctions i n to a spec i a l l i brary for A s stated earl i e r, we tried to take a n object-or i e n ted approach "·hene,·er possi b l e . The alh·antages of ou r modi t\', a n d de l e te various objects. • i n to message c a ta logs ri.1r later tr:mslation to d i fter­ ple bcc:tme i nvol\'cd with the software dn·e l op m e n t . fe asi ble . I n te rn ati o n a l i z a ti o n . An i nrcrn,n iona l i zation tac i l ­ i t\' g l lO\\'i n g : U S E R LISTING, R E G I STRATION CALENDAR APPLICATION DATABASE. I N D E X I NG/S E A R C H . G U I L I B RA RY. ACCESS CONTROL, I N T E R NATIONALIZATION S U P PORT, P LATFO R M - S P E C I F I C F I L E SYSTEM. AND T I M E S U P PORT. . . Fig ure 1 A l ta Vista Forum Too l k i t a n d Appl icJtion Layers 68 B u i l t- i n d a tabase . The appl i cnion uses a b u i l t- i n i s �1 ,·en· s i m p l e re l a t i o n a l m od e l with a n object the need to r soft'll' a re dc,·c l op m c n t for the Internet open software as b u i l d i n g b l ocks w h e n ever poss i b l e . com bination of both C and Tel code cb t<1base r o s t o n: i ts object i nsLm ccs . The data b ase stand�1rd . As more and more u sers access the I n ternet, O u r p h i l osophy a l s o incl uded the re use o f e x isting a that creates the fol lowing i n terEKc components: D • • G Llp h i ca l o bj e c ts s u c h as d e fi n i tions of bu ttons, Parameters a rc p<1sscd to the d i spatcher fro m seg­ roo l bJ.rs, various objects that a rc parr o f a r(x m ments of the URL. The d i s pat<.:her p<1rses the U RL i nt o ( e . g . , s e l ec t boxes, rad io b u t tons, c h e c k boxes, text t h e p i e<.:es that p rm·itk the Ol'era l l control o f t h e pro­ boxes ) , and icons. gra m : D �1t �1 b�1 se e n rries, the defi n i tions o h l w i r J.ttr i b u tes, name, ( �md d e fa u l t values. • Us e r i n te r bce aggre ga te o b j ec ts such as t<.mns, views, d ia l ogs , and • error m essages . De b u l t access control p ol ic i es, i n c l u d i ng d c t� u l t groups, access righ ts , and t h e i r m :.1 p p i n gs , t o c o n ­ tro l w h o c a n access i nd i vid u a l t(J r u m s a n d w h a t actions t h e v c a n rake w i t h i n t h e m . This a pproa c h t: n c a psu bt es t h e d e ta ils i n lo\\· - l e\'e l tions to re use the objects. c:ts\' argu mellts. Ea<.:h to r u m is an i nst�Hl l'C of an app li<.:ation o bject. For example, m a n y d i sc u ssion for u m s are avai l a b l e o n va rious topics. Each d i sc u ssion <1pplicati o n Gln be used to man rum h scr1·n that ( e . g . , n a ti1·e Lm guJg<.: and preferred d ispLw t(mn a ts ) . specific I n a d d i t i o n , grO U f)S c a n be created a n d m od i fi ed r u ms . As d i scussed e r u ms a nd l in ks to pcd()rm ,·arious tasks, c g . , rcgistcr oneself, look u p reams o r join a te a m p crt(mn Air:� V i s ta Forum :� d m i n ­ isrrati,·e tasks (if an ad m i n isrr:�to r ) , and so on . For this :111<.i d iscuss r h e lessons 11·e l e a rn ed a l o n g the \\'J\'. uscr to log i n , In this secti o n , \\'e sum marize some of our experie nces �� As res u l t o t· o u r d ecision to rclv on the VVe b brmYscr �1s t h e u n i\'e rsa l use r i n te r b c e , \\·c had to rcsoh'e some reason , the co m m u n i tY ,·isra is also c 1 l led the s u m m i t . u n i q u e user i nt e r bce issues . B c c 1 u s c \\'C c h ose to usc l n m u c h t h e s a m e \\·a,·, Te l t(Jr dc,·c l o p i n g h i g h e r- b·c l o bjects , \\'C h ad to cope �� tom ,·ist:� keeps track of a l l the t(> r u m s and l i n ks tor :1 gro u p o f users, an d a per­ so n a l vista p e r forms this fu nction Fix a s i n g l e user. with using a n i n terpretive b n g u a ge . We designed the dat;:J b�lse a n d i n d e x i n g a n d scarch i nt c r b ccs based Both tcam and perso n a l vistas ctn own torums that a rc on e x te n si b i l i ty a n d port a b i l i ty go:� l s . F i n a l ly, in t h e nor visi b le to the p u b l i c comnHmity vi s ta . d e s i g n o �· �1ccess con t ro l w e h a d t o cardu l l y wei g h cons o f s i m p l i c i t)' a n d Hcxi b i l i t)1• , r b c pros a n d Discussion M u c h a l i ke b u l l e ti n - board d i scussion gro u p o r D i gita l 's D EC Norcs so fT\\' a re, tb is appl i ­ Coping with the User Interface Defined in HTML Gtti o n permits users to s h a re i d eas o n a set o f re l a ted Ve ry c a r l v i n t h e d esign phase, \\'C decided to m a ke to pics. Users create top i cs and replies th:n to rm a h ier­ A l ta Vista Forum c l i e n t - i n d e p e n d e n t , " i th the e xcep­ a rc h i c a l tree ( a lso kno\\'n as t h rc1dcd topics), prO\·id i ng tion of dtpende nce on tb e vVc b lm)llscr. This decision J 11·as b�1scd on the bet that t h e \Vcb browser was a l rcadv " a v klr users to n avigate through ex isting d iscussions. arc also provid e d . These i n c l u de c h ro n o l ogically n avi ga t­ i n g t h rou g h i tems not read ; l i sti n g u n read i tems o n l y avai lable on most of the pi:. Hf(mm. VVc e x p e c te d b rowse r to become �� u bi q u i to u s nct\vork fi·on t e n d , a l l ow i n g us t o foc u s on b u i l d i n g groupware fi.m c­ a n d selectively read i n g them; a n d sc: t rc h i n g tor topics tions on the server. This mcmt th:H \\'c were bced w i th a n d rep l ies conta i n i n g ccrt:� i n words that \\'ere e n tered the O t h e r methods of read i n g the existing d iscussion d u r i n g a p a r ti c u l ar time period b\' a ce rtain a u thor. ti·c c l y th e t�1sk of d esign ing the user i nrcrbcc usi n g l-JTM L'' S i nce H T M L was C\'Oh' i n g, o u r ti rst step was to ro d di n c gt·�l p h i c a l objects in more :�bstract constru cts d i scuss d i ffe rent topics; this is true t(Jr the fol lowi n g su pporte d by o ur roo l k i t . Each construct encaps u l ates Users can also create m u l ti p l e d i scussion fo r u m s t he s p ec i tics i mo a re prese ntation oL1 gra p h i c a l artifact �1 ppl icuions as \\'el l . i n HTI'-1! L i n the too l ki t . T h u s :1s H T M L e,·okes, or as Doc u ment Sharing T h e d o c u m e n t s h a r i n g a p p l i c J ­ to o rga n i Z-e docu m e nt s o f the same tlJ i d crs. ln ad d i tio n it keeps tra c k of versions of the d o c u m e nts, attach ­ me nts, a n d comments . As w i th t h e d i sc u ss i o n a p p l ic a ­ tion, users can bro\\'se through J n d search f(x spec i tlc d o c u m e nts using a \':triet\' of methods. t i o n e n �1 b les users type i n to h i e rarchica l l y orga n i zed , Newspa per Th e nc"·spaper a p p l i c :�tion l ets u s e rs select a speci fie source of i n t(J rm;ttion :� n d t h e n ddi n e h i rers t o present o n lv those i te m s of pote n t i a l i n terest. A good e x a m p l e of a n i n hJrma rion sou rce o n t h e I n te rn e t is o n e of t h e real - r i me news teeds. Usi n g t h e ncwsp;�pcr a p p l ication, i t i s Jlso poss i b l e t o read a n d m o n i tor o t h e r i nform:� tion sou rccs, e . g . , c - m J i I s e n t to a d istri b u ti o n l is t or i nfo rmation ;� ppcari n g on a set of \V W W s i re s . Calendar The f orum s e L e c t box s . L a n g u a g e -ma p t o L a n g u a g e \ -Label "Select a language : " -LabeL break s . language a d d _o p t i o n Eng l i sh s . language a d d _o p t i o n French 2 1 sel ected I n this e x :Hl1 p l e , a select bm is d di n c d to b eg i n \\'ith �� L1 b c l a n d some s p a c i n g 1: 11d t h e n to contain t\\'O options: E nglish a n d Fren c h , with E n g l i s h as t h e dcb u l t . The val ues l and 2 a rc i n t e r n a l representations o f t h e se lecte d values. Also, the nn p ro S\\'itch spec­ " - " i �ies that t h i s o bject m u st c o r respond to the l a n g uage �lttri b u rc in the database, a katurc that 11·as incl uded to s i m p I in' d a ta base u p d a te . calendar :�p p l ication p e r m i ts u s e rs to e n ter a set of sched u l ed C\'C i l ts ( or a to- d o l is t ) a n d prese n t t h e C\'e nts as a c a l e n d a r ( sometimes c:� U c d a d i a rv ) . T h e a p p l ication s u p ports req u ests to a d d i tems t o the c t l e n d :�r, rhus :� I l o w i n g the c a l e n d a r t o b e 70 are:� needs to be upd�lted . For exa m p l e , a s e l e c t box object on a form m av be defined as fol lows: t h e page d esign c h a n ges, o n l v o n e Vol . tl N o . 3 1 996 a ! J b c l is spe c i ti c d , no speciti C l ­ represent r h :�r l ;1 bc l i n a parti c u l a r t( m t o r tvpeface . Neither is t h e :�c tu:l l S[XK i n g to r l a b e l Note that a l t h ou g h tion is prO\·ided t o break specitied . These d ecisions :trc mad e i n the for u m select box part of the toolkit proce d u re , w h ich trans­ language, w h i c h supported o u r goal of rapid a nd lates this object i n to HTML. iterative deve lopment of collaborative applications Most of the early Web b rowsers were single-window tor the w w w. base d . This l i m i tation was especi a ll y p roblematic for us vVe e x te nd ed sta n d a rd Tel to p rovide a set of com ­ because most of our applications provide some orga n i ­ mands and objects that fo rmed t he AI ta Vista Forum zation t o the i n tixm a tion conte n t . A m u c h more n a t ­ too l ki t : d a ta base, HTM L ge n e ration , access control , u ra l w a y o f brows i n g tor o u r e nviro n m e n t would i n ternationaliza ti o n , user protile m a n age ment, a n d i n clude at least rwo w i n dows: one showi n g the context p l atform-specitlc support. M a n y of th ese e x tensions and the other s h owi ng the content of a speci fi c i te m . s u pported a n object- based e nvi ron m e n t ( i . e . , the For this reason, w e i n troduced m u ltiple n a \rigational enviro n m e n t s u p po r ted sta n d a rd Tel objects and o u r methods. For example, t h e d iscussion appl ication s i m p l e i n heritance mechanism ) . T h e use of these • Allows hierarc hical navigation ( p revious, next, u p ) • Al l ows navigati o n i n c h ronological ord e r ( ne x t u nsee n , what's new) • • e x tensions made it easier to develop appl ications than i t wou ld have been with Tel (or a n y other language ) , a l o n e . As a resu l t, t h ese extensions r(x m t h e basis tor fu ture development tools. Provides a category view that lists topics accord i n g From the begi n n i n g , we k new that the c h oice of to their category an i n terpretive l a ngu age was goi n g to i nvolve tra d e ­ Su pports content- based search or an i n d e x - l i ke oft$ . I n fact, performance, w h i c h w a s o u r m o s t cri tical fi.m ction Newer versions of Web browsers s u pport tl·a mcs, which h ave m u l t i p l e win dow - b rows i n g capa b i l i ties ( a l though the standards in this area are sti l l a bit vague ) . We are u pdati n g our applications to take advantage of these new features. Usability studies guided our decisions as we we r e d esign i n g forms and d i a l og boxes . I t i s l i kely that m a ny pote n ti a l users o f our p ro d u c t are fam i li a r w i t h vVi n d ows- stvlc u s e r i nte r face objects. Because th e earlv Web b;·owscrs ( e . g . , M osaic) were U N fX - based , littl� attention was given to provi d i n g a h u ma n ­ com p u te r i nterface that rese m b l ed the m o re w i d e l y u s e d W i n d ows i n rert:1 ce. However, o u r u sa bi l i ty studies ind icated t h a t m • S ho u l d access c o ntrol be d c ti n c d fo r e\'erv poss i b le access/action tvpe> tvp ica l ly execute o n l y o ne transaction a t a time. • O ne key design decision was ro i n c l u d e a n i nd e x i n g a n d search e n g i n e a s a b a s i c component of t h e pro d ­ uct. A lthough the database is often t h e centra l piece o f a groupware product, a n i n d e x i n g a n d search engine often plays a si m il a r r o l e t()r a WWW s i te . This deve l ­ opment i s compl etely consistent with t h e p h i l osophy of the W vV W- i n formation is l i n ked as needed , not necess a r i l y fo l lowi n g any structure. D a tabase use i s m o r e s u i ta b l e for i n formation objects that h a v e some u n i form ity i n their defi nitions. The basic function of the indexing engine i s to map a set of words to a d o c u m e n t c o n ta i n i n g t hose word s . 72 Di�it.1 l Tec h nical Journ a l Vol . 8 No. 3 S h o u l d t here be arbi trary fl e x i b i l i ty 1 11 d e fi n i ng gro ups ( i n c l u d i n g nesti n g ) > Indexing a n d Search: The Way of the Future ? 1 99 6 On the o t h e r h a n d , users h a v e stated that they do not l i ke products in w h i c h access control operations arc c o m p l e x , especially i n the case of a prod u c t that is s u pposed to help people co l l a bor:nc . I n a majority of sce n a ri os , they argue that very l i ttle access control is need e d . f o r t h i s reaso n , w e tried to stri ke a b a l a n c e between ad m i n istrators' needs and users' prdcrences. Although vve recognize the i mportance of access control , we did not give it precede n ce over prod u c t usabi l i ty. S i nce usabi l i ty was o u r priori ty, a n d the time avai l a b l e to work on it was l i m i ted, we d ivided o u r e fTorts between the future as long as it makes management of access making access control flexibl e a n d choosing defa u l t control policies easier. options t h a t would promote collaborati o n . vVe defined access control tor the whole database Futu re D i rections ( foru m ) , rather than for individual entries and attrib­ u tes of entries. However, some entry-level access con­ To date, we have received encou raging feedback from trol is necessary. For example, it is preferable to l e t only users. Of the ways tha t we can continue to improve the owner ( o r the creator) of a n e n try modify and the Alta Vista Foru m product, we fee l the tollovving delete that entry. As a resu lt, we allowed the group def deserve the h i ghest priority. inition to include entry-specific logical users, rather First, we need to provide better ways to help users than provide a general mechanism for entry-level access deal with i n formation overflow. Although we have contro l . Therefore, a group may contain a member b u i l t ways to tllter and search information into o u r who is the owner of the current entry. D uring access application, further simpli tlcation i s necessary. We control checking, the cu rrent entry's owner is looked are working on smart agents that brin g the relevant up and matched against the c u rrently logge d - i n user. information to the user's fingertips . Instead of letting the administrator defi n e access Second, a n umber of the functions that we provide control for each possi ble incoming access/action, our can be more easily performed on the client machine. framework allows the application defi n ition to group The Java language is the best candidate tor providi n g accesses together into logica l access rights. For exam ­ these functions since it enables us to handle a wide ple, for the discussion application , we defined the fol ­ variety of client platforms. I nitially, we arc looking i n to lowing access rights: using Java to improve certain user interface problems, • Read -Includes all read U RLs ( d i frerent v1ews, whether for a single entry or a l ist of entries) • • • machine to notifY users of new information. Third, synchronous collaboration using video, Conni b u te-lncludes adcling a topic or reply audio, and whiteboard will soon become feasible a n d us Modi r)r-l n c l u des any form of modification or cost effective. I t is important tor deletion together through both sync hronous and asynchro· Moderate-Includes such functions as creating key­ nous methods of collaboration. For example, users words, polling options, controUing n u m ber of levels of replies, and setting certain entries as hidden • such as opening additional windows on the client Administrate-Change access control or other kinds of resource consu mption policies By defi ning these access rights, the administrator only to help bring users should be a ble to use the calendar application to schedu l e a meeting over the I nternet, and Windows should be available to the user a u tomatical ly. Fourth, as the AltaVista Forum software matures, we hope to add to its performance and increase its scalability. As its environment evolves, we are look­ needs to estab lish who can do these five operations, ing i n to ways to bypass the CGI interrace and usc a rather than define nu merous other kinds of opera­ compiled l anguage tor more of the toolkit implemen­ tions. I t is still possible to c hange and add to this tation . We also hope to add support tor l a rge commer­ gro u p of access rights by making sim ple modifications cial databases. to the appl ication definition. Finally, we wi ll continue to add innovative applica­ O u r basic strategy for making access control easy to tions to o u r produ c t . We recently built a prototype of manage is to set up defa u lt policies of access control a customer-support application that keeps track of that apply to as many situations as possible, within rea­ problem reporting. We are looking into other applica­ son. The deEmlt policy is added to the appl ication def­ tions such as project management, group review, and inition. I f the administrator is satistled with the defa u l t survey and decision-support systems. pol icies, t h e n the access control can b e u s e d a s sup­ pl ied . For the discussion application, the defa u l t policy Acknowledgments is the following: We wish to thank the AltaVista Forum development • Read -Al l users, including a nonymous • Contribute-All users, excl u d ing anonymous • Mod i r)r-Owner ( c reator) of e n try and moderators for h is leadership in starting the effort; Ra l p h DeMent, • Moderate-Owner of the forum Bob Travis, David Marques, a nd Rick Fran.kosky, who • Administrate-Owner of the forum To simplir)r implementation , we chose not to a llow nesting of groups. Our design allows tor adding it in and management teams for t heir contri butions to the product. In particular, we wish to thank Peter H u rley have worked with us throughout the l i fe time of the produ c t and with whom we have devel oped a special camaraderie; and D a n Kalikow, who was the fi rst adopter and has cheered us on ever since. Digital Technical Journal Vol . 8 No. 3 1 996 73 References and Notes 1 . D EC �otes is a d i sc u ssion ;l p p l i carion r u nning p r i ma ri h· on VAX svs re m s connected on a [) 1 T n e t network. Sti l l a ,· c r v pop u l a r ro o\ w i r h i n D i gi ta l , it is used r(x c o l l a borat­ i n g o n many top ics , r:m gi n g ri·om p rod u c t developmem, cus to m e r support, and m a rk et i n g to various person;l[ i nt e n:st topics. 2. The prod uct was ori g i tl a l l v c a l l e d Wor k g ro u p Web Foru m . It was su bseq u e n r l y mngcd i n to a l a rger tJ m i \ ,· o r· produ cts a n d t h e prod u c r n a n K b e c a m e Al ta V ista Fo r u m . 3 . F or tnorc i n fo r n1 a tion on th e Wo rld \Vide ·we b Consor­ ri um/M I T La bo r a tor v r()l· Com p u ter S c i e nce Wo rkshop on the World Wide Web and Colbborarion held Scp­ r t: m b e r 1 1 - 1 2 , 1 99 5 , sec h rrp:/jwww.\\' 3 . org/pu b/ W W W/Col l a borati on . 4. O usr er hou t , Tel a n d the Tk 7 ()()//d! ( Read i n g , Mass . : J. Addison - Weslev P u b l is h i n g Com p;l l J I', 1 99 4 ) . S. 6. U 4 .) BSn r u ix l 'm.� J W I I I I I i l li{, . \ fm z ual Nej� ere nee G'u ide ( U n i ,nsirv of ( :., l i r( n n i a , B c r k c lc \ ·, ! 986 l . . \ 0/J. t tr. ' During this rime, ) ;wa wa s sri \ \ o n rh c dra,,·ing board, o r ;lt lcasr nor gene�·a \lv supported b,· Web bro11'Sers. We did c \pccr ro usc Ja,·a ro e n h a nce o u r user i mc r fJ c c 01-er r i m e . 7 . For access t o D i gi t ;l \ 's i n d n i n g .l l l d search c n gi tH.: , visit r h c Al taVista Web si re ar h rr p : //cl l tavista. softwat·e. d i gi tc1 \ .com . Biograph ies Dah Ming Chiu ] ) ;l h i'vl i n g Ch i u ,,·as a c on s u l t i n g e n g i n e e r i n Digita l 's I merncr Software B u s i n ess U ni t- elm(\ r e c h n i ca l leadet· i n de,·clopi n g rhe Alta Vis tel F or u m grmqm c1rc prod uct fo r the I n re m e r . B ctore that proj ect , h e worked r(Jt· rh e Nen,·orks At·c h i rccture G rou p on co n gesti o n c1 11d rlow control , n c r ­ work m o n i toring, n a m e serv i c e , a n d the X . S O O sta n dard . Prc,·ious to r ha r, he worked on p c r rimna n cc modeling and ;mc1 l ysi s nen,·ork p ro toc ol s ;wd gr;1phic1 l workstation de si c:n . D c1 h :\'l i n g is c u r rc nrh· ,lll cl rc h i tecr in tbc I n te rn e t : Sol t t i ons Group �f S u n M i c t os,·s rems, I n c . He recei,·ed a ; P h . D . in c1 pplied ma r hc m cu i cs ( 1 98 0 ) ri-om H c1 n·a rd U n i ­ \ c t·sit\' cmd a B . S c . i n e l ectrical e n g i n e e ri n g I 1 9 7 5 ) fro m rhe l t ; lpcrial Col lege, Lo ndon U n i ,nsit\'. H e h o l ds th ree pare 1 1 ts in th e at·eas of co nge s ti on con rml a 1 1 d 11em·ork m o n i tori n g and i s a coauthor of Netu ·ork Jlo n itoriug 1�\jJ/o i11ed. 74 D i � i l .l l Tcclm ic1l journal Vol . 8 No. 3 1 996 David M. G riffin l )a\ e (; ri ni n J O i n ed D i gi ta l i n 1 98 1 . H e is a p r i n c i p a l sofr­ ,,·are e n gi n e e r in rhe A l raVist,\ C o l l a bora t i o n En gi n ee r i n g G ro u p , \\' here he leads t h e A l ta Vi st.l Forum Tool ki t te cl lll t()r \ cr s i o n 3 . 0 . Da,·e a l s o l ed rhc roo l k i r tum r()r ,.L'rsion 2 . 0 and was r h e primarv designer ;llld i m plen1 c n tc r o f the ( Workgroup 'We b Foru m ) version J . 0 too l kit a n d rhc ;l l t th or o f r h c doc u mcm- s h a ri n g cl pp l i ca r ion for 1 \: r s ion 1 . 0 . Pri or ro t h i s \\'Ork, Dave led th e J ) E C : d n s s e r\ ' LT p roj ­ ect ( 11an o f t he DECnerjOS! prog ra m ) a n d design e d a n d i m p l e m e nted the h i c rarc h i u l c e l l s cmd c e \ 1 - renarn i n g lcK i l i r i cs i n t h e DCE C e l l Direcron Se n i c c . H e has been i ll \ ohcd i n tlK d e ,·c lo pm e n r o f .1 n u m b c r o f d i :,tri b u ted i tl tcmllc:rjo S):.;fcms ( December 1 99 5 ) . J. Rose, a n d T . Spooner, "The Ek ctrom i l,' I'" Proceedings o/ the 13rd Omji:rence of the Societ vfor Tech n ical CiJI I 7 111 1 1 1 l icalion ( M ay 1 996 ) . W. 13ow h i l l , "A 300M Hz CMOS JUSC M icroprocessor," 1/:l-:Fjou mal o/Solid Stale Cirw its ( No1·em bcr 1 9 9 5 ) . C. Bre nc h , " M o d e l e d J n d iYicrks," !U:F lnjuculll '96 Proceedings o/ /)ocumenlo I ion ( f�brua r1· 1 996 ) . K . G e h l e r t c\ lld D . S c i p i o ne , " I n Situ Mon itoring: o f J)rod u c r \Vlll:n, ," Solid Stale Tech nolo,'..!, I ' 1 1\ brch 1 996 ) . !be I 5th A 1 1 1 1 / tUI OJI/fen:nce 0/ the TF.F.F Computa and ) . Grodstc i n , E. A . C :h c\1'1\\', "Tirne,c\ le Analysis for Explicit Rate A l location aided f)es�-..: u ( N o1·e1nber 1 9 95 ) . Conl l l l l t l l icat ions Sociel ies ( f,t\arch l 996 ) . in ATM Ncr�Vorks," 1/:f:.L lnjucom 96· Proceedings o/ Leh m a n , H . H a rkness, and Y. Wata nabe, " l . o!!;ic Decompo�ition D u r i n g Tec h nolo f!0 Mappi ng," /l: F />;IC.JI /u /ernmio nol Co nference 011 OmljJitler­ the J '5th A 1 1 1 11W/ Co n/erence q/ the //:;'/:;£ Computa and Co rm n 1 1 1 1 icalions Societ ies ( March 1 996 ) . P . Gronowski, "A 4 3 3 M H z 6 4 b QuJd - i ssue !USC J . C l e me n t, " l'u lsed -currcnr Duty Cycle Dependence of Solid Stale Circu its Conference ( Fe br u a r)' 1 9 96 ) . M i croprocessor," Oigesl o) Papers, !r:t:F lnlen wliolwl Co nducro�·s," lf:FF I:Iecrron Ueuice Letters ( M a y 1 9 96 ) . P . Gronowsk i , " D vnamic Logic a n d L atc h es Pa rt I I ­ . l'r;Ktical l mp l c m c nta r i o n M e t hods a n d Circ u i t E x a m p l e � Elccrronic License Distri bu ti o n , " c llapter 10 in Jnte­ ( J u n e 1 99 6 ) . the 1-i!l l rlh lntemalional Sr1 1 1posium on 111/(�!.!, mled M e rhodolo).!;l tix PSG and PETEOS f i l rns in a H ighl1· E lccrromi�rc\ti o n - i n du ced Stress Gener:1tion i n A l u m i n u m T Co l l i ns, "PO LYC :ENTER Li cense Syste m : Enabling gmted Netu •od! Mwwge111e111 V Procf:'edin/.;:s o/ Used on rhe Al ph :1 2 1 1 64 , " \- 'LSI Circ u its .�J'/1/jJosiu m E. ;\'et uwf..: , \Ita l lt!,f.!, C:111Ci l l ( London : C ll Jp!llclll & H ;� l l , H ;\ mo n c1 11d H . Wood�Vard, " P rocess C o n trol 1 n l l T;l Lri l·e :Vi u l (i p rocess CVD S1·stcm," Adt ·o u ced Semi­ J S R "\ 0- 4 ! 2 7 1 - S 7 0-il, l 9 9 S ) . conductor . \ Itt n t t/itcl l l l ing Co nfere uce and Win l'sbop T. Col l i n s, "The Wolf a s a Metaphor for Sohware Age n t , " ( !\! me m be r 1 9 'J 5 ) . A . Con n , "Time A ft(mia nce s: The Time Factor i n Diag­ f)el 'ices c h a pter ti1·e i n !Jots wul Internet fleasties ( I nd i a napolis, I nd . Sams . net P u h l i s h i n g , ISBI' l - 575 2 1 - 0 1 6 -9, 1 996 ). nostic l \.1 b i l ity Heuris tics," J-Ju nwn Faclors in C:u lllplll­ in,r,;. O-ff '() 5 Proceeclin,r,;s ( i\IL\y 1 99 S ) . Z. C1·etc1novic, " Per·fmmance Characterization of the Alp he< 2 1 1 64 M i cr oproce ss or Using TP and S P EC \tVo r k ­ loads," J>roceedings (�j'the IF.Hf:. Second !ntenwtion(f/ S) •mpusitun on H(u, h-pelj(Jrnnmce Co mputer A rchilec/l!re ( h bruc\rv 1 9 9 6 ) . M . Desa i , R . Cvijcti c , a n d J . Je n se n , "Sizing o f Cl oc k Con/('reuce (J u m: 1 99 6 ) . M . Desai a n d Y. Ye n , "A S1·stcmatic Tec hniq u e tor Veri t\• ­ i n g Critical P' ll:T:F Transactions 0 1 1 !:lec/twt l 996 ). H. ) a kieiJ, "Performance Visua l i zation of a Distri buted Sys 1 c rn : A Case Studv, " Co mp u ter ( N o 1-e rn ber l 995 ) . R . Kelsey, " H Jd h xes, Change Speci ri c a tions c1 nd Li n­ gui�ric C : on s r r:1 i n ts or\ l'roblem D i c1gnosis," Sojil l 'ore �:·n,f.!, ineen·l l,f.!, i\'olf!S ( M arch 1 996 ) . Ke rn , "The C h icke n i s l rwoh·ed, .But the Pig i s Com ­ m i t te d - B u i l d i n g Com m i tment Throu gh C Jscad i n g Distri b u tion Ne rll'orks ror H i g h Performance CPU Digit.l l Tcch nic11 )ou rn:ll Hucmg, j b r ice l l i , � . Kh a l i l , a n d R. Rio�, "An c Accurate c.;,, te Lc mnh L \ traction M e thod for S u b - 4 u arrn C : . - L. j. Ch ip�," J>mceedin,f.!,S o/ the 33rd Des(u,n A u tomation 76 L. El l i o rr, R . S h u m a n , S l '/ 1/f}().\ittlll Pmceedings ( Apri l 1 99 5 ) . Kno\\'ledge- based Performance C. Bouti n , " h·om M�1 n a g er Srmpos i1 1 111 (Apri 1 1 99 6 ) . Tu n�stcn Tu ni ng A(h-isor," (Jf(;CJ"i. Proceedings o/ the 2 lsi �\ a n d C :nritication , " ret/: 34th A n n ual Spring Reliu hilil l ' 1996 · rcc\11\s," (>uolity Prugress ( October 1 9 9 5 ) . � . Klu l i l , F a rice l l i , J n d H uang, "Two - d i mensional Dopc1 1 1 t Prolili n g ofSubrnicron MOSFETs Usi n g N o n l in<.:'! <)6 ( J u ne 1 996 ) . "C B . i'vl i rm J n , t()r Thernnlh· Lo:�ded W . Nagorski , W. M c G e e , E . Picciol i , and L Gair, "A u ra ­ marie Test C : h i p Documcnrarion S y n th es i s , " l 'rncel.'din,r.;s uj' the I 99o 1/J;F luternational Con)c'rc'nce on Micro­ electml l ic 'fi'st Structu res ( March A . Villani and H . Ngu,·cn, "Corre lation of rhe Mechanical ro the ThcrnlJI S t re n g th of Ceramic Packages," A merfC(/ 1 / Cera m ic Societ ) ' Ti·m,soclions: 1-lrhrid Microcleclr o n ic Malt>rials ( �o,·embcr 1 994 ) . \•V. Za ha,· i , " M o d e l i n g the Pcrt(mnance Budget-A Cast Study," Cl fG'95. Pmceedinp,s of !be 2 1st A n n uc!l Co n ­ ferencej (Jr rlw Neso1 1 rcl! M a na,r.;emenl a n d Pe1jcm1 1 a n c e EL•aluation ojLnterprisc Co mp u ting Srstems ( Decem ber 1 99 5 ) . 1 99 6 ) . L . N o o1 c k J n d M . Kantrowitz, " I 'm Done S i m u l ;uing; No11· Wh,ul Verification Co,·erage ArLl l vs i s and C :or­ Checking of the DFCchip 2 l \ 64 A l plu l·ecnh.:ss M ic:roprocessor," Proceedings of the 33rd /)es!gn A uti!II ICII ion Co nfe re n ce ( J une 1 996 ). 0. 1\;l iJl,l b i , " A(bpti,·e Absorbi n g BoundarY C o n di ti o n s in Fi n i te - d i ftl: renc e T i m e Dom a i n Applio tions t(>r E M C Si m u Luions," !Fl:'!//i"(I /'ISactioJIS u 1 1 f:'lectru J J WP, I ICtic C(m!jXIIihilil) ( No,·ember 1 995 ) . · 0 . lbmahi ct a \ . , " Dynamic An;llvsis of V ·rr-,lllsmission Li nes," Conference Proceedings oj' lhe I 2lh A n n ual Ncl 'ieu· ofPm,r.;rcss in Applied Computolionol Flec/mJJ /!.I,r.;nel ics ( Ma r c h 1 99 6 ) . S . Rege, "A Distri buted Svstem Cl i e nt/S e r v e r Arc h i tecture t(lr .l n teracri,·e M u l timedia Applications," C0. \ 11'96 Digest q/l'ojJeJ :,· · i lsi IFFI:' Computer Society In len w I iono I Confi·rence ( rcbru ar\' I 996 ) . R . Ri os , � - A ro r a, C : . - L . H uang, :--.J . Kl d i l , ) . 1-'arice l l i , a n d L. Gr u b e r, "A Physical Comp;lCt .v! OSJ-'ET M o de l , l 11 c l u d i n g Quantum 1Vlechanical E ffc crs , for Statistical C i 1· c u i t D e s if' n Appl ications," ff.F!:/Eiee!ron /)el 'fces 'f('ch u iwl ! J1:� cst ( Decem ber 1 99 5 ) . . K. Rose l l e , " E st i rn :� r i n g Crossta l k i n M u l ticonductor fj nes," !EhF Tt"(l /1Sclclious on Components. Tra nsmission Pod�o.r.; ing and Mmwfac/urinp, 7 e ' ch7! olog ) ' f'o rl /3: A dm u ccd Packa,r.; in,r.; � - R u bi n , ( M ay 1 996 ) . " Efticienr Instruction Sc h ed u l i n g Usi ng h n i te SrJte A u ronu t,1 , " Procecdinp,s o/ lhi! 28/h A n n uol lu!crurtlioual .S)'I IIposi/ 1 111 0 1 1 .l licmorch itecluu· ( �m·c n 1 ber l 99S ) . C. S m i th c1 11d T . Val l one, "Men rming: P ro , · i d i n g Profcs­ _, ion:�l and Organi zational Benetirs," f'mu'('(lin,r.;s u/ tlw 1,-!rd Cnuference ol tbe Societ ) 'ji!l· 'lh'h u icol Com l ln l l l icotion ( M a1· 1 99 6 ) . N . S u l livan, S . Dass, G . P o ll a rd , W . Jones, ,1nd ·r. Linds:�y, "A C :o111pc11·ison ofSratc-of- the-Arr D U V I .c:: nses," Pro­ Ci'l'rlin,r.;s o/ the Society qjPhoto-Opt leal Ins! m men/a­ t io n l:n,!!, ineel:,· ( l'cbruarv 1 99 5 ) . H . kq!,a n , " Distributed Performance Monitor U s i n g S N M I' V 2 , " I!:F!:/!HP Net/l 'ork Opemtions u u d M(.//1ll,l'.I'IIWIIt S i 'l l lfJOSi/ 1 111 ( April I 994 ) . M . Tsu k , ''I'D-T D Anah-sis o f Electromagnetic R.c1diation ti·o1n Mod uks-on- Backplane Con ti g. u r.1ti o n s , " ll:l:F 'f i w !soelions on Electronwgnetic Coll lp(ltfhilil l ' (August 1 99 5 ) . Digi tJI Tc( h n i (J I j o u nL\1 Vol . 8 N o . 3 1 9') 6 77 I Recent Digita l U . S. Patents Eq u i pment Corpora tio n . Titles and names s u p p l ied The r( >llowi ng patents were rece n tly issued to D igital to u s b�' the U . S . Patent and Trademark: Ofnce arc reprod uced as the\' a p pear on the origi n a l published pate n t . 1)3 5 3 )\ 0 0 5 , 3 7 1 ,807 5,37 1 ,822 5 ,3 7 1 ,868 5 , 3 7 1 ,870 M . S. Lc11·is, L. A . Tn:scder, R . M. Tu slcr, ;llld C . Slllcb M o d u b r E n c l osure tor Electronic Eq ui pment r:. H or\\ i tz and E . Thomson Rcgistn Method and Appar·atus for 'TC\t C: bssitlcation 1\' . Ka n n a n ;md 1\tl . S . C. 1'. Ko nig, H. S. Y:lng, and vV. H a11 c I'. M . Goodll'in, D . Smelser, and Method of Pack.l !,'.ing and Asse m b l i n g Opto-e lectron i c l nrcgran:d C i rc u i ts M e thod and App;li· fiber Optic ;l Track-and­ 5 , 3 8 2 ,8 3 1 B . Lee, E . At:� ko v, and J . Clement I nte grated Circ u it l'vletal Film Interconnect H a1·i n g 5 ,3 8 3 ,096 M . C. Benson a n d L. M . M a zzon e l/0 5 ,384,779 M . Patrick a n d J . A. D a l y State Machines for Con figura ti o n of a Com m u nications E nh a nc ed Resistance to Ele c rro m ig ration E x pan sio n Box Network 5,385,289 C. Bloc h, P. McKi nley, a n d R . Ra ng an a tha n 5 , 3 8 5 ,630 A. P h i l i possian, H . So le i ma n i , and B. Doyle Embedded Featu res for Registration Measurement i n E l e ct ro n i c s M a n u tac r u rin g Proc ess to r I n c rea s ing Sacrificial Oxide Erc h lbtc to Red u c e Field Oxide Loss 5,386,5 1 4 V. Boacn , R. Lary, B . Rubinson, D . Th i e l , C . Van l ngcn, W. Warson, R . Willard, and Q ue ue Ap para t u s a n d Mcch�nics tor I n t e rfa ce Arch itecture a C o m m u n i cat i ons E . A . G a rd ner 5 ,386,5 2 3 5 ,386,524 Ad d ress i n g Sc h e me for A ccess ing a Portion of ,l N. A . Croo k , M. ]. Se a m an, a n d D . L . A . B rash La r ge M e mory Space V. Bo a en , R. La r y, B. Rubinson, D. Thiel , S y s re m for Accessing Information in a Data P mc e ssi ng C . Van I nge n, W. Warso n , and R . W i l l ard Sy s re m J. C. K. Lee, M . C as tro , F. Tu ng , C. Lee, S e q u e nti a l M u l ti l a)'t:r Process for Using f l u orin a ted a n d A. A h mad Hydrocarbons as a Dielectric 5 ,387,530 A . P h ili p ossi a n and B. Doyle Th rc sho l. d Opt i m i z at i on For SOl Tran sistors t h rou g h Usc 5,388 ,099 N. Poole 5 ,387,495 5 ,3 8 8 , 2 2 2 of N e g a t i v e Ch:trge in the Gate O x i de Ba ckpl an e Wiring to r H ub in Packer Data Comm Lt ni c a ti ons Svstem L . A . P. C h isv i n , J. F. Ranta l a , J . K. Groo m s , M e m ory Subsystem I nput Queue and D. W. Hartwell 5 ,388,224 B . JY!Jskas Processor Identification M eclunism for a Mult i processor S y s te m 5,388,247 P. Goodwi n , K. T h a l l er, a n d B. M a s k a s H isrorv Bu ffer Contml to Reduce Unnecessarv AJ i oca ri ons i n a M emo ry Srream Bufkr 5,388,263 R. K. Peterson , ] . R . 5 ,389,757 E. Soul ierc 5 ,390, 1 73 E l l i s , a nd C. G . Nv l and e r Pr o c ess ors Ela s rom c r ic Ke y S wi tc h Act u a to r B. Spinncv, R . Si m c oe , G. Varchese, and R. . T h o m a s 5 ,390,286 Proced u re State Descri p tor S yste m tor Digital D a ra K. K. Ra m a kri s h n a n Packer Format in Hub tor Packet Data C o m m u n i c at io n s S y ste m Reti cular Discri m i nation N e twork for Spe c i t),i n g Re a l - time Co n di t io n s 5,390,299 S . L. Rege, K . K . Ramakri s h n a n , and D. A. Ga g ne S ystem tor Using Th re e Diffe rent Methods ro Report B u ffe r Memory Occupancv I n formation Regard ing F u II ness- rel a ted :llld/or Packer Discard - rclarcd I n formation 5 ,390,302 5 ,390,3 1 8 ]. Joh nson , M . Ho we l l , and C. W h itak er K. K. Ra m a k ri s h n an and P. Bi sw as Transaction Control C ac h e A rra n ge me n t for File System in Digir�l D�ta P r o cess ing Sys te m Method tor O n - l i ne Re org ani za t i on ofrhc D:tta o n a RAI D -4 or IW D - 5 A1TJV in r h e Absence of One Disk a n d r h e O n -line Resrorarion of a Rep b c em e nt D is k 5 ,390,327 C. Lubbers and D . Thiel 5 ,392 , 2 1 9 S . Birc h , G. Gavrel, and Z . Memon Dcte r m i nation of l nterconm:ct S t re ss Test Current 5 ,394 , 1 4 3 J. M u 1-ra y and G . A n r os h en k ov R u n - l en gt h C o m p re ss io n of I n dex Kcvs 5 ,394,347 R. !(i ra, S. Tremblay, and T. Lynch 5 ,394,401 M . Patrick and 5 ,394,5 2 9 ]. J. A . Daly Brown , J . M ey e r, and S . Persels Method and Apparatus tor G e n e ra tin g Tesrs for Stru ctures E x pressed as E xten de d Finite Srate Machines Arrangement tor a Token Ri ng Com m u nications Network Branch Prediction Unit tor High -performance Processor Vol . 8 No. 3 1 996 79 Call for Papers Networl( Products and Technologies The Digital Techuicctl .fou rnal seeks technical papers i n a l l areas of networki ng tec hnol ogy tor an issue to be pu blished in the fal l of 1 997. Digital's engin eers and ind ustry p:trtners interested i n participating in the special issue shou ld send topics and brief abstracts ( 1 0 0 words) by February 10, 1 9 97, to Jane B l ake , Managing Editor Dil; itctl Tech n ical]ou rna! Digital Equipment Corporation 5 0 Nagog Park, AK0 2 - 3/B3 Acton, MA 0 1 72 0-9843 Email : jane.bl ake@ljo.dec . com 508-486-25 44 Notice of the topics accepted will be sent to all authors by February The man uscri pt-sub mission d at e t(x accepted topics is M:ty 30, 1 997. 2 8 , 1 997. for i n ti: mn a ti o n on topics published in the Jo u rnal. the audience, writing guide­ lines, and the peer-review process, see Imp :/jwww.d i gital.com/i ntC"J/dtj/ dtj -guide. htm or contact the Managing Ed itor at the add ress above . ISSN 0898 - 9 0 1 X Printed i n U .S.A. E C-N72 8 5 - 1 8j96 12 1 4 2 l . S Copyright © Digital Equipment Corporation

  • Source Exif Data:
    File Type                       : PDF
    File Type Extension             : pdf
    MIME Type                       : application/pdf
    PDF Version                     : 1.6
    Linearized                      : Yes
    Has XFA                         : No
    XMP Toolkit                     : Adobe XMP Core 5.2-c001 63.139439, 2010/09/27-13:37:26
    Create Date                     : 2006:04:08 11:14:49+01:00
    Creator Tool                    : Adobe Acrobat 7.05
    Modify Date                     : 2013:01:11 07:12:16Z
    Metadata Date                   : 2013:01:11 07:12:16Z
    Producer                        : Adobe Acrobat 10.1.4 Paper Capture Plug-in with ClearScan
    Format                          : application/pdf
    Title                           : Digital Technical Journal, Volume 8, Number 3: Internet Protocol V6; Preservation of Historical Computer Systems; Fortran for Parallel Computing; Server Performance Evaluation and Optimization; Internet Collaboration Software
    Creator                         : 
    Document ID                     : uuid:64e396d1-e895-4d76-b276-63ed78e443e2
    Instance ID                     : uuid:49068ed2-7b23-4807-a2e7-339ec3d39f69
    Page Layout                     : SinglePage
    Page Mode                       : UseOutlines
    Page Count                      : 83
    
    EXIF Metadata provided by
    EXIF.tools

    Navigation menu